What are volumetric displays and how can you develop for them? I had the chance to speak with Albert Hwang, one of the gurus at the Looking Glass Factory, about that very topic! The Looking Glass team make commercially available volumetric displays that have huge potential. Their latest one is the “Volume” and has a beta program running right now. It is a really neat concept!
The Looking Glass Factory’s volumetric display in action!
What are volumetric displays?
Albert starts by taking a step back — “any display technology’s purpose is to present information. Your computer monitor, for example, converts the data of 2D things (text, images, video) into pixels arranged on a 2D plane. Conventionally, when you display 3D content on a 2D surface, you are displaying a projection of 3D space — similar to how a camera converts a 3D space into a 2D space.”
So where do volumetric displays come in?
“Volumetric displays present 3D information in real 3D space. 3D info could be 3D models, topographical data, volumetric video, 3D video game content, medical data, etc… and by real 3D space, I mean to say that this tech projects this 3D data into 3D volumes of space.” – Albert Hwang
The concept of volumetric displays isn’t an entirely new one it appears! According to Albert, it has been around for many years in a whole range of different forms —
“The techniques used to achieve this are as inventive and as weird as you may imagine. A handful of techniques include LED arrays, novel projection mapping, moving (swept or spun) LEDs / projection media / 2D displays, laser-generated plasma.” – Albert Hwang
Conventionally, Albert says that volumetric displays are typically cost around $10,000 – $100,000 or more! That meant that the technology up until this point has usually been confined to universities, laboratories and large institutions with the funds to justify a purchase of that size. The Looking Glass team are working towards bringing this price down and have done pretty well so far. Their current volumetric display in beta, the Volume, has an MSRP of US$2099, and was on a limited pre-order for US$1199 (recently sold out but you can still get in touch with them if you want to be a part of the beta program).
Excitingly, the price of this technology is ready to drop as the price of projector technology falls. Their volumetric display technology can also scale up and down really nicely:
“One of the cool things about our technique for volumetric display is that it has no moving parts and is projector-based. This means it scales up and down in a more cost-effective way. It also means that as projectors get cheaper, so does the core component of our tech.” – Albert Hwang
Overall, the goal of volumetric displays is the same as augmented reality — to project digital 3D content into real 3D space. Albert points out that both approaches reach this goal, just with different pros and cons, including —
Simultaneous sharing of the experience
Volumetric displays let everyone around see the 3D space you’re looking at, in 3D. As Albert points out, “Volumetric displays require no headset and the content can be seen by many people simultaneously […] people simply walking by will be curious and be more likely to engage, instead of having to wait in line.”
3D modelling with the Looking Glass Factory volumetric display!
It is more natural and social
It also brings a more natural and social way to share the experience than you’d have with an augmented reality headset. Albert says, “Seeing things as they appear in real 3D space is a more natural way to socially engage with objects, as opposed to everybody putting on the same gear and meeting on the “other side”, so to speak, with AR.”
It is limited to a set area of 3D space
Albert points out that the space available for you to augment objects into 3D space is more limited in volumetric displays —
“Unlike AR, which allows you to paint in the full periphery of the user, volumetric displays can only paint inside the space that the display occupies. Additionally, volumetric display tech is much younger than AR, which typically leads to either lower visual resolution or a higher price point.”
Volumetric displays provide x-ray vision
Volumetric displays have a decidedly x-ray feel to them. This can be a pro if developers and designers build with this in mind, however, if not, it can be a con —
“Volumetric displays provide x-ray vision. Remember, these displays are viewer-agnostic, so objects do not occlude other objects. Experiences that do not consider this in the design will likely suffer from visual information overload, while experiences that properly account for this result in totally stunning / beautiful experiences.”
Reduces eye strain
Unlike VR displays and regular computer monitors, volumetric displays just might be a bit better for your eyes! Albert explains —
“With 2D screens and HMDs, your eyes are put under strain because they are always focusing on the same distance. HMDs do this neat trick where they make things appear closer or further with stereo vision, but each pupil is still focusing on the image right in front of your face.
When looking at a volumetric display, your eyes are perceiving light as it would in real 3D space — your pupil muscles squeeze as they would when looking at a real 3D object.
Anecdotally, perhaps a more compelling reason that this is good for the eyes is that volumetric content simply exists in real 3Dspace — it belongs to the real 3D environment that contextualizes it. So, unlike when looking at a screen, this viewing experience encourages the eye to naturally wander across, over and under the volumetric content (just like how you engage with real objects).”
Works really well with existing 3D input devices
Controlling the Volume with a Leap Motion
It looks like volumetric displays like the Volume display from the Looking Glass team work really well with 3D input devices such as the Leap Motion, Kinect and Structure. The Leap Motion has worked really well so far in one demo they’ve shown the public so far:
“Our most crowd-pleasing demo is probably a Leap Motion demo where you use your hand to pick a flower. To produce this demo, we simply placed our technology on top of a Leap Motion SDK example project. We made a few other minor adjustments before publishing. We’ve even had the opportunity to share the work with the Leap team and they’re ecstatic with the results, as it shows their tech in a really good light.”
Albert believes that it has been hard to visualise current 3D input devices’ capabilities until now. VR/AR and volumetric displays can finally show off their capabilities!
“My theory about the major challenges facing 3D input devices (Leap, Kinect, 3D mice…) is that they can collect 3D data, but then that data is stuck behind a 2D screen. VR/AR and volumetric display technologies change this by providing a 3D output capability.” – Albert Hwang
Join the hunt for the ideal uses for the tech
Like so many areas of emerging technology, the ideal use for volumetric displays hasn’t been discovered just yet. We’ve got this brilliant technology and now we’ve just gotta work out what it can do best! Volumetric displays are one of those technologies that really does feel like it is moving us into a futuristic sci-fi future — so I can’t wait to see what developers come up with! When I asked Albert about what the ideal uses for the technology were, he hinted at the potential of volumetric video and using it as a tool for 3D development:
“To be completely honest, we don’t have a solid answer on this yet, although we do have a handful of theories. Some of the things we’re exploring at Looking Glass that have me personally excited are volumetric video and using this as a tool for 3D development.”
What is volumetric video?
Volumetric displays are a pretty foreign concept to many, so the idea of volumetric video is an entirely new concept that might throw people too! Albert explains,
“Volumetric video is video that is obtained from a Kinect / Structure / similar 3D sensor. Each pixel in this video has 4 channels of data: RGB and D, which is depth. Volumetric data like this really needs a volumetric display to shine. Projecting it onto a 2D screen makes it lose its sense of physical volume, although it allows everybody in the room to talk about it as if we were sharing at YouTube video. Watching this content in VR / AR means only one person can watch at a time and requires gearing up.”
Here is a video showing the Volume volumetric display. Volumetric video would have the depth that the moving horse has below:
3D development
When it comes to 3D development, we’ve already got 3D and VR development applications, however, Albert points out that the displays for developing in these aren’t quite ideal. Volumetric displays like their Volume volumetric display just might be the solution —
“I’m also excited about what Volume could mean for 3D development. I remember the first day I was developing in VR, I developed a rash on my cheekbones from continually equipping and removing the HMD. The first day I worked on content for a volumetric display, I was delighted at how it just acted like another monitor but had volumetric properties to it. I’m interested to see if volumetric displays can accelerate 3D development, VR or otherwise, like modelling or animation.”
How do you develop for a volumetric display?
The question on every developer’s mind right now is likely to be “that’s all sounding pretty exciting, but does that mean I’m going to have to learn a whole new application framework or programming language or some other madness just to be able to develop for yet another platform?” Luckily for us, developing for the Volume volumetric display works via the Hypercube Unity SDK! This means as long as you’re comfortable with Unity (or are keen to learn), you can get started working with their volumetric display! If you don’t have one yourself, they have an emulator too —
“All you do is drop in a Hypercube prefab — which is an outline of an empty box — and anything inside that box gets rendered to the volumetric display. It runs during execution as well as during edit, meaning you can manipulate 3D objects in Unity and see your edits in live, real 3D space.”
Just like with many areas of emerging tech, developers can take advantage of a huge reduction in the complexity of getting into this tech thanks to the Unity SDK. As difficult as it would have been for their team to build the Volume volumetric display, Albert points out that building volumetric applications can be just as complicated —
“To be candid and to risk being overly enthusiastic, I’ve been working in volumetrics for almost 10 years now, and the software is typically equally as challenging as the hardware. Hypercube takes the cake on easiest way to build volumetric content, hands down.” – Albert Hwang
Their team have the Unity SDK available on their website, as well as a video tutorial:
Tips for designing for volumetric displays
The industry is young and everyone is still trying out different ideas to see what works — that includes working out best practices for designing volumetric applications. However, Albert had a few tips that the team has worked out so far in their experiments:
As mentioned earlier, 3D input devices can finally look good!
Use theatrical lighting theory and techniques to make things feel more three-dimensional — saturated lighting from dramatic angles.
Stage your content like a stage or tabletop, not like a picture.
Remember people have x-ray vision in volumetric displays, so be selective with what you display.
All of these are topics that the team are looking to explore in more detail on their relatively new blog as they experiment with the tech. If you’d like to add your own tips to this list, feel free to get in touch with Albert and the team, they’re keen for more developers to get involved and build up more knowledge around best practices and what works best.
Interacting with a volumetric display
As an example of areas the Looking Glass team are exploring, there is the question of how best to interact with a volumetric display. Albert and the team are looking into using the Leap Motion to provide a generalised 3D navigation system for their Volume displays. More information on their progress in this space will appear on their blog. At the moment, the Volume also comes with a touch screen on the front plate that is accessible through their Unity SDK. The Looking Glass team also use game controllers and occasionally keyboard controls to navigate their volumetric displays.
“In terms of interaction design, designing for Volume is fresh and exciting with lots of mind-bendingly cool problems to work out. For example, in a 2D space, moving in any direction with keys is accessible with four keys (arrows or WASD) or a mouse. Not quite as simple and intuitive in 3D space. Should you imagine 3D as layered 2D or something else entirely?” – Albert Hwang
There is so much still to explore with this new technology. Even things like GUI placement and providing visual feedback is proving to be a challenge as the medium has not been explored before.
The possibilities are exciting!
Albert and his team recently released the video below showing an experiment where they use a volumetric display to explore a virtual world without needing to put on a VR headset to see it. This shows there are some very very cool possibilities out there for this tech! Mind blowing stuff!
Getting involved
If you like the look of this technology and want to get involved, ask questions, share your thoughts… etc, the team have a Slack channel you can join!
Want to see it for yourself?
If you are around Brooklyn, you should definitely visit their office and check out the technology. Albert and the team have an open policy for anyone to come visit:
“Unfortunately, it’s really difficult to get a sense of how this tech feels without actually being in a room with one. As such, we have a pretty open policy to whoever wants to come visit. Drop us an email if you want to come through. Additionally, we host bi-weekly meetups where you can come and just ask questions or hook in with your laptop and build some stuff.”
If you aren’t around Brooklyn, there are often volumetric displays travelling around with the Looking Glass team and their partners. Join their mailing list to keep up with the latest on where you might be able to try one!
Keeping up with it all
That mailing list mentioned earlier also will have updates including insights the team uncover, as well as notifications about updates to their software. Sign up and keep up with it all if you are really keen!
I’d like to finish with a big thank you to Albert Hwang and the Looking Glass team for providing insights and images that helped form this article! You can check out more on their tech on the Looking Glass Factory website, as well as that mailing list of theirs!
The post What is a volumetric display? appeared first on Dev Diner.