The AR/VR market is a bit of a mess.
On the AR side, Google nearly killed the effort with Google Glass. On the VR side, folks chasing price over quality and a lack of content have made VR nearly a non-starter.
AR has since shifted to the professional market and is doing OK, while VR struggles on the consumer side and is increasingly used for training on the pro side. But the concept of “mixed” reality was never truly met by either effort. VR rigs typically require reality to be prerendered and not live, and while AR rigs work with reality, their rendered images appear as ghosts largely unable to occlude the objects behind them, spoiling the effect.
Mixed reality promises the best of both worlds, a seamless visual tapestry of what is real and what is rendered. To do this requires a solution that can, in real time, scan and realistically instrument (if not render) the world around the viewer and inject realistically the rendered images.
This is what the new Varjo XR-1 does and it now comes closest to what mixed reality promises.
The Varjo XR-1 headset looks like a high-end VR set, much like the Varjo VR-1 but with two very high-resolution cameras in front. These cameras capture a high-resolution image and then the rendered image is woven into the video feed in real time, properly occluding the items behind it so the presentation is near seamless. The high-resolution screens in the XR-1 headset then display the result in a visually accurate fashion so the user sees the resulting rendered image as their new reality.
This effort was undertaken initially with Volvo who wanted a way for someone to go on a test drive in a car that either didn’t exist (or, more likely, might not be on the local Volvo car lot). Thus, the latency had to be exceedingly low so the user not only didn’t get motion sick from the car moving out of sync with the image, but so the driver could use the headset while driving.
As you’d expect, performance requirements are high, with specifications on either an Intel or AMD 7 level processor and either a NVIDIA GeForce RTX 2018 or Quadro RTX 6000 graphics card. The device also uses Thunderbolt 3 for what must be impressive data throughput to make this all work.