Meta says his ultimate goal with his VR hardware is to make a comfortable, compact headset with a visual purpose that is “indistinguishable from reality.” Today, the company revealed its latest VR headset prototypes that it says represent steps toward that goal.

Meta has made no secret that it is pouring tens of billions of dollars into its XR efforts, much of it going into long-term R&D through its Reality Labs Research division. Apparently, in an effort to shed a little light on what that money is really achieving, the company invited a group of press to sit down and take a look at its latest VR hardware R&D accomplishments.

get to the bar

To kick things off, Meta CEO Mark Zuckerberg spoke alongside Reality Labs Chief Scientist Michael Abrash to explain that the company’s ultimate goal is to build VR hardware that meets all the visual requirements to be accepted. as “real” by your visual system.

Today’s VR headsets are impressively immersive, but there’s still no question that what you’re looking at is, well…virtual.

Within Meta’s Reality Labs Research division, the company uses the term ‘Visual Turing Test’ to represent the bar that must be reached to convince your visual system that what is inside the viewer is Really real. The concept is borrowed from a similar concept denoting the point at which a human can tell the difference between another human and an artificial intelligence.

For a headset to completely convince your visual system that what’s inside the headset is Really real, Meta says you need a headset that can pass that “visual Turing test.”

four challenges

Zuckerberg and Abrash outlined what they see as four key visual challenges that VR headsets must solve before the visual Turing test can be passed: varifocal, distortion, retinal resolution, and HDR.

Briefly, this is what they mean:

• Varifocal: the ability to focus at arbitrary depths of the virtual scene, with both essential focusing functions of the eyes (vergence and accommodation)
• Distortion: Lenses inherently distort the light that passes through them, often creating artifacts such as color separation and swimming pupils that make the existence of the lens obvious.
• Retina Resolution: having sufficient screen resolution to match or exceed the resolving power of the human eye, so that there is no evidence of underlying pixels
• HDR – Also known as High Dynamic Range, which describes the range of darkness and brightness that we experience in the real world (which almost no display today can properly emulate).

The Reality Labs display systems research team has created prototypes that function as proofs of concept for possible solutions to these challenges.

varifocal

To deal with varifocal, the team developed a series of prototypes they called ‘Half Dome’. In that series, the company first explored a varifocal design that used a screen that moved mechanically to change the distance between the screen and the lens, thereby changing the focal depth of the image. The team later switched to solid-state electronics, resulting in a significantly more compact, reliable and quiet varifocal lens. We’ve covered the Half Dome prototypes in greater detail here if you’d like to know more.

Virtual Reality… For Glasses

As for distortion, Abrash explained that experimenting with lens designs and distortion correction algorithms that are specific to those lens designs is a cumbersome process. Novelty glasses can’t be made quickly, he said, and once they’re made, they still need to be carefully integrated into a headset.

To allow the display systems research team to work on the problem more quickly, the team created a “distortion simulator,” which actually emulates a VR headset using a 3DTV and simulates lenses (and their corresponding distortion correction algorithms). ) in the software.

Doing so allowed the team to iterate on the problem more quickly, where the key challenge is dynamically correcting lens distortions as the eye moves, rather than simply correcting what is seen when the eye looks at the immediate center of the lens. the lens.

retinal resolution

On the retina resolution front, Meta revealed a never-before-seen prototype headset called Butterscotch, which the company says achieves a retina resolution of 60 pixels per degree, allowing for 20/20 vision. To do so, they used extremely pixel-dense displays and reduced the field of view, to concentrate the pixels into a smaller area, to about half the size of the Quest 2. The company says it also developed a “hybrid lens.” ” that would “completely solve” the increased resolution, and shared through-the-lens comparisons between the original Rift, Quest 2, and the prototype Butterscotch.

While there are already sights that offer retina resolution, such as Varjo’s VR-3 riflescopes, only a small area in the middle of the view (27° × 27°) hits the 60 PPD mark… that area drops to 30 PPD or less. Apparently, Meta’s Butterscotch prototype has 60 PPD across the entire field of view, though the company didn’t explain how much resolution drops toward the edges of the lens.

Continue on page 2: High Dynamic Range, Downsizing »