What Is Virtual Reality Optics?

Definition of Virtual Reality Optics

Virtual reality (VR) optics comprises the specialized components within a VR system that create an immersive visual experience for the user. Examples include:

• Cameras that capture raw data for scene simulation
• Fiber optics used in gloves and clothing to send and receive data
• Head-mounted displays (HMDs) that generate 3D perception
• Immersive and semi-immersive projection displays
• Sensors that track the motion of the user and their eyes.

Currently, the virtual reality optics of most interest are head-mounted displays (HMDs), which are also known as near-eye displays.

For virtual reality to work, there must be an optical system in an HMD that will project an image on a display in front of your eyes.

In this optical system, an HMD includes light sources (display), receivers (eyes), and optical elements (lenses).

• The light sources in an HMD are microdisplays, such as organic light emitting diodes (OLED) or liquid crystal displays (LCD). A binocular HMD typically has two displays that provide separate images for each eye and generate 3D perception through stereoscopy. In a holographic HMD, the light source is modulated coherent light from a spatial light modulator (SLM).
• The receivers in HMDs are the eyes of the user.
• The optical elements collect light from the source and generate renderings of a 3D virtual world. An ideal VR HMD must be able to provide a high-resolution image within a large field of view (FOV) while supporting accommodation cues for 3D perception and have a large eyebox (exit pupil) with a compact form factor [2].

Figure 1. Structure of VR HMD. Source: https://news.skhynix.com/the-world-shaped-by-semiconductors-virtual-reality-in-glasses/. [3]

One of the most important requirements for HMDs is good ergonomic design, meaning the headset is comfortable to wear and view for prolonged usage. To be comfortable to wear, the headset should be compact and lightweight. Ideally, the weight and size should be no more than a pair of eyeglasses. To be comfortable to view, the headset should provide appropriate viewpoints based on the user’s head position and gaze point. The headset should also have adequate eye clearance, a large enough pupil size to allow for natural eye movement, appropriate interpupillary distance (IPD), and low divergence and dipvergence.

The key optical design constraints for HMDs are pupil (eye-box) size, eye clearance, divergence, dipvergence, and IPD (see Figure 2).

Field of Vision (FOV) Is Important

An important design goal for VR HMDs is to match the image characteristics of the human visual system. The FOV of the human eye is roughly 120 degrees vertically and almost 360 degrees horizontally considering eye rotation and head movements. The binocular FOV within which an object is visible to both eyes is about 114 degrees [5].

Figure 3. Field of vision diagram. Source: https://www.electrooptics.com/analysis-opinion/meeting-optical-design-challenges-mixed-reality. [6]

Considering Aberrations in the Design

The effects of aberrations on image quality in HMDs are similar to those in other optical systems. Aberrations such as axial chromatic aberration, spherical aberration, coma, astigmatism, and field curvature introduce blur. Aberrations such as distortion, coma, and lateral chromatic aberration induce warping. Aberration control is important in the design of VR HMD optics.

Other Factors in HMD Design

Advances in HMD design take advantage of aspheric surfaces, diffractive optical elements (DOEs), holographic optical elements (HOEs), tunable lenses, and plastic optics.

• Aspheric surfaces help to control lens aberrations and reduce the number of elements.
• DOEs provide interesting dispersion properties with negative chromatic dispersion in positive lenses.
• HOEs have a small form factor and can function like a beam splitter.
• Tunable lenses can extend the depth range.
• Plastic components are low cost and light weight.

Although you can improve the VR device's eye-box size and FOV using these advanced technologies, it often compromises the form factor. To alleviate this problem, new directions for development include eye-tracking integrated HMDs, multi/varifocal displays, occlusion displays, holographic displays, and light field displays.

Optical design software is an important tool for designing VR optics. Quality VR optical system design requires multiple types of software:

• The optical engineer needs software to create and optimize the imaging system, analyze straylight in the optical path, and design diffractive optical elements.
• The mechanical engineer needs a CAD package to draw the system layout and accomplish thermal and structural analysis.
• The electrical engineer may need software to track eye motion and send the signal to the optical system.

Keysight provides a complete set of tools to simulate AR/VR devices.

Here is a workflow for designing optical systems using optical design software:

• CODE V optical design software can trace rays through the optical system, optimize the system to reduce aberration, decrease distortion, and increase resolution.
• LightTools illumination design software can model illumination, stray light and ghost images. LightTools can also optimize illumination uniformity. Stray light may cause artifacts of the image and bright spots.

Diffractive gratings couple light into the waveguide plate and couple the light out of the plate into the eyes. You must design gratings properly so that the optical system produces good images. You can design and optimize gratings based on diffraction angle, efficiencies, and so on, in any order or combination of orders.

To design gratings, you can use Keysight RSoft Photonic Design tools:

• DiffractMOD RCWA is a very efficient tool to rigorously calculate diffraction properties of transversely periodic devices.
• FullWAVE FDTD is another powerful tool to rigorously calculate diffraction properties of transversely periodic devices.
• MOST optimization in the RSoft CAD Environment provides a convenient method to optimize gratings with either FullWAVE or DiffractMOD.

Once the gratings have been built, you can export the Bidirectional Scattering Distribution Function (BSDF) information and layout files directly to LightTools to define a surface property. The RSoft BSDF files contain information about how a surface (thin film, patterns, etc.) scatters light, including all diffractive properties.

In VR, the display only needs to output the simulated environment. In augmented reality (AR), the display is often see-through to combine the simulated environment with the real environment.

There are a few differences between VR and AR optics:

• First, AR requires high luminance displays, especially for bright environments such as outdoors and surgery rooms.
• Second, the dipvergence should be less than 1 to 3 arc minutes for see-through HMDs for AR.
• Lastly, the see-through HMDs often follow a folded design to enable a wide FOV and compact form factor. See-through HMDs must integrate an optical combiner to combine reflected light from the virtual scene and the transmitted light from real-world objects. Prototyping often uses a beam splitter as a combiner. HOEs can reduce the form factor of the combiner, as they are thin and flat and can function like a beam splitter for a specific wavelength.

• Education and training: Military flight simulation and battlefield combat training, medical training for surgery and emergency scenarios, and patient education incorporate VR to help individuals experience procedures and understand what to expect. VR also supports remote learning by re-creating classroom environments or historical scenes, and offers immersive experiences in museums.
• Engineering: VR assists in 3D design and virtual prototyping, enabling engineers to visualize and refine products before physical development.

• Social interaction and commerce: VR enables virtual interactions with colleagues or customers, provides display rooms for online shopping, and offers 3D experiences for real estate tours.
• Entertainment: VR enhances gaming experiences and tourism by immersing users in interactive environments.
• Medical rehabilitation and remote surgery: VR supports psychological exposure therapy, rehabilitation for conditions like Alzheimer’s disease, and remote surgical procedures.