Virtual reality

Beat Saber, an example of a game in virtual reality

A man wearing a virtual reality headset

Virtual reality (VR) is a type of 3D human-computer interaction that puts the user in a virtual world using an egocentric display. The virtual world is typically a 3D spatial environment generated by a computer. It can be done by using a virtual reality headset. It uses position tracking and responds to the user’s inputs.

Virtual reality is related to augmented reality.

Virtual reality based on a head-mounted display without vestibular stimulation is incomplete because it does not cover the full human perceptual space.

A limitation of normal virtual reality technology is that the motion tracking is inaccurate. It goes through a filtering process which removes the transients. This gives a floating dizzying effect. This is visible in 2D movies made of VR experiences. This is present in IMU+optical tracking systems including the Oculus Rift CV1, the HTC Vive, the Valve Index, and the Quest 2.

For facial interface headsets like the Rift CV1 and the HTC Vive, the motion tracking issue is compounded by the mass imbalance of the headset, where it is heaver at the front, making the user's head movements unusual. This is the case for halo headband style headsets too.

Technology

Head-mounted displays

VR is enabled by head-mounted displays (HMDs) such as the Oculus Rift. HMDs use displays, where the display is split into a left and right half, and the left eye sees the left part, and the right eye sees the right part. These displays use lenses along with motion tracking hardware to give the illusion that the user is physically inside the virtual world.

The headset tracks the movement of your head and changes the images shown on the display according to the movement. This process creates the sensation that users are located within the virtual environment. Users of these gadgets are able to interact with the virtual environments. Various input methods, from the traditional game controllers and keyboards to the futuristic hand gestures and voice commands, are available or under development.

More primitive ideas than VR can be implemented using a fixed stereoscope, or panoramic murals (or 360-degree murals). These fill the viewer’s field of vision with the intention of making him feel a sense of presence at a certain event or scene. Stereoscopes work because the brain processes different two-dimensional images for each eye into a single three dimensional-ish object. A stereoscope gives some sense of depth and immersion.

3D tracking

There are 2 classifications of tracking: rotational tracking and positional.

Rotational tracking tracks the 3 rotational movements: pitch, yaw, and roll. It is performed by IMUs such as accelerometers, gyroscopes and magnetometers.

Positional tracking tracks the 3 translational movements: forward/back, up/down and left/right.

Motion tracking can also be used to track your hands and rest of your body through various control peripherals.

3D tracking can be done using fiducial markers.

History

Virtual reality has for a long time been limited by the amount of computational power available.

Ivan Sutherland's head-mounted display

Ivan Sutherland's head-mounted display

Ivan Sutherland created the first headset with graphics that came from a computer. Beforehand, there was the Philco headsight, which used a live analog camera feed and a motion tracker.

Ivan Sutherland and Bob Sproull created "A Head-Mounted Three-Dimensional Display", an HMD that was held by a mechanical arm mounted on a ceiling. The gadget was connected to a computer and displayed simple wireframe graphics to the user. The arm tracked the user’s head movements but was difficult to use. The contraption was also too heavy and bulky for comfortable use.

Ivan Sutherland also shared the concept of the “Ultimate Display”. This gadget could simulate the natural world so realistically that a user could not tell the difference between actual reality and virtual reality. The concept comprised of a virtual world viewed through an HMD and had augmented 3D sound and tactile feedback; computer hardware that created the virtual environment and maintained it in real time; and interactivity between users and objects from the VR world in a realistic way. Sutherland suggested that the gadget would serve as a “windows into a virtual world”, and his idea would become a core blueprint for the concepts that encompassed the field of virtual reality.

Technology developments in the 1980s

The mid-1980s saw a blossoming of activity spawned by technological advances in small LCD screens and electromagnetic 3D tracking systems. These advances brought the cost of the components of virtual reality systems down to an affordable level.

LEEP, NASA, and VPL

The first VR headset ever made

LEEP Systems was a company that developed early VR optics and head-mounted displays. It was headed by Eric Howlett, who developed LEEP optics. LEEP Systems was the first company to offer a commercial head-mounted display. LEEP developed the Cyberface.

LEEP enabled VPL and NASA's VR.

The VIEW project at NASA’s Ames Research Center was founded with the purpose of producing a VR system that allowed astronauts to control robots outside a space station. The HMD that was developed had super-wide optics (almost an 180-degree field of view).

NASA worked with Eric Howlett at LEEP to develop the optics. LEEP optics were used by VPL for the first EyePhone.

Oculus Rift, LCD-based VR

Popular interest in VR increased after Palmer Luckey created the Oculus Rift and launched a kickstarter campaign for its development. Facebook bought Oculus VR for 2 billion dollars. After this, virtual reality's popularity hugely increased. Multiple companies invested in the development of their own VR systems.

The rise of smartphones with high-density displays and 3D capabilities has also enabled the development of lightweight and practical VR devices.

Google Cardboard happened. SteamVR further popularized PCVR. Facebook made all-in-one headsets starting with the Oculus Quest, and sold millions of them, with changed models being the Meta Quest 2 and the Meta Quest 3 and Meta Quest 3S.