Camera-based tracking

Figure 1. WorldSense positional tracking. (Image: UploadVR)

Camera-based tracking is a type of 3D tracking that uses cameras and computer vision. It includes SLAM and VIO.

It is used on the Oculus Quest and Meta Quest 2, 3, and Pro. It is used by the Intel T261 used in the Project North Star. An open source method is Basalt.

Infrared point-based camera-based tracking was used in the Rift DK2.

Typically, a global shutter is desired.

Methods

• Markerless inside-out tracking (SLAM/VIO) - This is used on headsets such as Meta Quest headsets and the Project North Star. An open source method is Basalt. Markerless inside-out tracking can use visible light or infrared light, which can be artificially emitted.
• Marker-based tracking - Camera based tracking that uses markers in the environment, which is simpler to implement.

List of camera sensors for camera-based tracking

• Onsemi MT9V034 (Previously sold as the Aptina MT9V034) - Used in the Oculus Rift DK2
• Onsemi AR0135CS
• STMicroelectronics VD55G1

Examples

• WorldSense