Apple has filed a patent for a complex mixed reality headset, potentially including sensors that track users’ facial features, eyes, and hands. Variety reported on the patent application earlier today; it was published last week and originally filed in March 2019. It’s part of a long string of Apple mixed (or augmented) reality patents, and, like its predecessors, it indicates that Apple is broadly examining how AR might translate from phones to glasses.

The “Display System Having Sensors” patent covers a head-mounted display, a collection of cameras and other sensors that track the external environment, and a different set of sensors that track the wearer’s eyes. The components all work together to display a virtual 3D image that’s somehow based on the real world. Beyond that, there are a lot of potential variations. The headset could include hand sensors, “head pose sensors,” sensors that capture detailed information about real-world lighting conditions, “eyebrow sensors,” and “lower jaw sensors” that track movement of the mouth and jaw.

As Variety notes, Apple describes a system that’s different from most AR glasses, which generally project partial images that still let you see the real world. At least one variant of its system uses pass-through video in which cameras capture a video feed, then the headset composites digital objects onto it and displays the resulting images on a VR-like screen. That tallies with an earlier CNET report that said Apple was testing a combination VR / AR headset. However, the patent is open-ended, and this is just one of many options.

Eyebrow and jaw sensors, meanwhile, would be invaluable for porting Apple’s Animoji and Memoji avatars into a headset. We know Apple’s competitor Facebook is already building VR headsets that can track facial expressions and translate them onto virtual avatars. And hand tracking is a well-established control option for VR and AR, although it’s still not the primary option for any major devices.

As usual, we don’t know whether Apple is building products based on these patents. DigiTimes notably claimed this month that Apple had put its AR glasses plans on hold, although that rumor remains unconfirmed by other sources, and there are reasons to doubtthat Apple is abandoning AR glasses. However, Apple also hasn’t said anything publicly about plans for glasses, either. It’s mostly stuck to promoting its phone-based ARKit platform.

By TheVerge