During the pandemic, everyone got many miles from changing Zoom backgrounds to live thin meetings. But it looks like Apple is considering the same kind of technology for its upcoming AR specs, the Apple Glass.
First discovered by Patently Apple, the patent describes how a headset could perform chroma testing – replacing a background of solid colors with something else. This may not sound like a burden, but the trick is to make it a hardware solution on the headset, because even slight delays can otherwise promote motion sickness. Add to that the fact that a headset is constantly moving as opposed to a laptop-enabled laptop, and you can see why this can prove quite a challenge.
First filed in February, but only released publicly this week, Apple’s patent is about creating a 3D stereoscopic background for low latency for Apple AR. What comes to mind here is “a portable immersive head-mounted display (HMD) that performs real-time color testing with embedded stereo RGB cameras to capture the environment and a display system to display real-world images enlarged with virtual content . “
It seems that Apple has had some success with this challenge. “The system runs at a high frame rate (typically 75 fps or more) and it achieves a latency of less than one frame by capturing images and formatting them for display in the HMD itself,” the patent states.
How did Apple do this? By loading the processing to the headset, which, according to the patent, can ‘format camera images, detect the selected color range and create a composition with the virtual content. “Slim stuff.
Along with the fact that people move their heads when wearing a headset, there is another issue that Apple has to deal with: all of this has to be done twice, once for each eye. “The system achieves mixed reality by allowing a user to look at a display through wide-angle lenses (FOV is typically 110 degrees or more),” reads the patent. “Two cameras located behind the HMD display capture the environment from viewpoints located a few inches in front of each eye.”
All very nice, but what’s the point? As you might expect, the patent here does not go into great detail, keeping things vague: “games, videos, and virtual objects.”
We will have no concrete answers shortly. People in the know differ on when we will see Apple Glass for the first time, but the earliest estimate is March next year, with well-connected analyst Ming-Chi Kuo saying first 2022. A report from last year suggested that 2023 was more likely.
It’s also worth remembering that Apple patents a lot of things – more than 2500 bits and pieces in the last year alone – and not everything is used. So, while you may be applying funny backgrounds to your Apple Glass in 2023, it’s equally likely that this is another feature left on the cutting room floor.
