-
Google Depth API in action. This whole gallery is gifs.
-
The depth map.
Google
-
You get real physical simulations, where virtual objects can interact with real geometry.
Google
-
The dominoes hit a wall.
Google
-
Samsung’s Quick Measure app can measure something instantly.
Google
The latest version of ARCore, Google’s augmented reality development platform for Android phones, now includes a depth API. The API was released as a preview in December, but is now available to everyone in ARCore 1.18.
Previously, ARCore would map walls and floors and scale AR objects accordingly, but the depth API allows for things like occlusion, allowing AR actors to appear to be behind objects in the real world. The other great feature enabled by depth detection is the ability to simulate physics, like the ability to throw a virtual object down real-life stairs and bounce it realistically.
3D detection
While Apple is building more advanced hardware on its devices for augmented reality, i.e. the lidar sensor on the iPad Pro, ARCore has generally been designed to run on the lowest common denominator in camera hardware. In the past, that meant that ARCore only used a single camera, even when most Android phones, even cheap $ 100 Android phones, come with multiple cameras that could help with 3D detection. (Qualcomm deserves some of the blame here, as its SoCs often only support running a single camera at a time.)
In version 1.18, for the first time, ARCore may use some of this additional camera hardware to assist with 3D detection. While the Depth API can be run in a single camera mode that uses motion to determine depth values, it can also extract data from a phone’s time of flight sensor to improve depth quality. Samsung was one of the companies that was specifically called to support this on the Note10 + and Galaxy S20 Ultra. Please note that both are the highest level skus for these devices. Tons of phones have multiple cameras like wide angle and telephoto, but many phones have ToF cameras.
To guess the future of ARCore, a good idea would be to look on the other side of ARKit, Apple’s augmented reality platform. A very deep feature in ARKit that doesn’t seem to be mentioned in Google’s blog post is “people occlusion” or the ability to Moving objects to hide virtual objects. Google demos only show stationary objects that hide virtual objects.
Depth API is available on Android and Unity SDK. For users, you will need an ARCore compatible phone. Google maintains a great list here.
