The creators of Mario Kart Live talk about the future of video games

Mario Kart Live: Home Circuit from Nintendo is a fascinating little toy. It is not the first of its kind, but for its performance and price it is one of the best. In a way, it’s a robot, or a connected advanced peripheral, that can interact with invisible obstacles in video games. It can make me feel like I’m a tiny person crammed into another space, exploring it in detail I might not otherwise grasp. It combines augmented reality and the real world at high speeds.

Robotic toys were a big thing a few years ago and then they faded a bit. Anki Cozmo, all WowWee creatures, all Sphero Star Wars droids and cars. Some died, some remain. But those were mostly RC vehicles. What Mario Kart makes me think about is telepresence.

One field trip I regret not taking, years ago, was a drone race in the Las Vegas desert organized in part by headphone maker Avegant. I had engagements in huge, tech-filled corridors at CES, and I couldn’t take a trip to fly drones and see them through the screen of my headphones, piloting them like little combat ships in a live video game. It was my loss, but years later, as I ran a Mario Kart across my dining room and looked at it through a Nintendo Switch screen, I felt like things had come full circle. A little drone race was here at my house in 2020.

Karthik Bala and Dan Doptis, the CEO and game director of New York-based Velan Studios, the company that created the Mario Kart Live AR-enabled kart, along with Nintendo’s Yosuke Tamori, Hiroki Ikuta and Yuji Ichijo, spoke to me. via email. about the process of creating Mario Kart Live: Home Circuit.

Playing now:
See this:

Mario Kart Live Home Circuit turned my house into a racetrack

3:44

“The project started in mid-2017, which was initially inspired by drone racing and the fun of RC cars,” Bala said. “We built a prototype RC car with a camera and sensors. The feeling of driving at high speed in the real world was fun, but we are game developers. We wanted to build a game around that, not just a fun toy. led to the idea of ​​a “mixed reality” experience. As the prototype took shape, it became clear to us that we had to show it to Nintendo. “

Bala explains that much of the AR work is done in tandem with the physical kart and the Switch, with the kart doing the camera and mapping work normally done on phones, tablets, and headsets in AR. “The kart itself is a ‘mini-console’ on wheels. It has an HD camera and sensors that connect wirelessly and directly to the Nintendo Switch system,” says Bala. “It’s a custom solution that offers low-latency HD video, computer vision, and SLAM (real-world kart positioning) to deliver a compelling mixed reality experience for gamers. The work is split between the kart and the Nintendo Switch. system, which does a lot of the heavy lifting. “

“When it comes to the camera on karts, we worked hard to sync the camera video with the sensor data and the controls to make the AR look natural,” adds Nintendo’s Ikuta. “On top of that, we optimized the system to minimize any lag that could have a major effect on the game. Once the footage goes through all that processing, it is sent to the Nintendo Switch system via wireless communication. After the footage is received from the camera, the system processes the graphics to bring the AR experience to life. ” Ikuta also added that the kart’s behavior had to continue to be modified to take into account different lighting in the home and to drive on carpets at low speeds.

Robots are not things we control, but things we embody

Telepresence robots have been around for several years, too – remember those big, rolling iPads that became a tech industry joke? And yet, at a CES show years ago, I remember being moved by a virtual tour in which dozens of those wandering screens on wheels were driven by real people elsewhere, unable to travel, maybe even unable to walk. These devices were a way of being in another place.

William Gibson’s recent books, The peripheral and the agency, explore many ideas surrounding people who embody other robots and drones through advanced wearable sensory systems. I’ve wondered about a world where thousands of robotic devices could exist, mapping the real world through depth-sensing sensors like those in the AR headset and the iPhone 12 Pro, available so you can jump at any time. It’s a classic sci-fi thing, and yet when I think about the potential of home robot toys and devices, I feel like this is exactly where a new wave of products could be headed.

Bala, from Velan Studios, seems to agree. “The groundbreaking technology is that we have fundamentally decoupled the camera and sensors from the host device (the Nintendo Switch system), which opens up all sorts of new possibilities. It’s a rich new paradigm to explore in the future!”

A way to explore impossible places?

While running Mario Kart around my miniature house, I started to think about the possibilities of building miniature theme parks that you could explore through small devices. I can see a small animated Mario Kart world sprouting around my furniture on my Nintendo Switch screen, seen from a doll-sized point of view. What if I could cower in a VR headset and step into a little Disney ride that existed elsewhere? What if I were a tiny person running away from large life-size people? What if I was traversing the secret interior of the Millennium Falcon in the park via a droid that I was controlling via my headphones?

Trapped at home this year, I have been escaping in virtually many ways. But incorporating another device, or robot, or drone, is not something that has been done until Mario Kart Live Home Circuit. And I’m surprised more toys haven’t explored this yet. I have used AR to hold a virtual lightsaber, but that didn’t put me in another point of view … it increased mine.

“I think in Mario Kart Live: Home Circuit, you can see Nintendo’s philosophy of developing new and exclusive entertainment from embedded hardware and software,” added Nintendo’s Tamori. This isn’t Nintendo’s first hardware experiment, in other words, and it won’t be the last.

But if Nintendo and Velan Studios were able to get this to work well on the three-year-old Nintendo Switch, what could it mean for more advanced phones and tablets? AR already exists on phones and will work on headsets. But it could also live inside robots.