Dinosaurs and Superheroes: Why Your Car May Become a Virtual Playground

If you’ve ever wondered what it’s like to play Pokémon Go in augmented reality or Marvel Powers United in virtual reality while you tool around in your self-driving car, just stick around: extended reality is coming to autonomous vehicles. It’s just a matter of time. And yeah, we’re talking about some really fun stuff.

Extended reality experiences such as augmented reality (AR), mixed reality (MR), and virtual reality (VR) play two crucial roles in the adoption of fully autonomous vehicles — as in the kind that can drive themselves without any human intervention. And we can help you figure out how to adopt extended reality in vehicles — contact us to learn more. Here’s how extended reality is paving the way for a future where autonomous vehicles do all heavy lifting while people sit back, relax, and enjoy the ride: 

1) Preparing the Market

How do you actually get people ready for the experience of being in a self-driving car? And how do you test self-driving cars for safety? These are huge issues affecting the uptake of fully autonomous vehicles. Augmented reality is already addressing those questions.

Until AR came along, it was unacceptably time consuming and costly to use closed tracks in order to test how well autonomous vehicles react to real-world hazards such as another vehicle running a red light. But fortunately, researchers at places such as the University of Michigan’s Mcity test track are figuring how to use AR to overcome those impediments by creating realistic hazards on test tracks. There’s still a lot of work to do, but AR is already making a difference preparing the open road for safe autonomous vehicles that sense and respond to the unpredictable ways that human beings on the road can act.

But convincing people to trust a self-driving car is another challenge completely. Trust takes a lot of educating. And here, extended reality, especially AR, can make a big difference. For example:

  • Civil Maps is a startup whose mission is to crowdsource maps for self-driving vehicles. Civil Maps has created an AR experience that makes it possible for people to see the world through the lens of an autonomous vehicle while a person is actually in the vehicle. A screen gives the rider a visual representation of sensor information as the vehicle processes it.
  • Nissan has developed Invisible-to-Visible, an application that visualizes real and virtual world information through augmented reality. The interface relies on 3D and to merge the real world and virtual world, which enables an autonomous vehicle to share information that the driver would not otherwise see. Per Nissan, this information “includes potential hazards hidden behind buildings, and obstacles ahead of a blind corner– all displayed seamlessly to the driver to support a confident driving experience.”

Both the above scenarios have something in common: they make humans more comfortable with autonomous vehicles, by increasing comfort in either their safety or in their utility. 

2) Navigating and Entertaining

Now for the really cool stuff. 

Tech companies are well on their way developing extended reality applications that make autonomous vehicles more fun and useful at the same time.

Consider Swiss company WayRay. WayRay has created a holographic experience that overlays on a windshield virtual content that changes continuously as the vehicle’s route changes. That way, the driver can see content such as clear route directions, points of interest (such as a pop-up menu for a nearby restaurant on your route), all done with immersive and colorful displays.  

But wait, there’s more! WayRay has also created a software development kit that people can use to create their own AR content overlaid in the real world – including a dinosaur roaming the city streets in front of you. A dinosaur jumping out to say hello on Main Street would be incredibly distracting and dangerous if you were actually driving – but in a fully autonomous vehicle, the T Rex is, well, fun.

How about getting fully immersed in virtual reality while you’re behind the wheel of a self-driving car, though? How realistic is that? Answer: very realistic. Some day.

Look no further than Disney and Audi. At CES 2019, the two giants teamed with startup Holoride to showcase an in-car entertainment system that uses VR headsets to turn a vehicle into a moving theme park. That’s right. As CNBC reported at the time:

At CES, Disney created a demo for Holoride called “Rocket’s Rescue Run,” based on Marvel’s “Avengers” movie franchise, where a person wearing a VR headset feels like they’re racing through space, shooting at asteroids and enemy ships, along with characters like Rocket Raccoon and Iron Man.

This  experience really fulfills the promise of extended reality and self-driving cars. One in which your car does more than take you from Point A to Point B. One in which climbing into your car means spending some quality time in the Marvel Universe. 

If these all sound like forward-thinking concepts – well, they are, but they aren’t, too.  These AR experiences build off work being done in cars that are manually controlled the old fashioned way. For instance, Isobar worked with Volkswagen to create an in-car experience, Road Tales, to entertain families during road trips in the Netherlands. Road Tales combines a voice-activated audio book and AR to create for kids immersive stories that change as the car’s location changes. The really cool part: Road Tales uses AR to transform ordinary road objects ranging from trees to windmills, into magical characters of a story. 

The only difference between Road Tales today and Road Tales tomorrow: in an autonomous vehicle, the driver gets to play, too. 

Next Steps

Businesses don’t succeed by waiting around for technology to hit the mainstream. Innovators – both startups and well established industry leaders such as Disney – sense and respond. How about you?