Microsoft insists on calling AR and VR ‘Mixed’ Reality. Maybe we should too.

Back in 2015, when the the world was just beginning to pay serious attention to virtual reality, Microsoft surprised everyone by announcing HoloLens. Instead of surrounding you with virtual imagery – like Oculus and every other VR company – HoloLens brought the digital world into the real. To date, it remains one of the coolest things Microsoft has ever made.

But along with HoloLens came Mixed Reality, a term that seemed to confuse pretty much everyone not working in Redmond. ‘Virtual’ and ‘augmented’ reality were the established lingo, and HoloLens seemed to be just a fancy form of the latter. ‘Mixed’ reality reeked of marketing buzzword, it sounded lame, and so I avoided it as much as possible.

Ever been to a tech festival?

TNW Conference won best European Event 2016 for our festival vibe. See what’s in store for 2017.

Now I’m starting to budge. In the last two years – culminating in this week’s Build conference –  Microsoft has laid out a foundation for its grand scheme to fundamentally change how we interact with our devices. Knowing why Microsoft so stubbornly adheres to mixed reality is crucial to understanding how it plans to get there.

HoloLens was just the start.

Mixed Reality includes AR, VR, and beyond

Over the past few months, I’ve had several discussions about mixed reality with Greg Sullivan, Director of Communications for Windows and Devices at Microsoft. In each, he’s repeatedly drilled into my head the idea that mixed reality is a spectrum – one that encompasses VR, AR, and everything in-between. In fact, the term has academic origins that long predates modern virtual experiences.

So no, you’re not wrong to call HoloLens an AR headset, it’s just that AR is but one part of the mixed reality spectrum. The term also encompasses a wealth of other device categories:

  • Fully immersive VR headsets like Rift, Vive, and Microsoft’s offerings coming later this year.
  • Camera-based AR, like Snapchat’s funky masks or games like Pokemon Go.
  • Augmented virtuality experiences, whereby real-world elements are brought into a virtual experience.
  • Potential future devices, that can do the best of both worlds, changing between being opaque like Rift, or transparent like HoloLens.

While HoloLens happens to be an AR device, Microsoft would be doing itself a disservice to limit its scope to that bit of spectrum. HoloLens was the first device in the Windows Mixed Reality platform, which seeks to enable immersive experiences in all the above form factors.

But why go with HoloLens first? After all, VR headsets seemed to have more immediate consumer applications for things like gaming, and are a lot less expensive to produce than HoloLens.

According to Sullivan, Microsoft wanted to start by solving the more difficult problem. VR headsets have the luxury of obscuring the real world. You can create more interesting experiences with some environmental interaction, but VR can be fun even while your feet are stationary.

HoloLens, on the other hand, required an understanding of the environment around the user. It was a deeper problem to solve, but one that would reap rewards; Sullivan says Microsoft used what it learned from HoloLens to create affordable VR headsets that beat the competition to the punch with inside-out tracking.

That means that, unlike Oculus and Vive, Microsoft’s partner VR headsets can map your movements in the real world without the need for messy external sensors. I’ve spent a fair amount with the Acer dev kit, and it works incredibly well.

Add the fact that they also require lower minimum specs, and are much cheaper than similarly spec’d competition, and it’s clear Microsoft was onto something. Here’s hoping those fancy new controllers work just as well.

Mixed reality is about shared experiences

There’s a second meaning to the ‘mixed’ part – Microsoft wants different device categories to not only coexist, but to seamlessly interact with one another.

At Build 2017, I was able to participate in a multiplayer, mixed reality minigame where some players used HoloLens, and others used Acer’s VR headset. The combination of AR and VR was a pretty trippy experience.

Players with the VR headset inhabited a small virtual island, while players with HoloLens saw a birds-eye view of the action. The two had to work together to solve basic puzzles and navigate through the island.