Unity's Head of VR Offers a Much-Needed Reality Check

A peculiar but predictable thing is happening with VR and AR. As with any sci-fi technology, opportunists are spinning them out into la-la land; not quite mainstream or even there yet today, but truly necessary, some say, to a fantasy future anyone has yet to pin down. And now that the metaverse has become a part of the calculation, tech evangelists are excited to announce that, as a ticket into this fantasy future, you’ll need to strap on a headset.

Few are more plugged into the reality of VR and AR than Timoni West, vice president of augmented and virtual reality at Unity Technology. “People use Unity to make most XR today,” they tell WIRED from Unity’s San Francisco headquarters. West believes firmly that, in the future, almost every creative tool will have some XR component. But they also have their feet planted firmly on the ground; they have to, or they wouldn’t be equipped to act as XR’s unofficial hierophant.

WIRED sat down with West to sift fantasy from reality and pin down what XR is actually good at. And it may come as a surprise that a lot of it relies on collecting a lot of data. The following interview is a transcript of our conversation, lightly edited for clarity and length.

WIRED: So let’s start with sort of an ontological question. There’s been this idea that we’ll be in or go to the metaverse, or several metaverses, which tech companies posit will exist in VR or AR. Do you see VR and AR as being more of a tool or a destination?

Timoni West: That's a great question. I would actually say neither. I see XR as one of the many different mediums you could choose to work in. For example, we actually have an AR mobile companion app [in beta] that allows you to scan a space and gray box it out, put down objects, automatically tag things. So I'm using AR to do the things that AR is best for. I'll use VR to do the things that VR is best for, like presence, being able to meet together, sculpt, or do anything that's, you know, sort of intrinsically 3D.

So, for you, it’s a medium. It’s been interesting seeing tech companies oscillate between describing VR and AR as a tool and a place—a way to do or access things, or a location of immersion.

Isn’t that funny? Yeah, I disagree. I think it's going to be a combination of both. It's like cyberspace. Remember when people said “We're going to surf the web in cyberspace”? We still use this language. You go to a website. You visit the home page. We are using that language of a journey. Is it true? No. But I guess it's just kind of how we think. Maybe we think about going through time as a journey. When we talk about time, traveling through time, we use that language of movement. And so I wonder if languages that don't use journeying as a metaphor [in the same way] would choose some other metaphor.

I have no idea! But thinking about it a little more, there’s increasingly this level of immersion with VR and AR that’s, like, if you’re always connected, you’re always participating in it. There’s a lot of GPS awareness that goes into AR today, and especially mobile AR like Pokémon Go. Isn’t that kind of one point in favor of “being in” AR, or “being in” a metaverse?

Totally. Yeah. Have you seen the Unity Game Engine before?

Yes.

OK, so you know that there's this window and inside the window is like an infinite plane and a really boring blue sky. I wish it was more interesting, but anyway, say we hooked someone up with phone signals and they’d walk around and you could see them in the Unity scene moving around. And say they’re holding a phone and scanning the location. It starts to get to the point where this is in the Unity scene, and this and this, and if it does human body tracking, then we’re in it, too. So it kind of feels like: enter player one; enter player two, into the real world. As long as you're tracked, then you kind of have a digital presence. I don't mean, like, VR-style presence. As long as a computer is aware that you're there and you're interacting with stuff, then yeah, we're just in a meta game at that point. Or the reverse.

It reminds me of the sort of immersive theater people do with AR. Like, at the Game Developers’ Conference in San Francisco a couple years ago, I tried out this interactive AR murder-mystery-type game where different clues showed up on my phone when my phone was scanning parts of this IRL set. It was like, we’re playing the game and we’re in the game and we are the game.

Exactly. That's one thing I'm really interested in. Like, I'm a big theme park fan. Suddenly you're in this new place. There are new rules. You can interact with characters. It's all highly thoughtfully designed and obviously that has really strong correlations to VR. They’re augmenting your reality with design, but then also with digital stuff now too. Like, have you been to Harry Potter World?

No, I haven't.

It's great. They have these wands you can literally interact with the scene. You can move the wand and it causes some magical spell to happen. And that's all just combining the digital on the real.

Yeah, it’s become a huge thing. Theme parks can literally track you now and input you or your dataset-avatar into their digital simulation of the theme park. It’s funny—that made me think of this controversy years ago when Disney World filed a patent for technology that would, like, track visitors through their shoes. It was to provide data on popular rides and guests’ paths around the park. Have you ever heard of that?

I think that'd be weird with really popular shoe brands, but it makes sense. Right now there's this Disney app that you can download, and you can use it to play games—regular games or AR games, trivia games. But also, once you have certain achievements, it knows when you go on every ride, it gives you an award. And I think, long-term, the idea is that you go to the cantina and it recognizes who you are, robots can talk to you about things that you've done. Like, imagine you go home, you go to your VR headset, and you go to the VR version of Disney World and the characters are again even more interactive. Their biggest problem is scale.

It makes me think of Pokémon Go creator Niantic, too. They digitally map the area you’re playing in on a very granular level and incorporate player feedback into their games.

Yeah, and they get pedestrian information, which none of the other big players have. Because everyone else just has self-driving cars to go off. Niantic actually gets people walking.

Is that a philosophy you take in and design off—collecting data to build these virtual worlds?

Yeah, exactly. We love Niantic. We think they’re amazing. And it’s really cool for us because this is a new, novel dataset that no one else really has at the same scale. They've actually made tools built on Unity as part of [Pokémon Go’s underlying platform] Lightship, so people can go make their third-party game on it. Interiors can be a little tricky because of the privacy stuff. But if you really want to have an experience that follows you around and really understands you, you're going to need that level of granularity. And there are a lot of people who are working on how to make sure the data is just processed real time, it's not stored or you can't backtrack it for privacy reasons. I'm hoping other companies can kind of get the flywheel they have, where they get more people playing their games, which makes the data better, which makes people play the game more, which makes the data better.

Yeah, world-building, in a way.

Imagine it with [Activision Blizzard convention] BlizzCon. A use-case is going to BlizzCon and, while you’re there, creating your own AR world, or everyone co-creates the world.

It would be so cool to see people’s World of Warcraft avatars over their bodies at BlizzCon. That’s, like, sci-fi though.

It’s not! Seriously, you can do that.

Really?

You just need body tracking. Let’s see... so newer iPhones have a U1 chip, which is directional. It has very targeted direction, so it knows where you are. Basically, the two things you need are body tracking, which we've got, and also me knowing specifically who you are, which would involve, probably, a device handshake. You could also use, like, a bracelet, any kind of electronic ID that would be fine. Or if you really want to get really boring, you could use an image market on a T-shirt.

Like a QR code?

Yeah.

I think that one of the more interesting directions AR could go in is unlocking the parts of people that are compartmentalized online. Like people’s avatars in video games, their Fortnite skins.

A hundred percent. It is interesting. I'm not going to name names, but I tried a VR experience recently that required me to have a pretty realistic avatar. I never do that when I'm in, like, Altspace or VR Chat. In VR Chat, I'm always a stick of butter with little, skinny legs because it’s so cute. But in Altspace, I have pink skin and white hair. I look kind of like a robot. [In that other experience] it kind of made me uncomfortable to look like a really boring version of myself in clothes I'd never wear in real life. I never felt so uncomfortable before in an avatar, and it got me thinking about how much people will want to express themselves or in what ways, given real options, you know?

Options are the thing. I play a lot of video games and I am also very impressed by the degree to which systems like Roblox are very immersive for people. I feel like we've learned so much about immersiveness, what that means, over the last five years. The extent to which people care about graphical fidelity, about motion, the accuracy of movement. What sort of lessons do you feel like you learned or what sort of assumptions do you feel were prevalent that might have been overturned?

Well, one of the things is that high fidelity is not that important. Actually, the feel is way more important than the visuals.

It's like the debate between latency and resolution, yeah?

Yeah. But I think it goes a little bit further than that, too. Due to the fact that the headsets are so slow, there's no way to look super photorealistic in VR, even if you wanted to. There's no way to bridge that final little gap. Even with movies, we're only just barely getting there. With that being said, often, when a VR experience is trying to look ultra-realistic, it feels the most uncomfortable and oppressive to me. I'm like, at least take the roof off, have a dolphin in the background or something. Why do I have to have a roof in VR? It ends up being a lot more about whether mechanics of the world are fun. And I think that's where Roblox is really interesting. Roblox has mechanics and then people make games on top of that. So it kind of changes depending on the game within Roblox. Minecraft is like that, too. It’s got really tightly constrained rules and people will spend a million hours in Minecraft, literally. That's as low fidelity as you can get. And yet I would argue it's incredibly immersive, too.

But for VR, it seems like you’re kind of at a crossroads. It seems like consumers want that sort of fidelity. That's what they think when they think about the next step of technology, the next step of immersiveness.

It’s a matter of education, right? You think you want it, so go try it, then. Go try a super ultra-realistic game and see how much you like it based on what’s possible. It's true, though, and people don't know at all. Actually, do you know [Second Life creator] Philip Rosedale?

Yes, I actually interviewed him a couple of weeks ago for an article on the metaverse.

Awesome. So my team hung out with him in High Fidelity and we're in this amphitheater. It’s a 2D spatial audio experience. We were like, why did you choose this realistic space? You can do anything. Why not do something cooler? And his response was when he first started Second Life, they really thought people would go nuts. Instead, people were doing recreations of their house in Florida. People start with what they know and they get used to it. And then over time, they can start to go nuts, but they have to start with that sort of realistic grounding. And I think that's why people want that fidelity in VR today. Over time, as you get more comfortable with the medium, you start to get more comfortable with what is good for VR.

I think there's a lot of skepticism, obviously, around that XR has a mainstream future. I’m sure a lot of that is because it’s not comfortable at this point in time. But do you think that within 10 years, VR will reach the point of mainstream saturation that, for example, PlayStation has? Or do you think that’s even a fair metric?

I do think it will. But I want to caveat that. I think VR headsets will become incredibly common and everyone will have them. What I don't think is that it's going to be like the way we interact with our phones. We are on our phones constantly. But if you think about traditional phones, by 1984, probably most houses in America had one, right? And were they on the phone all the time? No. They were ubiquitous and at market saturation, but they were used for a specific purpose. They're used to make telephone calls. Today’s phones pack in TVs and everything as a multi-device. If you only just used it as a phone, would you say you use it all the time? No.

Some people think that headsets will get rid of screens across the board. I'm not so convinced. Screens are pretty great. I think of computers as a tool. I'll use my computer when I need to type a lot and I need to look at a bunch of text. Because they really were designed for that. Computers have been around now since, like, the 1930s, and we just keep making them better for doing those things specifically. So I think VR headsets will be in every house and we'll use them for the things that they're good at, which could be games or social experiences or whatever. But they're not going to be something that you're just in 24/7.