Bailenson’s lab examines whether virtual reality can change your real-world behavior.
On a recent episode of Too Embarrassed to Ask, The Verge’s Lauren Goode and Recode’s Mark Bergen talked with Jeremy Bailenson, director of Stanford University’s Virtual Human Interaction Lab, about the future of virtual reality..
You can read some of the highlights from Bailenson’s discussion with Lauren and Mark at that link, or listen to it in the audio player above. Below, we’ve posted a lightly edited complete transcript of their conversation.
If you like this, be sure to subscribe to Too Embarrassed to Ask on iTunes, Google Play Music, TuneIn or Stitcher.
Transcript by Maya Goldberg-Safir.
Lauren Goode: Today on Too Embarrassed to Ask, we are recording at the Stanford University Virtual Human Interaction Lab. On the podcast today, I'm joined by Recode's Alphabet reporter Mark Bergen and Stanford professor Jeremy Bailenson. Mark and Jeremy, welcome to the show.
Mark Bergen: Thanks, Lauren.
Jeremy Bailenson: Welcome to the lab.
LG: Thank you, thank you for having us. So just before we started taping today Mark and I had the opportunity to get some demos, never-before-seen demos in the Human Interaction —
MB: See, Jeremy, we have a hard time because you chose not to call it the virtual reality lab! So is there a reason you call it the Virtual Human Interaction Lab?
JB: There is, actually. So we are part engineers, but our strength is really doing social science to understand how VR affects the mind, so when I was naming the lab in 2003, I didn't want it to be about the technology, I wanted it to be about the people, so we named it the Virtual Human Interaction Lab because we're really studying how people use the technology and what the applications are as opposed to building things from an engineering standpoint.
LG: So go back to 2003 when you first started this lab: What kind of technology were you actually working with?
JB: So, my PhD was in cognitive science and in 1999, I actually went to a lab at UC Santa Barbara where I learned how to do all the hardware engineering for VR. And the system I brought to Stanford in 2003 was a similar version to the one we'd been using at UCSB. It involved the same tracking system, actually, this optical tracking system that looked for LEDs on your body, so that part was similar to what we had today. For the head-mounted displays when we started the lab in 2003, we used the Virtual Research V8 — cost about $15,000, resolution 640x480 in each eye, field of view about 60 degrees, so, you know, not massively, massively different than what's out there today, just more expensive, more clunky.
LG: What would you say was a tipping point, or the first virtual reality headset you've used in recent years where you thought, "Okay, this could actually happen, this might be something that consumers can access."
JB: The difference in the head-mounted displays has not necessarily just been quality. So the head-mounted display that we have over here is the NVIS SX111, it cost $40,000, it weighs about five pounds, it's really heavy and uncomfortable to wear. However, the resolution and field of view are, you know, in the same ballpark as the systems that we're using today, perhaps the frame rate is not as high, it updates at 60 frames, not 75 or 90, but in general the quality of that was ballpark of where you're at. The difference is it’s really uncomfortable and you can't have two of them. When this head-mounted display breaks or goes down, we would stop the lab, ship it all the way back to Boston, have to wait on our research, so the neat thing about today is A, how light and fluffy and comfortable they are and B, just that you can have a lot of them. You can do research in different types of ways.
MB: So, the videos we went through — you guys can see what we tested out earlier on our Facebook page, facebook.com/recodedotnet — we want to talk a little bit about what we did. The first one was the plank walk, which I will happily boast that I was able to walk across but Lauren did not, and it sounds like from her conversation with Seaneen that there's 25 percent of participants that you bring in here are — you're in a virtual room and there's a plank and there's a huge drop off and some of them are just too nervous to walk across. Um, can you talk about what's the point of the exercise and where do you see a lot of... what's the point of it?
JB: The reason we have the virtual pit is twofold. The first is if I'm gonna convince you guys we can use VR to change your attitudes about yourself, reduce racial bias, help people do conservation which is not something people want to do, the first thing I have to do is convince you that VR feels real. I like to say, we should think of VR not as a media experience but closer to an actual experience, so the point of the pit is before I frame all the other demos that we're gonna show you about education and about empathy — things of that nature — I need to show you that it feels real. The pit is designed to do what's called "presence." Presence has been a concept studied in communication for decades. Basically, we define it as the illusion of non-mediation, meaning when VR's done right there's no gadgets, there's no helmet, you're just there and it's an experience. The second reason we have the pit is that some of the research we do in this lab which is one of the long historical successes of VR is treating phobias. So you can use VR to have people conquer their fears, for example the fear of heights, and there's a strategy called systematic desensitization, which is you can have people slowly confront their fears, get closer and closer to them and show that that strategy helps you overcome.
MB: This is something the military's been doing with PTSD.
JB: So the strategy to treat PTSD is slightly different than systematic desensitization, that's called exposure therapy, and that's basically when you've got trauma, one of the ways to treat that is to bring people back to the traumatic event so you can then treat it. VR more quickly gets you to that spot so you can then use cognitive strategies to overcome it, which is slightly different than cognitive behavioral therapy, which is slightly different than the phobia therapy, which uses systematic desensitization.
LG: So in my experience, I mean anecdotally, I had this really strange reaction to the pit demo. My palms literally got sweaty and I started sweating, and I decided not to walk across, even though, by the way, I've done the plank walk before in this lab and was okay with it before. So is the solution if you're using that to sort of treat a phobia or extract some type of behavior from the participant to then tweak the VR software to iterate by saying we're gonna change this game in a sense, this application, so that we get a certain result, or is it that you have the human continue to interact in a repetitive base until they change?
JB: Well, first off, Lauren, you shouldn't feel bad! Neither Roger Goodell the NFL commissioner nor Adam Silver the NBA commissioner would step off either, so you're —
LG: Oh! It looks like I have a future being an NBA commissioner!
MB: Just like Roger Goodell! [laughing]
JB: A lot of very tough strong people don't do that, so first of all you shouldn't feel bad. Second of all, it's probably a rational behavior to not step off that plank, so when you said what would we do for treating fear of heights, if you had an actual fear of heights I would have never used that exact simulation because one can make the strong argument you shouldn’t walk that plank. It's not a safe strategy for anyone to do.
LG: See Mark, you're irrational.
MB: Well, I mean, [laughs] when people are designing games and experiences now in VR is there sort of a guiding principle that you wouldn't do this, or you'd warn consumers that this experience — if you do have a certain fear of heights, maybe you shouldn't play this game?
JB: Well, let me finish up on the systematic desensitization, then we'll get to that pretty big question that you just asked. So, in general yes, the way it works — say it's fear of flying, and let's forget VR for a second — the way that a clinician would have you overcome it, the first session you'd just drive by the airport. Session 2 you'd actually go up to the counter and slowly you'd get closer and closer to your fear where you can kind of build up leveraging and bootstrapping those past experiences. So that's how it works in the physical world. What VR has to offer is you can do very expensive dangerous things that you couldn't do easily in the physical world. For example just forget dangerous, it's expensive to pay for the ticket to actually go through security if you're just gonna use that for systematic desensitization. Now onto the games, in general what I like to say about VR is more so than any game technology we have now, the brain treats a virtual reality experience more like an actual experience. So Lauren, your hands sweat just now when you walked the plank, I would venture to say your hands don't sweat when you watch a movie and they don't sweat when you see an extreme event in a video game. And of course our research has demonstrated this. We've worked on this for close to 20 years, and so what I always urge game designers to consider is that when you're putting players through these experiences, consider that they are actual experiences, they're not media experiences.
LG: I guess my question, though, is if you are actually looking — there was another demo we saw, for example, that's aiming to sort of treat pain for someone who maybe is suffering from pain or someone who has to go through physical therapy. So I think what you're saying here is that the onus is actually on the creator of the app, the game developer, whoever is making the content to sort of tailor it or consider who their audience might be. It shouldn't be, like, some type of an irresponsible immersion therapy where it's just like "let's just throw these people into this environment and see what effect we get."
JB: Well, one would hope that no patient would seek clinical help outside of using a clinician, so I'm certainly not suggesting we should outsource therapy to software. This is just another tool that a clinician would get to use.
LG: One of the applications that we were unable to experience but is brand new here, and I think you have a particular interest in, is the Crystal Reef app in VR. Tell us a little bit about what this is.
JB: So the Crystal Reef is a product of now two and a half years, it's a seven-minute field trip where one becomes a marine scientist, you become our colleague Thea McKelly, and you swim through a reef and discover how carbon dioxide is affecting the oceans. We premiered this at the TriBeCa Film Festival a few months back where we set up two booths using the HTC Vive system and thousands of people got to swim through and become a scientist and we were actually collecting data there to see how it affects their willingness to learn more about climate change and the degree to which they believe climate change is a serious threat. So this is one of the first self-contained experiences where you learn about science.
LG: Do you have a follow up?
MB: Yeah. Can you talk a little bit about the findings?
JB: So we've been studying the effect of these immersive field trips in the lab and across a number of studies, I would say close to a dozen studies. What we're demonstrating is that VR, this experience, this embodied cognition where you're moving around and doing what's called experiential learning, these techniques are very effective at delivering information. One of the neat findings that is not so much about the quantitative effects of knowledge transfer — which we do study and we do show the effects of VR on — but something I'm excited about right now is just the motivation to learn, so at the TriBeCa film festival — I'm not exaggerating — we had a line of 100 people for 11 hours a day for seven days straight, and these were people lining up in the hundreds to learn about marine science, all right, and part of that is because the technology is novel and people want to see what VR is like and the experience itself is very engaging, but on the other hand if you were to tell me that I could learn about marine science by scuba diving in this very special reef off the coast of Italy, I would want to do that and VR gives you that wonderful bridge which is I get to feel like I'm there. The brain treats it as real, but you're actually learning.
MB: And this is a tactic rather than, say, putting people into Antarctica and watching ice caps melt or seeing polar bears die or something. I mean, that's also an immersive empathic experience.
JB: So in our Crystal Reef experience, you do learn about how climate change is affecting the reefs negatively. So it's impossible to get the story across without showing some of the dire effects that can happen, but we do end on a positive note where there's things one can do. You can sign petitions, you can talk to your congressperson, there's things that you can do to revert this before it's too late.
LG: So the big picture story here is that it's one thing to read an article about what's happening to coral reefs, it's another thing to watch a video in class, right, that's on a two-dimensional screen, but it's another thing to actually put someone in that environment. And you think this is going to be a more effective education tool than maybe some of those other in media?
JB: So our research is certainly showing in general when you have a VR learning that's immersive and leveraging something called embodied cognition that's quite effective.
MB: So what is embodied cognition?
JB: Embodied cognition: There's a theory that's been studied for decades in psychology. We know that the mind is in the brain, but it turns out that muscle memory, the way that your body moves, is actually part of cognition. So one of the more famous studies in body cognition is if you take a pen and you hold it in your teeth and clamp that pen with your teeth so you're doing a smiling motion, the muscles of your face are smiling, and then I tell you a joke, you're gonna think the joke is funnier when your muscles are doing the smiling movement compared to when I ask you to take a pen and put it in your lips, and when you hold it with your lips, the muscles make a sadder face. In other words, by holding the pen it forces your face to have the muscles move that are either consistent with smiling or with a neutral face.
MB: Does it depend on the joke?
JB: Um! So the study that was from the 1980s, they had them read comics, and the comics that you read were perceived as funnier when your face was in the smiling position than when your face was in the not. There's been dozens and dozens, probably in the hundreds now, of experiments that show that if the body moves in a certain way it makes you feel a different way.
MB: Yeah, and I mean, the technology is now moving where you're not quite there. I was seeing Laura in the Crystal Reef — the hands, they don't look like your actual hands, right? They don't look like you're controlling your arm. But do you see that happening in the next few years? Where we'll be able to have these immersive experiences where we can — it actually looks like our arm inside the headset?
JB: Certainly when we focus on the effects of the self avatar, what we call presence or body transfer, we do take some pains to make the skin color match your skin color, but psychologically what's more important than how it looks is how it moves. So we pay a lot of attention to what we call synchronous movement, if you're moving your arms in a way that is consistent with your physical movements and your avatar moves in that same way, that is called self presence. But I do want to close the why do I think the environment will work. The environmental research is working and why we're showing this in a lab. So Dr. Jane Lubchenco was the head of the NOAA under President Obama for four years, in her four-year window —
MB: I'm sorry, acronym?
JB: The National Oceanic and Atmospheric Administration.
MB: I didn't know that so I assume listeners don't either.
JB: I appreciate that. The reason why I say that is because I always get one of those words wrong, so. Ah, so she was the head of the NOAA for four years, and in that four-year window she saw more natural disaster appear in this country than any other four-year window combined; arguably much of that disaster is due to climate change. If you consult the scientists, Dr. Lubchenco tells this heartbreaking story — which is her job — was to go to the areas where there's been devastation from these natural disasters and to help the victims. That's her job. And it was a great job because she got to help people. She tells this anecdote: It doesn't matter if you're in a red state or a blue state, if a flood has destroyed your town, you believe in climate change. In other words, when you have direct experience with how human activity is changing the planet, when you've got that direct experience, then politics go out the window. And she can tell all these stories about climate change deniers that she went and visited who, when it was their hometown, now are very supportive of trying to help on this. So I believe that I don't think we should just force people to watch polar caps melting. It should be constructive. There should be science involved in the teaching of this and we should really think about how we're doing this, we shouldn't just try to scare people.
MB: Do you see it also — are you doing longitudinal studies? Does it have a lasting effect? Like someone comes out of this and they're like, "I'm really passionate about climate change!" and the next week it's like, "Oh yeah, I know it's a big deal but what do I do about it?"
JB: So one of the goals of the lab is to do more longitudinal work and in the history of immersive VR there's been two longitudinal studies, meaning we track your behavior over months. Both of those studies in the history of VR have come out of this lab. In general, when we run experiments on seeing if a virtual experience changes physical behavior subsequently, in general I would say in about 20 or 30 studies we've looked a day later, maybe a week later and across those studies the effects of an immersive VR treatment tends to last longer, say, than video. You know, it's not — the picture's more nuanced than that. The truthful answer is we need to do more longitudinal studies because we simply don't know how long these treatments and experiences last.
LG: I think a lot of the studies we've heard about from this lab are — maybe the focus is a bit on the positive impact. You were talking about people being interested in climate change. I believe you did a cow, um, app for a while here. Was there a cow app that was related to meat eating? Meat consumption?
JB: That study’s gonna be published in about three weeks, maybe four weeks, so we'll talk more about it then.
LG: And then the flying one. I got the flying app earlier (which you can watch on our FaceBook Live on Recode) showing that people feel more capable, are more hopeful after they've experienced flying in VR. So you know, Mark was a New York Rangers goalie.
MB: Terrible, terrible goalie.
LG: Give yourself some credit! But I imagine there are some athletes that are really finding the positive benefits of using these applications. There also has to be a downside. So what are some of the potential negative aspects of being in such an immersive environment.
JB: Well, first off, I can’t look at you guys and say that the VR experience is so compelling on the brain that it causes behavior changes later on, and only claim it works on the good stuff. Right, that just doesn't work scientifically. So you know, if you take the mantra that VR is closer to an experience than a media experience, then you know you can let your minds go with that concept so you know everything we're doing in this lab, which is designed to be these really intense, self-transformational designed experiences to make you feel better about yourself and others, if someone was trying to do the opposite, you could imagine the effects that could have. We choose not to do that in this lab, though I think it is important research to understand what the effects of negative behaviors are, too.
LG: And for people who are interested in reading more about some of the studies you've done, where can they find it?
JB: On our lab's website we have links to all of our academic papers.
LG: And you also had a paper that was just published? Recently? That addresses some of these topics.
JB: Yes, my colleague Anthony Wagner and his postdoc Thackery Brown, they just led a paper where we helped on the tech side and a bit on the procedural side to understand what one experiences in an event in VR — even if it’s low-immersive VR — and you measure brain activity in the MRI, what happens in the brain when one experiences a virtual event and then later on recalls it. The findings have demonstrated that the way one would predict the brain to respond with an actual event is what we've demonstrated when someone experiences in VR an event and then later on recalls the same event from VR.
LG: Excellent, and people can find that where?
JB: That was published in the journal Science a few weeks ago.
MB: I was gonna ask a question. What do you think, quickly, are the implications of that?
JB: The implications of the study we published in Science first and foremost are just really understanding the role of the hippocampus in memory and getting better explication of what happens in general in the brain when people are recalling events. A side effect of this nice basic science study is its preliminary evidence showing that the way we would expect the brain to respond with a physical event also occurs with VR.
LG: Every week, we ask our readers and listeners to send in their questions, comments and complaints about tech topics, and you can do that by tweeting us with the hashtag #tooembarrassed. This week asked our listeners to send their questions all about virtual reality. Mark, do you want to read the first question?
MB: Sure, I'd love to. This one comes from the user handle @cdf1982. They have a good one! "I'm seeing games for Oculus and Vive, but are there apps? What about non-entertainment, like movies?"
LG: And we have a follow-up from Aaron Cohen, @cohencomms, who asks, "What are the latest applications for VR?" It’s kind of broad but it seems like maybe they are interested in VR beyond games. What would you say is one of the areas we're gonna see emerge in VR?
JB: Well, the struggle right now in the field of VR is you've got all these massive tech companies putting in billion dollar budgets and many of them, not just Oculus and Facebook and Google, but there are companies all around the world that are putting lots and lots of money into VR. The challenge — and this is why a lot of people come to this lab, because they want to understand more of the psychology of it — is what are experiences that people are actually going to want to do in VR? And the answer is it’s challenging, it really is a challenge. So we'll talk about movies. You know, the classic reason why movies are gonna be difficult in VR is there's a thing called a director and the director, she's brilliant because she tells you where to look, when to look, and in VR, the whole point of what makes VR special is it’s ... exploratory. You get to do whatever you want when you want, and you get to walk when you wanna walk and look when you wanna look and you can miss a key moment where the bad guy, he passes something to the other person or gives a sidelong glance and you missed it because you were staring at a plant and the plant looked really cool. So there's challenges directorially. There's one issue with film, so one of the home run apps we've seen here — Mark, you did the quarterback trainer? So the first time we ever showed that quarterback trainer to a football coach, the football coach said two things. He looked at it, turned around, first thing he said was, can you put this in here tomorrow? The other thing he said was, what can I do to make sure no one else gets this but me? It was the first time in 20 years of VR that I'd actually seen ... Every time people come to the lab, they say, "That's really cool! And I can see how some day that I'm gonna need this and it’s gonna change everything." This was the first time that we'd built something that people needed this second. And it just leverages everything about what makes VR special: You have to turn your head to look around. It’s an intense, experiential, expensive rare moment, meaning it’s hard to get 22 people on a practice field — and in fact the unions legislate how long players are actually allowed to be on the field and it's not very long — repetitions are good to practice, so how do we practice, by doing things over and over again. Once you capture that play in VR, a player, he can just do it over and over and over again. So it was one of the few cases that it was a perfect storm, that everything that made practice special, VR worked in that way, now. Those things are rare, those things are very rare!
MB: And if the coach gets his way, only that team will have it.
JB: Well, there's now eight NFL teams using it, about 20 major college programs.
LG: Is that why Roger Goodell was here?
JB: Roger Goodell came here, to his credit. Adam Silver came here, he’s the NBA commissioner, he came here because he was learning about trying to expand his fan base by having people experience the game from midcourt. When he came, I politely told him, Adam, I don’t think that's a really good use of VR for a number of reasons, let's just take time —
MB: Why’s that?
LG: Does that mean people are wearing the headset midcourt, while they're watching the game? Or they would be at home, watching a VR game?
JB: So Adam Silver came to the lab because he had thought that people in Asia could put on the helmet and feel as if they were sitting on the courtside, therefore expanding the price of a seat to the world and just gain lots more fans. There's a few reasons why I politely told him I didn't think that was a great use of energy from the NBA: The first one is similar to what we talked about with movies. It turns out that the camera people are really good at capturing this perfect moment, telling you where to look. It’s hard when you're at the game. The second reason is that I'm looking at you guys and I can tell you've done a lot of VR, I could ask you what’s the longest you've ever worn a helmet. And even with the Oculus CV1, which is a great HMD, and the HTC Vive, which is a great HMD, after a while it gets a little bit cumbersome!
LG: 20-30 minutes?
JB: Yeah, and we have a 20-minute rule in this lab.
MB: That eases into the next question from our reader, @irfanbhanji: "Are these people really going to be putting these headsets on, at the mainstream, or are these niche products really?" And here's a good burn, seems like class again.
JB: But just to finish the question, Goodell to his credit, he came not to talk about fans, he came here to talk about empathy. He really wanted to understand how to think about issues of race, issues of gender, a lot of our lab’s research is about having people think about becoming someone else, and that's why Goodell came.
MB: The point is that no one will wanna put it on.
LG: Right, I think the point, too, is how accessible this is going to be for people who are not, you know, scientists ...
JB: So I strongly think that VR is a really spectacular tool that can give you these really intense experiences that can change the way you think about yourself, give you really rare moments that are from a fan experience, or just a fun experience. That being said, the amount of effort it takes to make VR work really well, it’s a lot of effort! I believe that for the next year or so, I don’t think VR is gonna be in living rooms, I think you're gonna see early adopters do that but I think if I were someone who wanted to leverage what makes VR great, I would go for this intermediate phase where you've got external places you go to experience — basically the arcade model or, you know, we call this B to B instead of B to C, business to business instead of business to consumer. I don’t think it’s really ready for the homes because also, if you're only using it for 20 minutes, I'm not gonna spend the money on the system and I'm not gonna spend all the time and effort on setting it up and making it work right. That's why your intuitions about, I don’t want to wear this thing for eight hours a day and it’s pretty clunky, is why I think in the short term, for the next year or so, we're really gonna see flourishing centers where one goes to do VR for various reasons, whether it’s training athletes or whether it’s learning about field trips or science, or whether it’s about just some fun stuff, like becoming a hockey goalie. I don’t think it’s ready — I don’t think we're ready to have — I won't call it obtrusive, but fairly involved systems for tracking movements and for rendering them.
LG: So like a VR destination or something?
JB: Yeah, VR activation is what they're calling them.
LG: I'm gonna ask you a question quickly that's our own question and not a listener’s: Do you see a difference between something like the Samsung Gear VR experience that's powered by mobile and it is mobile, you can move around your living room or wherever — it’s only a hundred bucks, right? Versus these tethered systems that are really high powered. Will there be a convergence of these two things?
JB: In general, when we think about systems like Samsung Gear and compare them to others, the biggest difference is called translation tracking, so Samsung Gear, as amazing a device as it is, it only allows you to rotate your head. It doesn't allow you to physically translate, that means walk around the room and make it update. So you can walk wearing the Gear, but the scene doesn't update. So I think what makes VR very special is you get to move around using your natural body facilities with three degrees of freedom. The rotation-only systems like Samsung Gear, Google Cardboard, all these other systems — they're fine, but I think what makes VR special, in my opinion, is you gotta be able to walk around.
MB: The next reader question comes in from Alex Carter, @AlexCartaz: Is Apple making a big VR play? Doesn't it seem natural as the next generation of luxury hardware?
LG: I'm sure we're all equipped to answer exactly what Apple is doing next.
MB: Exactly. [laughing] Tell us what Apple is doing, their roadmap please.
LG: Has Apple been in this lab?
JB: Of course they have, yes, but so have all the tech companies, so that's not a unique data point. Look, Apple I think like all the companies, they're really thinking through their strategy. I mean, the data points we have are — they bought this company called Faceshift which does facial tracking for avatars, they've hired this amazing guy Doug Bowman from Virginia Tech University, who's one of the really good scholars of VR. These are the data points we have, I don’t know.
MB: As a scholar in VR are you worried about the tech companies, I mean Apple, Google, Microsoft, Facebook, they've all been hiring a lot from academia. Is that a concern for you?
JB: Well look, my job here is to give my grad students their dream jobs, and if their dream jobs happen to be at one of the big tech companies, it makes me sad that they don’t wanna become professors but I have to, you know, let them do what they wanna do.
MB: Isn't there some role for independent research, that comes out of academia and not these big tech companies?
JB: Well I think the professor job is just too good to go away. I don’t feel like all the VR folks are gonna go that way. It’s a pretty sweet gig!
MB: Fair!
LG: Okay, let's move onto the next question. This came from Dory on Facebook. She said, not exactly a question, more of a statement: "Would be curious what interesting stuff is going on with VR and medicine." We touched on this a little bit earlier, when I referenced the demo we did — it was basically a physical therapy demo, which was pretty cool — but it seems like there would be a natural segue here, for VR to impact the way people are being treated in certain ways. Who do you think is the most compelling case you've seen so far?
JB: Look, medical VR is gonna be filled with applications. In fact, one of the historically successful areas of VR has been things like training surgeons and the different types of tele-operations. The domain I'd like to talk about today is pain relief. So we got approached by the Mayday foundation, and they said Jeremy, we'd love to fund your lab to do an academic study on how to use VR to reduce pain. It turns out that the same reason why presence works — meaning you feel so absorbed in the simulation, you forget that there's not a pit there — the natural fallout from that is that you a little bit get out of your body, meaning you're so distracted by the simulation, you forget your physical body. One of the worst-kept secrets in VR — and I call it the worst-kept secret because we've known for decades, since Hunter Hoffman did his pioneering work in the '90s — that VR is so distracting it actually reduces the psychological, the objective aspect of pain. So pain of course is objective in the sense that there are stimuli going on, there's stimulus going on in your skin. However, you can override that with various mental strategies, and Hunter Hoffman demonstrated in the '90s: If you take a burn victim and you put him in Snow World and distract him by — he's lobbing snowballs at these penguins — that the subjective experience of pain can go down massively. Since then there's been dozens, probably close to 100 studies on this, in very reputable journals. So when the Mayday foundation approached myself and Skip Rizzo, who is my colleague at USC, and Walter Greenleaf, who is a visiting scholar here in the lab who's an expert in VR medical applications, we decided instead of doing just another study in pain, what we were going to do was to host an event where we brought the best scholars on the planet who study academically virtual pain research and pair them with decision makers, people from various government organizations as well as corporations, who are in charge of figuring out how to move forward on pain. This event is occurring in Los Angeles in September and it’s gonna be a small event but we're really excited about finally getting all these scholars in one room, with some decision makers that can actually think about, you know, how can you get insurance to reimburse for VR for reducing pain, what’s the relationship between reducing opiate use and VR, how do you integrate it in hospitals, things like that.
LG: Makes me wonder how expensive a proposition this actually is for hospitals and clinics to get this technology. I mentioned earlier during our livestreaming, I had knee surgery within the last six months and I went to physical therapy. Fortunately I've been through it before so I kinda knew what to expect, but it’s like the equipment in there may have been from the '80s for all that I know and it was the same physical therapy I'd done 15 years ago and I think, okay, I would have used an application similar to the one that we saw if I'd had access to it but what kind of investment do hospitals and clinics have to make in order to get these systems in their buildings?
JB: So I'm just a VR guy, I study VR and how it affects the brain. The reason we're having this conference is to take the VR academics and pair them with the decision makers who understand what it costs to put something in a hospital, what the constraints are, because I don’t know.
MB: This is like the first time the medical industry has talked to VR?
JB: No! No, VR pain research, there's been a lot of brilliant scholars working with a lot of smart and talented governement people along the way. The conversations have been had often and early. What’s different about this is because the hardware is now costing hundreds instead of tens of thousands, it actually — if the question is cost — it is a viable strategy to give them. But hospitals actually give people systems in homes, facilities, physical therapy, things like that.
MB: Yeah, it’s a viable B to B, it’s not really a consumer application, right?
JB: Yeah, but in general I strongly feel that's the next year or two, those are the home runs. Don’t try to stick it everywhere, just find the place where it works. VR doesn't work for everything! Movies are still great! They should stay movies. The written word is still important for lots of things. The key for VR — you know, when students come to my lab and they pitch me ideas — whether it be for research projects or, we're Stanford, everyone's got a startup idea — if you give me 20 ideas to use VR to solve a problem, I'll say 19 of them would be better solved using video or the written word or some other technology. In other words, VR isn't for everything. And immersion is not free, it’s distracting, it’s perceptually taxing, it pulls you out of your world, so we should reserve it for the things it’s actually gonna be good for.
LG: Our last question is from @MantasLuko on Twitter, who asks: What will be more impactful in the next decade, VR or AI?
MB: Big topics here. A light question!
JB: That's actually an easy one. There is no virtual reality, there's three separate technologies: There’s tracking your physical movements, we call that tracking; there's rendering, which is redrawing graphics and sound from a new location, from a different angle, which is mostly computer graphics; and then there's display, which is how to figure out the optics to show people sensory information, how to position speakers using spatialized sound, and then scent and touch if you wanna do those. Each of those separate three technologies would be nowhere without the advancements in AI. So I would make the strong case that AI is so pervasive throughout every enabling technology in VR that I would vote for AI over VR.
MB: Can we tweak the question now? They talked about this last week on the podcast, but Pokémon Go has been this huge trend, it’s still going strong, and a lot of people are saying what the tech industry has said: VR comes first and then we can get to AI. And now with Pokémon Go, people are saying, well, it’s sort of an AI application, maybe we can get the AI sooner and in more hands of consumers than VR.
JB: I think Pokémon Go is a brilliant game. I'm not convinced how far it was — it’s just a cute little picture over a video image, but they did something right, didn’t they! [laughs]
LG: Have you been playing it?
JB: I study VR and consequently don’t have time to play games, unfortunately. I played games in high school.
MB: Do you also do research on what they call next reality, and AR as well?
JB: In this lab we study pure VR full immersion. I like a world in which people can put on the helmet, have complete attention to some kind of simulation, learn, experience, entertain and then pull the helmet off and go outside. I love the world in which VR exists, and we do it for half an hour a day, an hour a day. The big money — of course they make more money, the big companies — comes when you are using a product more often, so from their perspective, AR is wonderful because they can have you use it for six, seven, eight hours a day as opposed to half an hour.
MB: Or 24.
JB: Well, you know, why not?
LG: How long do you think it will be before we start using VR to interact with other humans in a virtual environment? What if people started mapping their loved ones, and then those people passed, and then you entered a virtual environment where you could interact with them again? How far away are we from simulation in that way?
JB: The technologies to allow capture of the appearance of one's avatar and the gestures and nonverbal nuances of one's avatar, that is basically here. If you think about the different companies that are doing really good motion capture and photogrammatic capture, those technologies are here. On the personality side, we're seeing a lot of advances with things like what IBM's doing with their AI capture technology, things like Siri. So I think the enabling technologies are there. Certainly there's motivation to capture one's personality; there's a great guy named William Bainbridge who does work for the National Science Foundation, he's a sociologist who studies a lot of this personality capture, he's an expert if you wanna check out his work. But certainly it’s a controversial topic to be sure, but you can imagine lots of smart people will be putting energy there.
MB: So you're saying all the tech pieces are there.
JB: I’m saying the tech pieces are there to create the 3-D model that looks just like you. The tech pieces are there to capture your smile, your nuances non-verbally, the way you move and tick, on the personality side, so the tech is there for the most part to capture what you look like and how you move, if it were recorded. The question is... if I captured your avatar and then 30 years later asked your avatar a question about some sports team that wasn't around 30 years ago, would it produce a novel response, one that actually sounded real, the way you would answer it? We're not there, in my opinion.
LG: Sounds like it's both an AI question and a VR question, a lot of that technology is going to be contingent on AI actually being able to process new information to be able to process that kind of realistic simulation.
JB: I would say it’s more AI than VR on the interactive sie. I mean, there is an advantage or disadvantage depending on your philosophy of this very big question, which is: If I could simply capture you right now, and everything you're doing now is there, it’s in 3-D, it was in high fidelity so it felt like you, 50, 60 years from now my kid could walk around, my grandkid could walk around and see you from different angles, sit on your lap. That technology is largely here.
MB: Great!
LG: Lets get started! I am going to live forever.