Virtual upside-down world with Leslie Shannon
Podcast episode 27
We’re turning to video games for both work meetings and staying in touch with family and friends. But after COVID-19, will any of it stick?
Nokia’s trend scouter, Leslie Shannon, explains why she thinks the unexpected ways we’re using technology under quarantine signal a substantial shift in society.
Below is a transcript of the conversation. Some parts have been edited for clarity.
Michael Hainsworth: Are we living in the upside-down? Sure feels like it. Most of the world continues to battle COVID-19 and the real world feels like a dream. And the virtual world, our reality. Nokia's Leslie Shannon sees all this clearly. As its chief storyteller, she's discovered we're not waiting for the boss to figure out which digital tools to use to keep our corporate world on track. Employees are turning to video games.
Leslie Shannon: Yes, and in fact, what's fascinating is that this is actually, I think, the beginning of a shift, taking activities that happen in the real world and moving them into a digital world. And this is the kind of thing we've been talking about for a long time, but now it's really happening. And it's really happening at a grassroots level being led by ordinary people, simply because they need to solve a problem that has been thrown in their path by the whole lockdown pandemic. ‘Ah, I can't do what I need to do in person, so what can I do in another creative way?’
MH: And what's fascinating to me when it comes to the world of technology generally, is that we often find ourselves in a scenario where tools get built for one particular purpose, but then the user takes those tools and applies them in ways that the builder of that tool could never have predicted in the first place. And that's extending into the types of nontraditional technologies that you're talking about.
LS: Yes, exactly. And really a lot of the best examples that are coming right now are coming from people leveraging gaming platforms, and taking real-world, real-life activities and moving them into gaming platforms and sharing them with other people in there. When you think about it, this actually makes a lot of sense. Gaming, whether you're talking about PC gaming or, even better, virtual reality gaming, these are immersive universes that have already been created and that if people have played the game, they've already spent a lot of time in the universe and quite often they're very comfortable there. It's a really fine alternative reality like the VR game, Moss, which I highly recommend for anyone by the way, anyone who's just getting into VR. Moss is an absolutely beautiful, wonderful game.
In Moss, you're working very closely with an adorable little mouse, solving problems and beating the bad guys, and then finally solving the whole thing. Moss is a game that only takes a few hours to play through. And I found that when I was done, I just wanted to stay in this beautiful environment of the mossy forest floor, hanging out with my little mousy friend. You know, I didn't want to leave. It's like, I don't care the game's over. I just want to stay here.
This kind of creation of magical, mystical, safe places, enchanting places where wonderful things happen and people are starting to actually take real-life activities and have them in here. Here’s one famous example. There is a British children's book author who famously tweeted out that she started having her editorial meetings with her team in Red Dead Redemption II. So they're all players of the game and they've all gotten to a certain point in the game where they can actually meet and sit around the campfire and actually talk about things in the real world, in the avatars of their gameplay.
MH: Do you have to start talking like this?
LS: Yeah, of course.
MH: You are talking to the boss now.
This whole thing is pretty much done. We're more of ghosts than people.
LS: She said one of the problems is that the button to sit on the ground is the same control as strangling the person next to you so you really have to be careful when you sit down at the campfire. And so the wolves are howling in the distance and every now and then a character will come and raid you but it's just like, you know, a coffee break or something and then you get back to it.
MH: It’s a team-building exercise!
LS: It’s a team-building exercise. And so that was just one of the first examples that I saw. Games that actually emphasize that already have a social aspect to them. Animal Crossing, of course, that dropped from Nintendo,
MH: Oh my God!
LS: ...right around the same time. Yeah, the latest version.
MH: I gotta tell you, thank God this game came out during coronavirus or my 13-year-old would be climbing the wall. But help me understand how you take a game that is hugely popular with 13-year-olds, and convince the boss that, you know, maybe we should have our virtual meeting in a Nintendo Switch video game.
LS: There's an AI researcher who is actually going to be hosting an artificial intelligence conference in Animal Crossing in July. And something like 200 people have already signed up for it. Exactly how this works, I have not got the vaguest idea. But this is the kind of thing that's happening.
And on the private side, one of my friends was online in the middle of the night, and she ended up getting invited to an actual wedding that was happening in Australia. The couple couldn't actually be with any of their guests, and so they were holding the wedding on Animal Crossing. And their friends invited all kinds of strangers who were online to come into Animal Crossing and to join the wedding. One of the things that you can do in Animal Crossing is give gifts to other people as part of the social environment. And so, when they said “I do” and they kissed the bride, suddenly all of these strangers from all over the internet poured in and started giving gifts to the wedding couple. I mean, that's just beautiful.
MH: Hey, go wait a minute. The gifts, these were clearly virtual gifts?
LS: Yes, virtual gifts of things within Animal Crossing.
MH: At the end of the day, you don't have to worry about getting two blenders from two different aunts.
LS: Well, you know, if you do get two virtual blenders, you can actually just trade one away pretty easily.
For us to kind of expand our understanding of what counts, of what's real, to extend that understanding to the digital sphere… that's a really important mental leap and social leap.
MH: So with this in mind, we always look at the adoption of new technologies and try to extrapolate out what that means five years from now, 10 years from now. I can't imagine that five to 10 years from now, we're still going to be having virtual conferences in Animal Crossing, hopefully not because of COVID-19. But when the tide goes out on COVID-19, when we look around, how much of this is going to stick?
LS: Right, well, this is what I meant by this being a really important moment for digitalization because one of the things about the new immersive technologies that are coming like AR, VR, and MR. What they do is they introduce a level of digitization into the real world. And so we start actually mixing the real with the unreal.
What I find so significant, is it's actually showing that this is the moment when we as humans are saying, ‘You know what? The digital world has validity to it. I can actually stand in a game as an avatar and marry somebody and have that actually count back in the real world.’ And for us to kind of expand our understanding of what counts, of what's real, to extend that understanding to the digital sphere as well as the 3D around-me sphere, that's a really important mental leap and social leap. I think that it would have happened much more slowly without COVID-19. But necessity being the mother of invention, because we don't have those social connections in person, we're moving those social connections into these digital areas and I think that's gonna stick.
MH: Well, I was fascinated to see, nevermind weddings, that the sports world was getting into the virtual and online arenas.
LS: Again, they have the same problem. You can't do these things face to face. So they've had some interesting examples of taking things online. The Tour of Flanders, a professional cycling event they had to cancel, I think it was in March or April. They actually worked with an existing gaming application and then they invited 13 of the cyclists who would have been in the original race. They invited them to actually get on a stationary bike at home and to fight each other out in this app. So they ran the last 30 kilometers of the race with the cyclists each at home and with a camera on them. And with professional commentators. It was fantastic. It's really gripping, especially because when you do it this way, you can see the athletes up close in a way that you can't when you're actually watching a live event when they're on a road bike.
MH: It's like NASCAR on two wheels.
LS: They actually did the same thing. They took a game, an existing car racing game, and then did an invitational to professional NASCAR drivers to actually come and compete in these pro invitationals. And then they televise these with professional broadcasters and so on.
Welcome to the virtual Homestead-Miami Speedway or the NASCAR Pro Invitational iRacing Series opener. It's a beautiful day in South Florida. The grandstand is packed with virtual fans.
LS: And again, each one of the drivers is actually at a kind of driving station, a gaming station at home, but because they're not actually really racing. They've got a camera on the driver and the drivers can talk as they're actually racing each other.
MH: So are they trash-talking each other?
LS: Yeah, but you're also getting insight into what the drivers are thinking.
LS: And it's a much deeper strategic view.
Hill in third, Briscoe fourth, Smith Lee fifth, here they come off the corner for the final time. They all come together and the winner is Hamlin.
Wow, what a finish, what a finish! Yeah, those fans should be cheering. They saw a heck of a race.
That is incredible. All right, let's see if we can get our winner on the line and do our virtual of victory lane interview, Denny Hamlin. Wow, congratulations. What a fantastic run through the field.
Yeah, that was unbelievable.
LS: So it's not a direct replacement for a NASCAR race. It's a different thing, but oh boy, is it satisfying. So some really, really great substitutions here for the sports world as well, moving into that world of digital, and who knew it could be so satisfying? To watch professionals who play in a real-world, 3D-live sport, playing each other on a game platform. I think we are gonna see a lot of continuity with this stuff.
MH: Well, there was a time when phone chat lines and online dating. Those were the realms of the nerds too afraid to go to a pickup joint. Then the mobile phone came along and the convenience factor was substantial enough that the general public started to adopt the concept of online dating. I wonder if it's a similar sort of scenario here where the general public is starting to see the benefits of these technologies that were generally reserved for your 13-year-old kid or a nerd boy like me who's got a VR headset. We're slowly opening up the rest of the world to the realization that there are alternatives to an in-person scenario.
We really are moving towards a future where we start moving away from smartphones and towards a much more integrated view of the world.
LS: I think you're exactly right there. And in fact, one example from Porsche, in 2018, they released an augmented reality headset where, if I'm a Porsche mechanic, and I'm fixing the car here in my Porsche dealership, I can actually put on my HoloLens or whatever brand it was, and then I can actually connect to an expert who is not with me. They can look through my glasses and they can see what I'm doing and tell me what I should be doing and do things like draw on their screen and then I see it, do it and I know exactly what they're pointing to and all that kind of stuff.
Porsche found back in 2018 that by using these glasses, mechanics could actually speed up the repair time by 40 percent. Really great, right? Well, it turns out that in practice, the mechanics are like, well, that's a little weird. It's a little complicated, you know, back to what you were just saying about the early days of online dating. ‘Ooh, that's a little geeky, I'm not sure that's for me.’
But now with COVID-19, the mechanics are alone. There are no experts, you know, nobody is gonna be stopping by and saying, ‘Hey, have you tried connecting this to that?’ And so what they found is that the usage of these, the headsets that they already had in place has gone up by 300 percent. So it's about the adoption factor of the stuff that might be weird and geeky.
Again, necessity is the mother of invention. If this is the only way that I can actually have somebody help me, well, by God, I'm gonna use it. And so what we really are seeing is the acceleration here of adoption of these technologies. The only change I would have made is seeing this adoption, I don't know maybe a year later because we have a hardware limitation. We have a hardware limitation for a lot of the new hardware. If you try to get an Oculus Quest right now, they're apparently on six months back orders right now, because so many people now at home really wanna try this. Well, everybody's had that thought.
MH: Yeah, good luck getting a Nintendo Switch as well.
LS: Exactly, Switches are sold out, Quests are sold out. If only there was a greater distribution of hardware, that would be the thing that would make this moment perfect for the rise of digitization.
MH: But to that point, we often come back to the William Gibson quote about how the future is here, it's just not evenly distributed.
LS: Absolutely correct.
MH: Whether or not you can get yourself the Oculus Quest VR headsets on Amazon or not, what about those bigger picture issues that not all of us have the type of technology that's necessary to take advantage of these virtual worlds? But we're going to start bringing the world along with us. Us geeks are gonna drag everyone, kicking and screaming into that 21st century. Take us three years out from now, five years out from now, once we have, for example, 5G wireless rolled out in a broader fashion, and we've also dealt with a lot of those technical limitations of virtual reality, augmented reality and MR.
LS: I was just in a conference for AR, VR and MR, all that. So XR (extended reality) is actually the term for all of them. And there was a really fascinating presentation in there from Niantic.
Now, Niantic is the company that's doing Pokemon GO. And the thing about this location-based kind of thing is, so, you know how Pokemon GO works. You're on your phone and you're walking around and you're actually out there in the world and you have to be in a specific location in order to catch a specific Pokemon. If the Pokemon shows up somewhere and if you're a quarter-mile away, you're not gonna see that Pokemon. In order to build that, Niantic actually had to build a software representation of the entire world, and so they basically have this digital twin of a basic map of the world, so that they can actually trigger something to happen when you are actually there.
They are not the only ones actually mapping the world. Instead of having a 2D map of the world, having a 3D map of the world, but a digital twin lets them then know exactly where you are and then create things that actually fit into that physical framework around you. This is really the beginning of what's called the spatial internet, and the idea that where you are, you can get information and entertainment that merges into the physical reality of the place around you. That can be something as simple as looking through your smartphone and then slowly over time, especially as 5G comes in, then moving to some kind of head-mounted device. This ultimately moves to the idea is normal looking AR glasses. And so the kinds of experiences you could expect with this is, you know, you look at a restaurant and you can see the reviews of the restaurant just displayed in front of the restaurant. Or it can be entertainment. You can have the world around you painted to look like a Fortnite game. Or it can be transportation. You can stand at a bus stop and get a real-time report on when the buses are coming. So all this information that's on the internet, actually moving out and moving into the real world and being made available to you at the location where that information is. And this is really kind of the next big thing for the internet.
So Niantic was saying that the basic work that needs to be done to create this spatial internet is actually that 3D mapping of as much of the external world as possible, and also interior spaces too, but that's a whole other challenge. And then the actual segmentation, understanding this is a vertical wall and this is a horizontal floor, and mapping them respectively. And then once you do that, then the labeling. Okay, this is a house and that is a bench and that is a dog, and then being able to understand how they interact with each other. And then once you actually have this wise digital twin of the world, that's actually when you can start having those experiences that I was just talking about and creating the visual, seamless visual kinds of information that I was talking about.
Niantic was saying that they expect that foundational work of understanding the external world to take about the next two years. So we're really looking at about 2022 for the foundation, and you can really think of this as the third internet. This is the web underneath the third level of the internet. And so another two years for that. And then at the same time, we'll have, as you mentioned, the 5G networks developing, which actually helps the processing on the devices move into the network, which means the devices become smaller and lighter, and eventually we do have, you know, normal-looking glasses. Now that's not going to be in 2022, that's probably gonna be 2025 and later. But we really are moving towards a future where we start moving away from smartphones and towards a much more integrated view of the world, where again, we have this merging of the digital world and information in it and entertainment and the real world around us, taking place in devices that are smaller and lighter than the devices that we have now, ultimately moving into head-mounted devices. So that's really the vision.
MH: And that is sort of like the early stages of the concept of the singularity where eventually, we will just have all this hardware embedded in ourselves and it will become part of us.
LS: Yeah, that's a whole other thing and of course, there are tons of companies that are actually working on that. Yes, yes.
MH: And it all started with COVID-19.
LS: Exactly, exactly. And HTC, for example, has a really compelling vision. Of course, HTC makes the Vive VR headset. They actually see it becoming a sliding scale. So it's not that VR and AR are separate, because today they are. AR you can actually still see the real world around you, but VR is where the world is totally created and what you've got over your eyes completely blanks out the real world.
HTC's vision, so to speak, is a video pass-through. So you're wearing something that does actually cover your eyes, but it's actually giving you a video of the world around you. So you're still able to navigate within the real world. And so if you want to actually see everything as it is, great, you have full video pass through. If you want to start incorporating some digitization aspects of it, then fine. It's easier to actually integrate that with a digital feed of the video, rather than having something that you're overlaying onto a true, clear glass view of the real world. And if you wanna switch to full VR, go for it. Having this concept of pass-through video, enabling a sliding scale of seeing the real world to seeing some digitization or full digitization, I think that's a really important concept that we're gonna be seeing coming out as well.
MH: I think if I learned nothing more than this one point, it's that at least now I've got something to talk to my kid about, about Crossy.
LS: Yeah, you can talk to your kid about all this today. It might be stuff they haven't heard about, but if you wait until tomorrow, they'll be telling you about it.
If we don't have end-user control and the things that go with it, then we actually have a problem.
MH: That's kind of the interesting thing about all of this is that we are seeing a generation, and I'm broadly generalizing here, of boomers and gen Xers who previously never leveraged this type of technology that millennials and gen Z are currently doing. It's the kids bringing mom and dad along with them into the future.
LS: Oh, absolutely, absolutely. I mean, my kids have weathered the lockdown very well. They do their school in the morning, and then they just start gaming right after that and you know, they haven't really noticed they can't leave the house because they're just loving being able to spend all their time online with their friends in their games. My kids already had their main social interactions with their friends in the games. It's just the rest of us that are now catching up.
MH: What's your take on Peak Zoom? I gotta tell you a lot of these calls don't need video. And a lot of these calls don't need audio either, they could have been emails.
LS: Oh, you know, I have to say, I love Zoom. Now, you and I are actually speaking to each other right now in Zoom, and to be honest, I haven't turned on the camera because it's early in the morning and I'm in my gym clothes. So this is one of the fun things about Zoom is having to like brush your hair before answering the phone kind of thing.
MH: This is what we were promised during the Jet Age is that everybody would be doing video call conferences, but this comes back to the idea of predicting the future back in the '50s when they thought that we would be doing these types of things. At the end of the day, it all turns out, we really don't want to see each other.
LS: Well, you know, actually we do. We do because the call immediately before this was a group of people and these were all people that I don't know, I've never met them in person. And it was so helpful to be able to see the faces and particularly the dynamics of a group conversation.
If you can see the faces, then you know who has something to say and who's, you know, who wants to be able to speak next. And you can see who's maybe dissatisfied with the way the conversation is going. All of these dynamics that actually get missed when there's nothing but silence coming from some of the people that are on there. So I think being able to see the faces is actually extremely important. And when I give presentations online, especially to my customers, it's really hard to not be able to see the audience and to know what the feeling in the room is as you're speaking the words. And so I really appreciate the face thing.
However, the really important point, and this comes back to all the technologies that I'm talking about. You have to give the end-user control because, you know, if I'm in my gym clothes, no, you don't get to see me. I have to be able to control that. And we need to be able to take that end-user control paradigm all the way through all of these other, you know, exciting, but also frightening new technologies that I'm talking about. Because without that, if we don't have end-user control and the things that go with it, the privacy, the security, the basic fundamental respect for people as human beings, then we actually have a problem.