Father of virtual reality sees tech’s dark side, Facebook & Google as behavior modification empires
, New York Times|
Updated: Nov 09, 2017, 12.10 PM IST
Jaron Lanier is the most unusual person I’ve ever met. And I’ve met a lot of unusual people. A barefoot Buddha with dreadlocks, perched in a crazy fun house in the leafy hills of Berkeley, Lanier is a founding member of the digerati. The 57-year-old computer scientist, musician and writer has been christened the father of virtual reality.
"I’m a professional illusionist," he says. "In some ways, I might know more about making illusions than anybody."
Lanier is one of the few prophets who admits that the spawn of Silicon Valley could become evil, but he tries to stay on the sunny side. It helps that he avoids all social media.
"The popular ones are designed for behavior modification," he says, wearing his usual black T-shirt and black pants. "It’s like, why would you go sign up for an evil hypnotist who’s explicitly saying that his whole purpose is to get you to do things that people have paid him to get you to do, but he won’t tell you who they are?"
At this moment when dark clouds loom over Silicon Valley, Lanier is able to talk about the Lords of the Cloud with affection yet candor, as he worries that these tech gods creating new worlds may be getting “high on their own supply.”
"This is such a scary time, isn’t it?” he says, in his sweet, breathy voice. “I mean, it is for me. I had always feared we would create this social-manipulation technology out of computers.”
In his forthcoming memoir, Dawn of the New Everything: Encounters With Reality and Virtual Reality , the Microsoft wizard enthuses that VR “weds the nerdy thing with the hippie mystic thing,” high-tech but like a dream and “an elixir of unbounded experience.”
But he’s well aware of the “Matrix” dangers. He realized early on, he writes, that “it could turn out to be the evilest invention of all time.”
It’s a pretty simple proposition, he tells me: “If you control the person’s reality, you control the person.” Or as he writes in the book, “Never has a medium been so potent for beauty and so vulnerable to creepiness.”
Recently, the creepiness has been on display. Mark Zuckerberg stumbled into more trouble for tone-deafness when he used his cartoon avatar to take a disaster adventure trip, a “magical” 360-degree virtual reality tour of hurricane devastation in Puerto Rico to promote his new Facebook Spaces app.
“Oh, God,” Lanier said when he saw it. “It is scary and awful how out of touch Silicon Valley people have become.” I ask about social-media sites getting hijacked by Russians pushing propaganda aimed at putting Donald Trump in the White House. Vanity Fair has compared this juncture, with anxious lawmakers demanding accountability from the resistant tech companies, to the moment when we all had to start taking off our shoes at airports.
“Expect some smelly feet,” Lanier says. Unlike many here, he does not think of humans as ants in his experiments. “Hopefully, in this period, when we’re dealing with this really crude and early stuff like Facebook feeds, Instagram, Snapchat,” he says, “we’ll be able to get the politics straight and find a path for people to have dignity and autonomy before the hard-core stuff comes. Unless we all kill ourselves through this other stuff, which is a possibility, too. One of the great joys of the Trump era is having your 11-year-old say, ‘The former head of NATO said there’s a 1 in 10 chance of nuclear war. Is that right?’ And I’m like, ‘Oh, great. Thank you, Trump. That’s very nice.'”
Lanier believes that Facebook and Google, with their “top-down control schemes,” should be called “Behavior Modification Empires.”
“The whole internet thing was supposed to create the world’s best information resource in all of history,” he says. “Everything would be made visible. And instead we’re living in this time of total opacity where you don’t know why you see the news you see. You don’t know if it’s the same news hat someone else sees. You don’t know who made it be that way. You don’t know who’s paid to change what you see. Everything is totally obscure in a profound way that it never was before.
“And the belief system of Silicon Valley is so thick that my friends at Facebook simply still really believe that the answer to any problem is to do more of what they already did, that they’re optimizing the world.
“The Facebook business model is mass behavior modification for pay. And for those who are not giving Facebook money, the only — and I want to emphasize, the only, underlined and in bold and italics — reward they can get or positive feedback is just getting attention. And if you have a system where the only possible prize is getting more attention, then you call that system Christmas for Asses, right? It’s a creep-amplification device.
“Once Facebook becomes ubiquitous, it’s a sort of giant protection racket, where, if you don’t pay them money, then someone else will pay to modify the behavior to your disadvantage, so everyone has to pay money just to stay at equilibrium where they would have been otherwise,” he says. “I mean, there’s only one way out for Facebook, which is to change its business model. Unless Facebook changes, we’ll just have to trust Facebook for any future election result. Because they do apparently have the ability to change them. Or at least change the close ones.”
Why would Facebook change its business model when it’s raking in billions?
“I would appeal to the decency of the people in it,” he replies. “And if not to them, then the toughness of the regulators. It’s going to be one of the struggles of the century.”
He continues, “I think there are a lot of good people at Facebook, and I don’t think they’re evil as individuals. Or at least not the ones that I’ve met. And I know Google a lot better, and I feel pretty certain that they’re not evil. But both of these companies have this behavior-manipulation business plan, which is just not something the world can sustain at that scale. It just makes everything crazy.”
I remark that Facebook’s Sheryl Sandberg expressed surprise that their fiendish little invention could do something so nefarious.
“People in the community knew,” Lanier says, adding that he wrote essays and participated in debates in the early 1990s about how easy it would be to create unreality and manipulate society, how you could put out a feed of information that would put people in illusory worlds where they thought they had sought out the information but actually they had been guided “the way a magician forces a card.”
“So for somebody to say they didn’t know the algorithms could do that,” Lanier says in a disbelieving tone. “If somebody didn’t know, they should’ve known.”
So what happens when fake news marries virtual reality?
“It could be much more significant,” Lanier says. “When you look at all the ways of manipulating people that you can do with just a crude thing like a Facebook feed — when people are just looking at images and text on their phones and they’re not really inside synthetic worlds yet — when you can do it with virtual reality, it’s like total control of the person. So what I’m hoping is that we’re going to figure this stuff out so we don’t make ourselves insane before virtual reality becomes mature.”
He says that Silicon Valley has turned out both better and worse than he expected: “As far as the worse part, creating a global behavior-modification empire is worse than I thought. And creating a world that’s more opaque instead of less opaque is worse than I thought we should do. It’s also a physically uglier place than I thought it would be. It’s really a shame. If we’re the new Renaissance, why don’t we make this amazing Tuscany here? We have these gorgeous orchards. Why don’t we do something beautiful here instead of just filling it up with parking lots and horrible buildings?”
He says sometimes his peers in the Valley seem perfectly nice but then they will say something “I just can’t believe.” He cites Eric Schmidt’s comment on privacy on CNBC’s “Inside the Mind of Google” special in 2009, that “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.”
“Really?” Lanier asks. “It does give me this feeling sometimes that something’s going wrong with our culture in Silicon Valley and maybe it’s just that thing of power corrupting and absolute power corrupting absolutely, just losing perspective. Like Zuckerberg might think, ‘Well, I went around to a bunch of states and I ate barbecue and wrestled cattle so I’ve been around all kinds of people.’ But to have people who you respect and listen to have fundamentally different worldviews and question your core logic and think that you may be way off track, that’s a much harder thing to do. And there are people who can be very powerful and comfortable with that. I’ll mention one, whose name is Barack Obama.”
I ask Lanier about the sexual harassment and gender inequity problems roiling the Valley.
“Well, sometimes, I think there’s a kind of emerging new male jerk persona of the digital age, which would be some kind of a cross between the Uber guy and the pharma bro and maybe Milo Yiannopoulos and maybe Palmer Luckey and maybe Steve Bannon,” he says. “Because, there’s this sort of smug, superior, ‘I’ve got the levers of power, and I know better than you.’ It’s sort of this weird combination of a lot of power and a lot of insecurity at the same time.” He believes that Gamergate led to the alt-right. “It was one of the feeders,” he says.
He talks about another personality that is emerging from the digital age.
“If you’re a mark of social media, if you’re being manipulated by it, one of the ways to tell is if there’s a certain kind of personality quality that overtakes you,” he says. “It’s been called the snowflake quality. People criticize liberal college kids who have it, but it’s exactly the same thing you see in Trump. It’s this kind of highly reactive, thin-skinned, outraged single-mindedness. I think one way to think of Trump, even though he is a con man and he is an actor and he’s a master manipulator and all that, in a sense he’s also a victim. I’ve met him a few times over 30 years. And what I think I see is someone who has moved from kind of a New York character who was in on his own joke to somebody who is completely freaked out and outraged and feeling like he is on the verge of a catastrophe every second. And so my theory about that is that he was ruined by social media.”
Lanier plays me a song he composed to cheer up his wife when she was going through cancer treatments, a Cuban-style charanga flute solo played on a Japanese shakuhachi — “which is a crazy-hard thing to do and I pulled it off.”
Then he confides his fear that one of his older cats, Loof, would have been a Trump supporter.
“Loof is the sweetest cat in the world but she’s really an anti-immigrant voter,” he says. “She did not like the idea of young cats coming here. She really didn’t want the change. She really felt like they ruined everything. And I must point out that the new kittens who came are black kittens. They appear to be Bombay cats. Loof is not in a basket of deplorables. She’s just in a basket of blankets.”
As I get ready to leave, the grand illusionist offers an intriguing theory about why the internet is more obsessed with cats than dogs.
“I think we know that Facebook is turning us into trained dogs,” he says. “We know we’re being trained. We can feel ourselves being turned into trained circus animals. And we long for that independence that cats show. So when you look at a cat video, what you’re really seeing is this receding identity that you want to cling to and find again.”
Run a Business? Get Your Free Business Listing on Economic Times. Click Here
"I’m a professional illusionist," he says. "In some ways, I might know more about making illusions than anybody."
Lanier is one of the few prophets who admits that the spawn of Silicon Valley could become evil, but he tries to stay on the sunny side. It helps that he avoids all social media.
"The popular ones are designed for behavior modification," he says, wearing his usual black T-shirt and black pants. "It’s like, why would you go sign up for an evil hypnotist who’s explicitly saying that his whole purpose is to get you to do things that people have paid him to get you to do, but he won’t tell you who they are?"
At this moment when dark clouds loom over Silicon Valley, Lanier is able to talk about the Lords of the Cloud with affection yet candor, as he worries that these tech gods creating new worlds may be getting “high on their own supply.”
"This is such a scary time, isn’t it?” he says, in his sweet, breathy voice. “I mean, it is for me. I had always feared we would create this social-manipulation technology out of computers.”
In his forthcoming memoir, Dawn of the New Everything: Encounters With Reality and Virtual Reality , the Microsoft wizard enthuses that VR “weds the nerdy thing with the hippie mystic thing,” high-tech but like a dream and “an elixir of unbounded experience.”
But he’s well aware of the “Matrix” dangers. He realized early on, he writes, that “it could turn out to be the evilest invention of all time.”
It’s a pretty simple proposition, he tells me: “If you control the person’s reality, you control the person.” Or as he writes in the book, “Never has a medium been so potent for beauty and so vulnerable to creepiness.”
Recently, the creepiness has been on display. Mark Zuckerberg stumbled into more trouble for tone-deafness when he used his cartoon avatar to take a disaster adventure trip, a “magical” 360-degree virtual reality tour of hurricane devastation in Puerto Rico to promote his new Facebook Spaces app.
“Oh, God,” Lanier said when he saw it. “It is scary and awful how out of touch Silicon Valley people have become.” I ask about social-media sites getting hijacked by Russians pushing propaganda aimed at putting Donald Trump in the White House. Vanity Fair has compared this juncture, with anxious lawmakers demanding accountability from the resistant tech companies, to the moment when we all had to start taking off our shoes at airports.
“Expect some smelly feet,” Lanier says. Unlike many here, he does not think of humans as ants in his experiments. “Hopefully, in this period, when we’re dealing with this really crude and early stuff like Facebook feeds, Instagram, Snapchat,” he says, “we’ll be able to get the politics straight and find a path for people to have dignity and autonomy before the hard-core stuff comes. Unless we all kill ourselves through this other stuff, which is a possibility, too. One of the great joys of the Trump era is having your 11-year-old say, ‘The former head of NATO said there’s a 1 in 10 chance of nuclear war. Is that right?’ And I’m like, ‘Oh, great. Thank you, Trump. That’s very nice.'”
Lanier believes that Facebook and Google, with their “top-down control schemes,” should be called “Behavior Modification Empires.”
“The whole internet thing was supposed to create the world’s best information resource in all of history,” he says. “Everything would be made visible. And instead we’re living in this time of total opacity where you don’t know why you see the news you see. You don’t know if it’s the same news hat someone else sees. You don’t know who made it be that way. You don’t know who’s paid to change what you see. Everything is totally obscure in a profound way that it never was before.
“And the belief system of Silicon Valley is so thick that my friends at Facebook simply still really believe that the answer to any problem is to do more of what they already did, that they’re optimizing the world.
“The Facebook business model is mass behavior modification for pay. And for those who are not giving Facebook money, the only — and I want to emphasize, the only, underlined and in bold and italics — reward they can get or positive feedback is just getting attention. And if you have a system where the only possible prize is getting more attention, then you call that system Christmas for Asses, right? It’s a creep-amplification device.
“Once Facebook becomes ubiquitous, it’s a sort of giant protection racket, where, if you don’t pay them money, then someone else will pay to modify the behavior to your disadvantage, so everyone has to pay money just to stay at equilibrium where they would have been otherwise,” he says. “I mean, there’s only one way out for Facebook, which is to change its business model. Unless Facebook changes, we’ll just have to trust Facebook for any future election result. Because they do apparently have the ability to change them. Or at least change the close ones.”
Why would Facebook change its business model when it’s raking in billions?
“I would appeal to the decency of the people in it,” he replies. “And if not to them, then the toughness of the regulators. It’s going to be one of the struggles of the century.”
He continues, “I think there are a lot of good people at Facebook, and I don’t think they’re evil as individuals. Or at least not the ones that I’ve met. And I know Google a lot better, and I feel pretty certain that they’re not evil. But both of these companies have this behavior-manipulation business plan, which is just not something the world can sustain at that scale. It just makes everything crazy.”
I remark that Facebook’s Sheryl Sandberg expressed surprise that their fiendish little invention could do something so nefarious.
“People in the community knew,” Lanier says, adding that he wrote essays and participated in debates in the early 1990s about how easy it would be to create unreality and manipulate society, how you could put out a feed of information that would put people in illusory worlds where they thought they had sought out the information but actually they had been guided “the way a magician forces a card.”
“So for somebody to say they didn’t know the algorithms could do that,” Lanier says in a disbelieving tone. “If somebody didn’t know, they should’ve known.”
So what happens when fake news marries virtual reality?
“It could be much more significant,” Lanier says. “When you look at all the ways of manipulating people that you can do with just a crude thing like a Facebook feed — when people are just looking at images and text on their phones and they’re not really inside synthetic worlds yet — when you can do it with virtual reality, it’s like total control of the person. So what I’m hoping is that we’re going to figure this stuff out so we don’t make ourselves insane before virtual reality becomes mature.”
He says that Silicon Valley has turned out both better and worse than he expected: “As far as the worse part, creating a global behavior-modification empire is worse than I thought. And creating a world that’s more opaque instead of less opaque is worse than I thought we should do. It’s also a physically uglier place than I thought it would be. It’s really a shame. If we’re the new Renaissance, why don’t we make this amazing Tuscany here? We have these gorgeous orchards. Why don’t we do something beautiful here instead of just filling it up with parking lots and horrible buildings?”
He says sometimes his peers in the Valley seem perfectly nice but then they will say something “I just can’t believe.” He cites Eric Schmidt’s comment on privacy on CNBC’s “Inside the Mind of Google” special in 2009, that “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.”
“Really?” Lanier asks. “It does give me this feeling sometimes that something’s going wrong with our culture in Silicon Valley and maybe it’s just that thing of power corrupting and absolute power corrupting absolutely, just losing perspective. Like Zuckerberg might think, ‘Well, I went around to a bunch of states and I ate barbecue and wrestled cattle so I’ve been around all kinds of people.’ But to have people who you respect and listen to have fundamentally different worldviews and question your core logic and think that you may be way off track, that’s a much harder thing to do. And there are people who can be very powerful and comfortable with that. I’ll mention one, whose name is Barack Obama.”
I ask Lanier about the sexual harassment and gender inequity problems roiling the Valley.
“Well, sometimes, I think there’s a kind of emerging new male jerk persona of the digital age, which would be some kind of a cross between the Uber guy and the pharma bro and maybe Milo Yiannopoulos and maybe Palmer Luckey and maybe Steve Bannon,” he says. “Because, there’s this sort of smug, superior, ‘I’ve got the levers of power, and I know better than you.’ It’s sort of this weird combination of a lot of power and a lot of insecurity at the same time.” He believes that Gamergate led to the alt-right. “It was one of the feeders,” he says.
He talks about another personality that is emerging from the digital age.
“If you’re a mark of social media, if you’re being manipulated by it, one of the ways to tell is if there’s a certain kind of personality quality that overtakes you,” he says. “It’s been called the snowflake quality. People criticize liberal college kids who have it, but it’s exactly the same thing you see in Trump. It’s this kind of highly reactive, thin-skinned, outraged single-mindedness. I think one way to think of Trump, even though he is a con man and he is an actor and he’s a master manipulator and all that, in a sense he’s also a victim. I’ve met him a few times over 30 years. And what I think I see is someone who has moved from kind of a New York character who was in on his own joke to somebody who is completely freaked out and outraged and feeling like he is on the verge of a catastrophe every second. And so my theory about that is that he was ruined by social media.”
Lanier plays me a song he composed to cheer up his wife when she was going through cancer treatments, a Cuban-style charanga flute solo played on a Japanese shakuhachi — “which is a crazy-hard thing to do and I pulled it off.”
Then he confides his fear that one of his older cats, Loof, would have been a Trump supporter.
“Loof is the sweetest cat in the world but she’s really an anti-immigrant voter,” he says. “She did not like the idea of young cats coming here. She really didn’t want the change. She really felt like they ruined everything. And I must point out that the new kittens who came are black kittens. They appear to be Bombay cats. Loof is not in a basket of deplorables. She’s just in a basket of blankets.”
As I get ready to leave, the grand illusionist offers an intriguing theory about why the internet is more obsessed with cats than dogs.
“I think we know that Facebook is turning us into trained dogs,” he says. “We know we’re being trained. We can feel ourselves being turned into trained circus animals. And we long for that independence that cats show. So when you look at a cat video, what you’re really seeing is this receding identity that you want to cling to and find again.”
Run a Business? Get Your Free Business Listing on Economic Times. Click Here
