What I Learned About Reality in VR

Eva Wolfangel is currently a Knight Science Journalism Fellow at MIT focused on interactive journalistic research in a social VR space.

Virtual reality tore me out of my filter bubble. I met people I probably would never have met in material reality. I felt close to people, I hung out with them in a social VR room for weeks — and only on my long journey to Kuwait, Israel, and the US, where I finally visited them in their material lives, did I realize how far away these people are “in reality.” But this distance did not hinder us from having real relationships.

This experience changed my idea about reality — and it changed my work. As a journalist, I can’t help but see great opportunities for our industry in this medium. But journalists and communicators urgently need to become more creative in the use of VR. After all, VR offers much more than 360 videos in which — let’s face it — usually only one perspective is interesting. And all those passive “experiences” in VR do not use its full potential.

Why should our audience watch things silently and mostly passively, when they could be social and interactive? And what are the things we should work on now?

To understand that, I would first like to take you on a journey. On a journey that led me into the lives of different people and into my own brain. At the end of this article, you will find more details about my project: interactive journalistic research in a social VR space.

The journey began on a warm spring evening in June 2016:

I feel the sun on my skin, and it feels warm — even though it is late at night. It cannot be true and yet it is true. I feel it.

This was my first impression when I arrived in another reality for the first time. At first glance, everything seems to be familiar: It looks like Earth. I am standing under a tree in the evening sun. The shade of the leaves draws a pattern on the ground. I hear the murmuring of people around me. The sky is immense.

In reality, my body was standing in my small dark office, and yet my mind was somewhere else. I put on a bulky headset, and suddenly I found myself in this other world, another reality. This reality is called “Altspace VR” and it is a social meeting place in Virtual Reality. This is not about games, it’s about meeting people. I am not a gamer, and that is why I came here: I love to be social and I am naturally curious.

I could hardly believe how real it all seemed. At this point I did not yet know how much this research would change my life. It would lead me into new realities, into the real lives of other people.

What is real?

In The Matrix, life takes place in a parallel world that doesn’t really exist. It’s the brain’s imagination. The people in The Matrix don’t know this because the parallel world feels so real. The world I was in at that moment also didn’t not really exist, but I had to tell myself that again and again as I enjoyed the evening atmosphere that day.

Some researchers say that with virtual reality we have the chance to choose between several realities for the first time in history — beyond fantasy trips, delusions, or drug trips. “Virtual reality is as real as the physical world,” said philosopher David Chalmers in an interview. Chalmers has been working for years on consciousness and the perception of reality.

How do we know that the “real” world around us is not just a simulation? René Descartes conducted a thought experiment in the 17th century, when he asked: How can we know that we are not controlled by a demon who merely makes the world around us seem real? In fact: we cannot know.

Virtual reality forces us to think about this question again. The feeling of being in the virtual space is so strong that doubts arise as to whether there are any relevant differences to the real world. Chalmers clearly says no: “Virtual reality is not a second-class reality.”

On my first day in the other reality. I felt like I was rooted to the virtual floor as a large man came towards me, stepping much too close. I couldn’t move backwards because behind me there was a staircase going down — a virtual one, but in this situation it felt real. The man grabbed my chest and gave a dirty laugh. I froze. I knew it wasn’t my body, it’s “only an avatar.” He didn’t really touch me. But when I looked down at myself in virtual reality, I saw my avatar — and it felt like me — and I saw this man’s hand, touching it.

This key experience made me unpleasantly aware that virtual reality may not be material, but it is real. Compared to chats, video conferences or online games on a 2-D screen, virtual reality is a quantum leap. In VR, I am right in the middle of it, not just viewing it on a two-dimensional screen. The experiences are close, very close. I asked other women I met in VR and found out: sexual harassment in virtual reality is a frequent phenomenon.

But there were nice things about having a body in VR as well. When my friend Sana hugged me for the first time, I could feel her breath on my neck, I could feel the warmth of her body and I could feel her arms as a fine tingling sensation on my back. That couldn’t be true! Or can it?

To find out, I visited Thomas Metzinger, a philosopher at the University of Mainz in Germany. For many years he has been studying human perception. He points to the red sofa in the corner of his office. “Is that real?” he asked. He waited, then shook his head. “This red of the sofa, the colours, these are characteristics of a model in our brain.”

The red is created in the mind of each individual. What we experience is far less real than we think. “The fact that something feels real means that the brain produces a model with a high degree of predictive accuracy,” says Metzinger. We step under the cone of a lamp and now see everything brighter, as expected. We can hear people from far away and the closer they come, the louder their voices become.

I sat down on the red sofa, and it was soft, exactly as I expected. So it must be real.

And this is also the case with our sense of self, Metzinger explained: “According to my theory, the feeling of being yourself is a simulation of the brain, an inner model with many layers.” The brain calculates from all the available information the best hypothesis, he explains, the most probable variant of reality. The brain presents this hypothesis to us as reality. “If we do it cleverly, it’s quite possible that you might think you’re in another body.”

As early as 2007, Metzinger and his colleagues transferred test persons into a virtual body during an experiment. Back then, they created a video of the test person from behind. The person saw the video from him or herself in front of his or her own body. Then the researchers stroked the participant’s back. At the same time, the test person saw how the virtual body in front of him or her was also stroked. Thus, Metzinger said, the brain begins to believe that the avatar is somehow part of one’s body.

Thomas Metzinger speaks of a “myth of authenticity.” “Our real world is, of course, not real, but completely distorted.” That does not mean necessarily that there are no physical bodies and no outside world. But it does question the claim that our material reality is the only reality.

And I felt in my own body what Metzinger says: When I put on the VR headset, my body stayed in my little dark study. But my consciousness was beamed into another body in another world — if there is something like consciousness at all (but that is another question). One day, for example, I found myself in Sana’s VR room — she calls it “Time Machine.” She has sad poems on the walls and, in one corner, an open fireplace. I felt the warmth of the fire (at least my brain told me I did) and I heard the warmth in Sana’s voice. She sounded a bit sad.

At this fireplace we philosophized night after night — me, the German science journalist and her, the widow from Kuwait, a strictly religious Muslim. In those moments, I was really there with her in a room. I enjoyed those evenings with Sana. I like her, and we learned a lot from each other. Would we ever have met in so-called real life? Probably not.

I realized that this parallel world, into which I immersed myself deeper and deeper, was as much my real life as the material version that most of us call reality. It changed my real life. The melancholy after a thoughtful virtual evening with Sana mixed into my real next days. I worried about her when she said goodbye in a particularly sad way, and I felt the warmth of our encounter the very next day.

Going deeper

How is it possible that VR is both “virtual” and still manages to trigger such powerful feelings? I wanted to learn more about this and so I moved further into this reality. Every free minute I put on the headset and the headphones, my mind left the real world and moved to the virtual one.

Sometimes this became adventurous. Together with different people, I explored the many different rooms in Altspace. We beamed from one place to another, we flew, we wandered through a maze and fought with swords. When I swung a sword in the tavern, I swung my arm with the controller in my study. If another fighter broke through my cover, I ducked away and cowered on the floor — which, for me, were creaky wooden boards of the tavern. Luckily, I thought, nobody is watching in this real world.

Once I was standing on a high rock in front of hundreds of meters of nothing. I turned my head carefully. Behind me was a wall. There was no way to escape. I trembled and I could not move my feet. For a moment I thought of my other world, the safe one, where I was standing on a solid floor. But this idea did not help at all. The view from the rock felt more real. My body signalled: Danger.

What triggered my fear of heights is what researchers call “place illusion.” The feeling of being real on the spot is not dependent on perfect graphics. Even if the place does not look like it would look in the real world, my brain completes everything that is missing. The fact that the avatars in Altspace look more like robots is not a problem. On the contrary, it makes it easier for the brain to concentrate on the essential: The interaction between me and other people and the virtual environment is important for the “feeling of here.”

This is at least what scientists claimed after initial studies. Keisuke Suzuki from the Sackler Centre for Consciousness Science at the University of Sussex let users try different virtual worlds and found that “as soon as you can interact with people or things, it feels real.” Because our brain can test its hypotheses, such as whether everything looks brighter when I go closer to the lamp. “The brain wants a coherent story,” says Suzuki.

This has the potential to help others. Experts in exposure therapy, for example, are using VR to treat fear of heights and other phobias.

One of my virtual friends, Crystal, has social phobia in real life, but in virtual reality she is one of the most networked people I have ever met. Everybody knows her and she organizes popular 48-hour parties in Altspace. She even has built a virtual camping site with a starry sky for guests — because you might want to rest when it is night in your timezone and you are at a 48-hour-party.

The more time I spent in the virtual world, the more I felt doubts — and my friends and family were amplifying them. “You spend so much time in this virtual reality. You don’t even know these people.” “How do you know they’re not lying to you?” I grew defiant. Of course I know Sana! I hear her voice, I see her gestures, we have talked for nights on end — I am not naive after all!

I decided to prove it. “Sana, I’ll come and visit you in real life!” But my friend didn’t seem to be very excited about this idea. She didn’t show up in her virtual room for days. Finally, I convinced her and I booked a flight to Kuwait City. A few days later, she hugged me for the first time in the material world. And this hug, in a hotel lobby under heavily rattling air conditioners, felt familiar. Everything was like in virtual reality. Her voice. Her gestures. I felt her warm body.

The only thing that was unfamiliar was her headscarf. She does not wear it in VR.

In those three days in Kuwait City I often thought how important these new realities are for people like Sana. Sana told me that she is not allowed to meet men because of her religion. She also has the feeling that she does not belong in this world. Normally, she keeps to herself. I felt this during my visit: she wanted to show me Kuwait City, but we kept getting lost. Normally, without a visitor, she refuses this reality by not leaving her house unless it is absolutely necessary. But now she had a visitor, and she felt obliged to show me her hometown. “I don’t belong here,” she says again and again. It is different in virtual reality: There she feels at home.

Her loneliness became even clearer when we stood in front of a plaque of honor for the fallen heroes of the 1990 Gulf War. Sana points to a young man. Her husband. He fought against Saddam Hussein and died. “I miss him so much,” says Sana.

I don’t know what to say.

At the airport, as I left, I asked, “Sana, why didn’t you want me to come?”

“I have no friends in real life, and I was afraid of what would happen if you came into my real life now.” She worried about losing a friend as soon as she transitioned to the material world. But she did not. We had a great time together.

Back in my real life and, thus, within reach of my VR headset, I got to know Cattz. He is a dude, not especially polite, and always some rough words on his lips. Sana thinks he is a lout. Cattz frequently talks about his three ex-wives, his five children,and his terminal illness. But he’s fun to be with. He is extremely helpful and shows me tricks in different virtual worlds. Together we steer a spaceship and swim in virtual pools.

But when I wanted to meet him in his “real” life, Cattz was not really happy with this idea. “Don’t come, I am ugly,” he wrote to me. I finally convinced him, and flew to Spokane, Washington, where I found him in a place that could hardly be more desolate. Cattz was lying in the cellar of a littered house on a stained mattress, eating nothing but toast, Coca- Cola, and heart pills. When I saw him in this depressing situation, I cursed my research. I felt out of place. Why was I intruding on his personal life? What was I looking for, digging into the real lives of my virtual friends?

Cattz’s real life is so depressing that it seems to me much more unreal than his funky, funny, and sometimes rough way of living in VR. It was hard to convince him to leave the mattress. I painstakingly reconstructed what happened: His house burned down, he had nothing left, no place to stay, no VR headset. He hitchhiked for months across the US in search of his meaning in life. But the material reality is not his reality. He doesn’t want to be here.

After weeks of homelessness, he finally rented this basement room. He could not afford anything else with the little money he had from early retirement. Walking the streets of Spokane with Cattz, who breathes heavily, sweats, and trembles quickly, I realized how important virtual reality was to him. There he can walk, fly, run, and beam. Here his body is tired and ill. His real life is in virtual reality. But VR locked him out. He lost his connection to his real life, because he lost his headset. He’s gonna have to save a long time before he can afford a new one — and go home.

Later, I celebrated the engagement of Ben and Shoo. They met and fell in love in virtual reality. Ben has built his beloved a virtual house — on the ground floor, a bar; on the counter, a cake; on the first floor,a romantic bed with dark red satin bedding and a warm glowing bedside lamp. After this engagement party, I decided to meet Ben in the material world.

When I met him in Atlanta, his mood was depressed. He had recently visited his fiancée in London for the first time in real life. She was not a stranger to him, but during the long hours in the plane above the clouds, he realized that he was having a long-distance relationship. He never felt the distance in the cuddly shared house in virtual reality. They were together in one room. But now he felt the distance. Or, in his words, “Reality sucks.”

“I don’t see why you shouldn’t be able to live a life in a virtual environment that is as meaningful, as fulfilling, at least in principle, as life in a non-virtual environment,” says philosopher David Chalmers. Some things are going to be harder: the role of the body, hunger, thirst, sex, birth and death, he says. “Some things are going to take a while to build into virtual reality, but I suspect within a century there will be Matrix-style virtual reality, which is more or less indistinguishable from our kind of reality.” And, for some people, this virtual reality could be even better, says his German colleague Thomas Metzinger: “It’s not the case that the reality we have now works particularly well.”

People fight each other, for example, or have different interpretations of real situations. And sometimes a virus paralyses the whole world. So why not move to another reality from time to time?

My project

I am currently working on a project to use social VR as journalists as a Knight Science Journalism Fellow at MIT. Educational research shows that we understand and learn things better and more sustainably when we explore them together and interactively. There are already educational projects in social virtual reality, but many of them are just replicating classic classrooms: School desks and tables, and the teacher at the front of the blackboard. But this doesn’t take advantage of the medium. Education in VR does not have to be top-down teaching.

My project is interactive journalistic research in a social VR space. As a journalist, I will interview an expert — say, a quantum physicist or a brain researcher, together with my audience. We would meet in a social VR space and have three-dimensional models with us: an atom, Schrödinger’s cat, or the human brain, for example. These things are not visible in real life. But in VR we can not only see them, we can interact with them. While I interview the expert as a journalist, we and the audience can travel together into the atom, open the box with Schrödinger’s cat, or fly through the brain and explore it. Everybody in the audience can fly like a neuron or separate two entangled quantum particles from each other, bringing them to different planets and see how entanglement works.

And there is more: This technology also makes us open towards new people and new ideas. It makes holes in filter bubbles. I have met many people, for example, who are quite skeptical about journalists, but who were amazingly open in our conversations. I have answered many questions, and I have asked many questions. I only have anecdotal evidence for this: my experiences and those of my virtual friends. But many have confirmed to me that they have talked to people with whom they would not come into contact in real life or who they are quite skeptical about.

Journalistic research and experience shows that one of the only things that reliably helps in the current crisis of confidence in the media is transparency and meeting people in person. Many publishers have created elaborate dialogue processes with readers. This is a side effect of my project: the audience can be part of a journalistic process, they can explore a topic together with a journalist. They can see how we work, we can debate about things like journalistic research and fact-checking. We can meet in person — without having to travel, without flying, without polluting the environment. Even without transmitting viruses.

And after the first experiences in “remote learning” at Harvard and at MIT in the face of the corona crisis, one thing is for sure: everything that helps to give this learning true spatiality and the feeling of really being together makes digital learning and communication over distance more successful.

We should be optimistic about virtual reality.

Immerse is an initiative of the MIT Open DocLab and The Fledgling Fund, and it receives funding from Just Films | Ford Foundation and the MacArthur Foundation. IFP is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing media projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.