Revealing Reality in Imaginary Worlds

Revealing Reality in Imaginary Worlds — Part 1

A two-part series on how storytellers are using VR to create imaginary worlds where participants must uncover the truth

Edited on April 27, 2023

A city on water with floating buildings, bridges and gondolas, and canals for thoroughfares, it’s fitting that Venice is home to Venice Immersive, a magnificent display of 3D worlds explored through VR headsets. In VR, you find yourself transported to a colorful array of worlds, from animated Ancient Egyptian tombs to fantasy worlds like Gumball Lounge. By yourself or with others, your job is to explore these worlds and find the stories. Some worlds are completely fantastical while others are simulated real worlds created using data capture techniques such as photogrammetry or lidar scanning. All are created in game engines and use a conglomeration of languages and grammars from video games, immersive theater, documentary, film, dance and/or architecture among others.

At the intersection of these fields, new languages and grammars are bubbling up. Using real time 3D game engines, stories move forward with techniques such as treasure hunts, live actors and virtual scenes mapped onto physical sets that participants wander through, sometimes revealing what is real along the way. The scene changes depending on where the participant moves their hands and eyes. Documentaries at Venice Immersive ranged from photo real to animated.

I sat down with two artists who created imaginary worlds and characters to engage audiences with pressing real world issues. Participants had to discover the non-fiction content by making their way through the world and learning its language. What follows is part one of a three-part series highlighting each of these artists and the worlds they built.

“Fight Back” by Celine Tricart

Celine Tricart is no stranger to Virtual Reality having created the award-winning project “The Key” (2019) which won the Grand Jury Prize at Venice in 2019. She began her career as a filmmaker but now easily moves between film and VR. In our conversation, she talks about how difficult it is for filmmakers to work in VR who are used to framing and editing the scenes to control what you see.

Celine Tricart: “I think I understand the difference between film and VR, and a lot of people coming from film struggle with that difference, of being a first-person medium and having no in between, no transformation of your perception between you and the story. And so the way you tell the story, it’s completely different in film than it is in virtual reality. And it’s also very different in video games. So you just have to adapt.”

“Fight Back” is in the form of a game in which you have to liberate a certain number of stars from shadows. The project is about gender violence and the shadows are the participants’ own fears that they have to conquer. Each star that you liberate teaches you a move in self defense. You have to do the movement correctly (measured through hand tracking) in order to advance. Tricart shares, “They [the moves] are very basic. They are like stage one of self-defense, but it’s a first, very important step. And through the game you get to do those gestures over and over and over again. The game is about 40 minutes long. And over those 40 minutes, you create the first muscle memory and reflexes in your brain.”

“I’m trying to see everything through the eyes of my future players or participants. I use participants in the case of experiences and players if it’s games. It’s very important not to think from the position of the director or the storyteller. That’d be like, ‘Okay, people know nothing about my world and my story. They will put a VR headset on. What will they see first, and how are we going to guide them through the experience?’ It’s creating a world around them and guiding them gently through the experience but really trying to imagine what they will do and what they will see and how will they feel in that world that you create.”

“I think people who have experience in immersive theater, for example, really understand what it means to do storytelling for VR because they have to follow similar kinds of rules. And also, I’m a LARPer [live action role playing gamer]. I’ve been doing LARPs my whole life since I was a teenager. And this is a form of storytelling that I find fascinating that also prepared me and trained me into thinking for VR.”

VR’s use as a training tool is well documented, from medical procedures to training as a pilot. In its novel adaptations to immersive media there are still a lot of glitches.

We realized that hand tracking was not exactly ready at this moment in time. There’s a couple of things it’s very good at and there’s a lot of things that it is not very good at, for example, fast movement. Fast movement is really bad for hand tracking, and it’s kind of difficult when you do a game where you are actually fighting to not get too excited and move really fast. So we have to constantly remind people to stay relaxed, stay calm, and make slow and precise gestures at the right timing. So it’s more of an exercise in self-control. We had to transform that a little bit. And also originally, we had a lot of conditions to make the gesture work, to try to make people do a perfect gesture. And then we realized that this was making the gesture way too complicated, way too difficult.

The genesis of the project comes from her experience filming Yazidi women of Iraq, also known as Sun Ladies, fighting at the frontlines against ISIS. She is a martial arts practitioner and, in her own research and work, found a correlation between self defense and these women’s empowerment, especially in the context of healing from sexual assault. There was something empowering about the physical activity, holding a safe space, and practicing boundary setting. Tricart began to wonder, “How can we rewire our brain to understand that we deserve to be here? We deserve to hold our space, that we are physical beings, and we can say no.” While inspired initially by the healing journeys of Yazidi women, throughout our interview, Tricart stressed the importance and applicability of self-defense training for all genders and that anyone can face gendered violence.

Tricart often uses the language of visual metaphor, emphasizing real world issues in the foreground and backdrop of her immersive work. In “Fight Back”, she creates a celestial universe full of stars and only at the end do you learn that the stars represent various female historical figures who “fought back.” → As a slight spoiler, the stars come to life as actual historical figures and you learn their stories.

The vast universe filled with invisible stars is a poignant metaphor for women throughout history and today that fight back but lack recognition. “We are just shining a light onto the stories of those women. We picked five but there are thousands and thousands of them.” She talks about how many participants finished in tears of happiness as they learned about these powerful historical women and saw a glimmer of their own power.

Tricart hoped that people emerge transformed not only by the story but by the moves they have learned. Perhaps, they would walk a little differently, a little more forcefully, out of the virtual world and into the real one. “ Just having that moment when you punch and you have that *POW* and seeing how powerful you are. That’s profound.”

For more news, discourse, and resources on immersive and emerging forms of nonfiction media, sign up for our monthly newsletter.

Immerse is an initiative of the MIT Open DocLab and Dot Connector Studio, and receives funding from Just Films | Ford Foundation, the MacArthur Foundation, and the National Endowment for the Arts. The Gotham Film & Media Institute is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing emerging nonfiction projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.