Illustrating Reality in a 3D World

Illustrating Reality in an Imaginary 3D World

A behind-the-scenes look at Pedro Harres’ interactive and immersive animated film From The Main Square

It’s not often that a student thesis takes home a prize at an A-list film festival but such was the case with From The Main Square, an immersive experience by Brazilian director Pedro Harres. It won the Immersive Grand Jury Prize at Venice Film Festival in 2022 and more recently, the grand prize at New Images Festival in France.

The project has traveled far in the recent festival circuit. I first encountered From the Main Square at Venice Immersive and was taken by the imaginary world that unfolded around me and the profound truths that it revealed. There, I sat down with Harres, for a fascinating discussion about the project’s origins, his intentions and process and the teamwork involved. I encountered it again at South by Southwest in March, where the team offered some of the original drawings on paper at the end of the exhibit. The drawings were brimming with life and imagination, while also documenting a very specific time and place. The art director and illustrator Dani Eizirik drew many of them on site — in the woods and in the cities — lending the project another level of veracity.

Seeds of the concept

The film’s origins started ten years ago when Harres was living in Brazil and became deeply disturbed by the vitriolic comments unfolding online, which he saw as a bad harbinger. He set out to make an animated film that showed a “portrait of social disruption,” highlighting the risks of social polarization.

His original vision was of a single building with inhabitants fighting over its structure, each trying to modify it in their own way until the building collapsed. To bring this project to life, he wrote a script for Eizirik to illustrate, hoping to apply for a grant. The night before the application was due, Eizirik sent him an unexpected drawing — of a city, not a building.

While they didn’t get the grant, the idea of a polarized city stayed with Harres. Years later when he was a Masters student in animation direction at Babelsberg Film University in Potsdam, he saw an opportunity to apply for funding for a VR film. He was in a support structure that allowed for experimentation and thought this could be the moment to try something new. The unexpected drawing of the city that his art director had presented to him years ago would be the perfect start to explore an immersive project he decided. In this film, the city would rise and fall as the polarized inhabitants turned to beating each other up and destroying it.

He also kept the idea of having two groups — the Rounds and the Squares — that represent opposing worldviews and “two tendencies that you see in society.” The Rounds have a connection to nature and build their houses as artisans, while the Squares tend to build with concrete, iron and glass and prefer order over freedom. This manifests in the Squares’ preference for dictators and the authoritarian rule they use to control the space around them.

He added a subcategory to the Rounds: Indigenous peoples who were the first inhabitants of this world and are shown living in harmony with the land and witnessing all that unfolds. This group is the first one you encounter in the experience. Then, the Rounds appear and start building their homes out of natural elements. Next, the Squares enter the picture and start building industrial buildings. Soon after buildings go up, the interactions between the groups become harsher and violence erupts.

Police, whose main function is to suppress the Rounds, start to appear. We see people begin carrying signs and hanging flags of dictators on their homes. They spout slogans and tanks arrive. Protests erupt on both sides, trees are cut down and more beatings between the sides occur. In the chaos, both the Rounds and the Squares start setting buildings and each other ablaze. As a viewer in a headset, you spin around quickly to witness all the scenes unfolding simultaneously around you. You start embodying the chaos.

Embodying rapid city change through game mechanics

Harres wanted it to be an experience where viewers have the freedom to look around and choose which scenes to activate with their gaze. While gaze-based scene activation is a common mechanism in VR, Harres flipped it: content is swapped when you look away. It loads while you are not looking and when you look back, the animated scene has completely changed. It was a difficult game mechanic to implement but important to Harres because it captured the feeling of gentrification and rapid development that he experienced while living in Berlin.

In his words:

Berlin is undergoing a harsh process of gentrification. The city is modifying at a very fast pace. You go to a street and you don’t see a building there, and one week later you have a brand new building… [I wanted to capture] this feeling of spookiness to the viewer. There are trees and a mountain and then you look one minute later and ten buildings are there.

He went on to explain the difficulty of creating user interactions where the content was swapped once the user looked away.

This was also tricky to do because at the beginning, the content would be already activated by the borders of your field of view and then you would be missing a lot of stuff [by the time it is centered in your field of vision]… So we needed to put a collider on certain objects — that is, an invisible square on your field of view that whenever those things hit, it [should trigger an animation] in the middle of your field of view.

Genre-bending: Fiction or documentary?

When I asked Harres if he considered his work a documentary, he said it was a hybrid — both fiction and documentary.

Many of the situations depicted are based on real life situations including the burning of Galdino Jesus dos Santos, an Indigenous leader and activist who was in Brasilia for a conference and was sleeping on a bus stop bench when five white men approached him and burned him to death. (The murder charges were later dismissed.) Galdino became an icon of the indigenous resistance in Brazil.

Buildings are based on observation drawings. The city is a conglomerate of different cities referencing real world sites he had observed. He referenced buildings, streets and scenes in São Paulo, Berlin, Porto Alegre, Maquiné and some cities in Turkey. The languages spoken are a mix of actual ones. The Square language mixes Greek, Bulgarian and Swedish while the Round language mixes Igbo, Mongolian, Somali and Samoan. The dictator represented in the signs is an amalgam of dictators past and present from Indonesia, Hungary, Israel, Chile and Brazil. The themes in his work stretch far beyond his experiences in Brazil and Germany, speaking to global phenomena and political systems.

Harres collaborated with a big team. The art directors — Eizirik along with Paulo Lange — drew all the buildings on paper, and Eizirik designed the characters for all of the figures inhabiting the city. The animations were created by animators Sophia Shönborn and Samuel Patthey, the latter known for his expertise in straight ahead animation. This style starts with one pose and adds progressive poses without the smooth “in-betweening” we see in traditional animation. Without a clear guide between frames, this style allows for spontaneous, more “organic-feeling” occurrences that reinforce the documentary aspect of the content.

✍︎Straight ahead animation sample

In terms of future directions, Harres envisions distributing the project on Steam, an online game publisher, where incentives to return to the game are built in through achievements.

“It’s designed in a way that when you play it, you’re going to see around 70% of the content. Therefore if you want to see everything, you have to play again. “

Harres explains, “I saw people ending it and having this mix of amazement by seeing something that they had never seen before… but really worried about the message that was being portrayed inside. Since the first version of the script was written ten years ago, before this wave of authoritarianism started worldwide, it was already there. But back then it was much more fictional, and then it got somehow more attuned with reality. And now reality is scarily matching it.”

With its mix of references and a mesmerizing animated world, From the Main Square immerses you in a story that is both universal and specific and artfully blends fiction and documentary. It leaves you with a sense of delight at the magical world you inhabited but also with a deep sense of unease because of its resonance with the violent polarization that reflects our reality today.

For more news, discourse, and resources on immersive and emerging forms of nonfiction media, sign up for our monthly newsletter.

Immerse is an initiative of the MIT Open DocLab and Dot Connector Studio, and receives funding from Just Films | Ford Foundation, the MacArthur Foundation, and the National Endowment for the Arts. The Gotham Film & Media Institute is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing emerging nonfiction projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.