Field Notes: New Frontiers at Sundance Film Festival

New Frontiers at Sundance Film Festival

Human touch, social space, archiving for the future, and more

This year’s New Frontier exhibitions at Sundance showcased the explosive creativity that storytellers are tapping into as the technologies for immersive expression are maturing.

Several works invited participants to explore dissociative states of consciousness by integrating both narrative and embodied methods in VR, even introducing waterproof headsets, snorkels, and swimming pools to the mix. Others investigated issues of climate change through various genres, including playful approaches in animation, or a combination of live music, shadow puppetry, and dance to offer emotionally complex, layered performance. VR also proved to be a powerful medium to explore the spatial aspects of social justice, with projects that raised issues about incarceration and data privacy, or artificial intelligence and the nature of truth.

Divided into four venues based on themes and methods, New Frontiers offered festival-goers an array of immersive technologies to experience, ranging from 360 videos to social VR performances to AR works overlaid onto the festival campus. Innovation in storytelling, however, certainly transcended these exhibition halls, and marked a spirit for creativity in thinking about the future of media. Here are some key notes:

Bringing a human touch to VR

When makers integrate human touch into a VR experience, an unexpected tension surfaces at the boundary between technology and the self. The South Korean team from K’ARTS beautifully guides users along this line in Scarecrow, a one-on-one VR immersive theatre piece that brings a live actor into the virtual experience. Through various techniques such as eye-tracking, motion capture, and thermal haptics, users build an emotional connection to the character, the act of touch evoking the question, “Is this allowed in VR?”

Raising a similar question, social VR piece Metamorphic brings in two users to interact with the environment, then gradually with each other. The interplay between the users is drawn out almost like a dance, asking their permission to go beyond the virtual. The AR work Breathe, on the other hand, enacts this interplay through the Magic Leap and a belt that senses the user’s breathing, making the body the centerpiece of the story.

360 Video as Both Familiar and New

Several 360 videos were showcased in the VR Cinema sections, in which around thirty viewers experienced a series of four VR works in an hour-long block in a room together, followed by a Q&A with the artists.

VR Cinema 1 focused on pieces that offered a sense of presence in another locale, be it in the heart of a Cuban ritual as in Bembé or at an abandoned mansion of South Africa in Azibuye — the Occupation. These works were familiar in form but raised important issues about political strife, incarceration, and nuclear disaster in some of the best VR journalism.

VR Cinema 2, the second block, offered a more experimental selection that moved beyond straightforward 360 videos. For instance, Hominade constructed a surreal environment through x-ray imagery and anatomical reconstructions to tell the story of an invented species, while the conversation in Flowers & a Switchblade was represented through a video collage method that paralleled its theme about the fragmentation of modern life.

Thinking critically about space

Other projects complicated the relationship between the user’s body and social space it exists in.

Persuasion Machines takes users into a smart living room, where devices surveil and track personal data, opening into portals of big data and how it affects daily lives.

Still Here is a VR and AR piece accompanied by a photo series of six women and their perspectives on the criminal justice system in New York. It follows the character Jasmine as she returns to Harlem after being incarcerated for 15 years, only to find it newly gentrified, erased of all that once made it familiar. In this AR piece, users walked around the New Frontiers exhibition space with a tablet in hand while hearing Jasmine’s story as she tries to find employment. Given that New Frontiers is a space marked by consumerist technologies and an industry-specific audience, the piece adds another layer of critique to the festival space itself. How is the concept of the “new” laden with erasures of the past?

Artificial Intelligence and archiving for the future

Both the allure of AI and the biases baked into it were on display at Sundance this year.

In Chomsky vs. Chomsky, users are able to have a conversation with a simulation of Noam Chomsky, powered by extensive digital archives of his writings and Natural Language Processing. This piece challenges the expectations participants enter with in speaking to an AI-powered system. In the panel “AI, the Archive, and Creating Human Context on the New Frontier,” lead artist Sandra Rodriguez remarked, “There’s an irony in that you want to trust an AI system or want it to be free and open, but you don’t see where the controls are. The back-end really uses data that is highly biased.”

This conversation was echoed by other MIT researchers featured in Coded Bias, a documentary film about how facial-recognition technology continues a lineage of algorithmic bias that perpetuates the oppression of marginalized communities. In creating AI-generated artworks, then, key questions about the archive, the sample data from which the future is based, become political ones.

“The obsession for what is archivable or what is relevant as an archive or what isn’t, really suddenly becomes linked to phenomenons that are social or culturally driven,” said Sandra Rodriguez, “I want the machine to instead show us where the biases are, then at least we can start a conversation.”

Innovation in storytelling beyond headsets and wearables

From the World Documentary Film section, Saudi Runaway captured audiences with its striking and unconventional use of mobile technology to offer a window into the pressures of living in a Muslim conservative society as a woman. Shot entirely on an iPhone, Saudi Runaway offers an intimate glimpse into the life of Muna, a soon-to-be bride planning to escape from her marriage and society. She secretly films by holding her camera phone from beneath her abaya and uploading all her footage to the cloud before her father searches her phone.

The aesthetic produced by filming from a mobile phone represents both Muna’s courage in seeking a new life and creativity in producing her own narrative. Saudi Runaway brings to light the societal restrictions that Saudia Arabian women face, while also reminding us that innovation comes in forms beyond the newest consumer headsets and through the creative will to tell a story.

These field notes encompass only a selection of the expansive VR/AR/XR forms exhibited this year at New Frontiers. The rich intellectual and artistic worlds within each of these projects makes New Frontier a breeding ground for ideas that have only yet begun to be realized.

Immerse is an initiative of the MIT Open DocLab and The Fledgling Fund, and it receives funding from Just Films | Ford Foundation and the MacArthur Foundation. IFP is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing media projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.