Is There a Place in Human Consciousness Where Surveillance Cannot Go?

Ellen Pearlman on 2018-12-17

Noor: a Brain Opera

Noor: A Brain Opera is an opera where a performer wearing a wireless EEG headset triggers videos, a sonic environment, and a libretto with her brainwaves. The results are displayed in live-time as she interacts with audience members within the confines of a 360-degree immersive theatre. Premiering in 2016, it was the first performance of its kind.

I conceived and directed the work, and framed its theme around the rhetorical question: “Is there a place in human consciousness where surveillance cannot go?” Through sound, images, and spoken narrative, the opera presented the true story of the life of Noor Inayat Khan, a Sufi Muslim Princess and covert British operative inside Nazi-occupied France who was murdered at Dachau.

It touched upon issues of memory, faith, and the locus of self in light of increasingly invasive and sophisticated surveillance technologies. The performance also worked with the bodies of both the performer and participants by creating a responsive feedback loop in which the performer’s interactions with the audience — through movement, gaze, touch, and speech — visibly changed the performer’s brainwaves. During the event, these were displayed in live time as part of a five-screen immersive theatrical experience

Putting the Emotiv headset on the performer Saba Arat at the beginning of the performance, in front of the audience

I developed the opera over the course of two years. Once I created a working proof-of-concept that brainwaves could indeed trigger visuals, sound, and speech, I collaborated with a team of seven individuals in New York, Hong Kong, and Russia using a combination of Skype and Team Viewer. This allowed different programmers to take control of the computer screen and debug recalcitrant software code. The score, the pre-recorded libretto, and the videos were contained inside four electronic databanks that corresponded to the emotions of excitement, interest, meditation, and frustration. These databanks would be triggered according to pre-set thresholds that were calibrated during rehearsals with the performer’s brainwaves.

As an interactive brain opera, Noor took advantage of the recent emergence of consumer-based EEG brainwave headsets and various other devices capable of interacting with and augmenting new technologies. It was developed using an Emotiv headset, a model I selected because it allows access to more mental states and has greater stability than other similar consumer headset models. Different graphical and colored representations of the performer’s states would arise and fade out as per the levels of emotional intensity she was experiencing.

Images are trigged by the performer’s brainwaves as she gazes into the eyes of an audience member

The story and interaction took place over approximately a twenty-minute period. Wearing a brainwave headset for longer than this was not feasible for a performer, due to both the pressure from the headset on her head and the closeness of its Bluetooth wireless signals to her brain. The experience began as soon as the audience walked into the darkened theatre. The spectators were given a brief introduction as well as instructions about what the five screens and sounds represented. I explained that they would see colored bubbles on one screen and that the colors reflected different emotional states, which would be activated at various times throughout the performance.

The colors and their corresponding emotions were as follows:

The videos, sounds and libretto corresponding to those emotions would, at various times, be launched on one of the screens, with different images fading in and out, depending on the intensity of the performer’s emotional state. The performer was free to respond through her words and actions. The audience’s behavior affected the performer’s brainwaves as well, generating a continual feedback loop: story-performer-audience-story-performer-audience.

Performer touches audience member as her brainwaves trigger images, sound and a libretto in real time.

The question of how to represent the interior workings of someone’s brain guided my work. Humans are hard-wired to understand stories, and I wanted drama and narration to prevail in such a highly technical environment. I learned through my work that audiences were drawn in by unexpected twists in conflict and when the boundaries between the real and the unreal unraveled — when outer perceptions and inner realities became temporarily suspended.

The premiere of Noor: A Brain Opera proposed a new, content-rich environment for creative practitioners to explore. Although the first use of EEG sensors and the brain by artists took place in 1965, with a concert by Alvin Lucier, Music for Solo Performer, and has been used by various artists since then, the availability of consumer-grade EEG sensors has made the brain a new instrument for different types of performative interactions.

Is there a place in human consciousness where surveillance cannot go? These new forms are answers still await discovery.

Ellen Pearlman is a new media artist, critic and curator and Fulbright World Learning Specialist in New Media Art and Technology. Ellen is also Director of the ThoughtWorks Arts Residency, President of Art-A-Hack™, and Director of the Volumetric Society of New York, on faculty at Parsons/New School University and an Assistant Professor, Senior Researcher at RISEBA University in Latvia.

Immerse is an initiative of the MIT Open DocLab and The Fledgling Fund, and it receives funding from Just Films | Ford Foundation and the MacArthur Foundation. IFP is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing media projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.