Part 1: Exploring AI at IDFA’s DocLab
Does anyone really know what they mean when they refer to “AI”? My colleagues at MIT certainly think they do, although they are inclined to use more specific terms such as “machine learning and generative adversarial networks” (GAN). The marketers of smart-phones and today’s feature-rich appliances seem to know as well, referring to a vague composite of computational developments that makes devices easier and more intuitive. And judging from the many media representations that use AI as a provocation regarding what it means to be human as we model machines in our own likeness, the media world also thinks that it knows what AI means.
A cluster of competing technologies, definitions, and associations, AI is a buzzword. It is also on the rise as a technological panacea and investment opportunity. Those of a certain age might say that we’ve been here before, for AI has experienced several boom-and-bust cycles of investment, research, and collapse leading to “AI-winters.” Right now, we’re once again in the hype stage where all things are possible and research and investment are going strong. But winter may be around the corner, giving AI developments a bit of urgency.
A Dinner
With Frankenstein AI (Lance Weiler, Nick Fortugno, Rachel Ginsberg)True, we are working with significantly improved computation systems, both in the sense of processing power and pervasiveness in just about every aspect of our lives. This could prove to be a real game-changer. But we also have evidence that AI’s capacities and the uses made of them are outpacing society’s ability to cope. Consider the carefully curated media worlds that most of us inhabit and that seem to yield a social condition described by Sherry Turkle as “alone together.” Or consider the growing threat to our notion of reality posed by AI-generated deepfakes and algorithmically-targeted “fake-news.” Or consider the changes to many of our production pipelines, as decision-making shifts from man to machine.
AI is in a dynamic state, imminent with promise, plagued by uncertainty, and ripe for exploitation. Enter the arts, and particularly the documentary arts with their contractual relationship to truth, however defined. The arts interrogate. They confront us in ways that cut through the silos we inhabit. (and in AI’s case, that’s welcome work!) And they help us to see implications with fresh eyes. The arts, and in this case the documentary arts, help us to engage and imagine.
Taking a closer look
In this initial research phase, we had the opportunity to explore a number of AI-based systems that enabled, infused, or were reflected upon by projects at the 2018 IDFA Doclab, including:
- A Dinner With Frankenstein AI (Lance Weiler, Nick Fortugno, Rachel Ginsberg)
- Eat | Tech | Kitchen (Klasien van de Zandschulp and Emilie Baltz)
- The Social Sorting Experiment (Steye Hallema and The Smartphone Orchestra)
- I’ve Always Been Jealous of Other People’s Families (Shirin Anlen)
- Algorithmic Perfumery (Frederik Duerinck)
We interviewed the projects’ makers, learned about their visions of and relationships to AI-systems, and investigated various ways of understanding the user experience. The goal was to jump in and explore how best to frame questions and research methods, since no obvious precedents either in UX/UI testing protocols or audience research strategies offered a scaffold on which we could build. Our main questions had to do with understanding:
- How the artist defined AI — both in a technical sense (GANs, etc.) but also in terms of the creative process: Is it a barrier, an incentive, a site of inspiration or interrogation, etc.?
- What the role of the AI system was — a mere tool like any other? A distorted mirror, reflecting our values and operations? An agent in its own right? An opportunity for pedagogical reflection? Some other configuration?
- How the relationship between makers and AI was imagined — as tool? Partner? Provocation? Performer? Adversary?
- How agency was envisioned — as fully in the hands of the artist? Or shared or even deferred to the emergent AI system? Or shared and even deferred to users? Or some combination thereof?
- How the user’s relationship to the AI system was envisioned and experienced (as equitable? extractive? enhancing?)
- AI’s potential to stimulate a larger co-creative process between the designer, the system, and the user.
Eat |
Tech | Kitchen (Klasien van de Zandschulp and Emilie Baltz)Artificial Intelligence systems pose significant challenges to the business-as-usual of representation. They have entered just about every part of the film, video, audio, and print production pipeline, including the curation of the audience. We know this, of course, and get used to it (I’m grateful for spell-check, automatic formatting, and phrasing suggestions as I write this). But this increasing presence, and our increasing dependencies, merit serious consideration. That was the driver for these research questions.
We tracked maker and user expectations and perceptions by using scripted interviews. We were particularly interested in maker perceptions of the user experience, and user perceptions of the making process, correlating these against actual responses. Our analysis of the results is necessarily preliminary, but offers both surface-level data regarding attitudes and knowledge, as well as more fine-grained analysis regarding agency and control. If AI-based experiences represent a major shift from the more familiar subject-object dyad of documentary storytelling, our success in making a case for their aesthetic and documentary potentials will turn on careful analysis and understanding.
Modes of Engagement
Dziga Vertov’s Man with a Movie Camera (1929) turned the camera on itself to explore the filmmaking process and its embeddedness in (and aspirations for) the social world. It marked the emergence of a critical and self-conscious interrogation of the film medium by means of the film medium. And it did so eloquently and provocatively, answering Andre Bazin’s question ‘what is cinema’ decades before it was even asked.
We inhabit a similar moment with AI. “Artificial Intelligence” as a term has been around since 1955, but just as these two words mask multiple and quite different approaches to machine intelligence, they occlude centuries of aspiration and endeavor. Like film at the end of the 1920s, AI in its current configuration is still the object of hype, shrouded in mystery, and dazzling the public with its promise. And like film in the late 1920s, it is also sufficiently pervasive and well-enough understood that it has become the subject of interrogation by artists, who like Vertov, turn the system on itself in order to reveal and critically assess its potentials.
The IDFA projects we researched all shared a critical interrogation of AI in its various guises. But, of course, they did much more, which I address here in Part II!
In 2018, the International Documentary Festival Amsterdam’s Doclab initiated its Immersive Research and Development Program in partnership with, among other organizations, MIT’s Open Documentary Lab (ODL). For the kickoff period, ODL’s research focused on AI-based documentary experiences, investigating how makers and their audiences conceptualize Artificial Intelligence (AI); how they cope with the fast-changing state of AI; and how they understand their creative practice and agency together with these responsive systems. It sought to gain an insight into how users imagine AI’s role in the creative process, as well as how they interact with AI systems.
This on-going work builds in part on a field study that MIT’s Co-Creation Studio has undertaken on co-creation with, among other things, algorithms. The full report may be found here: https://wip.mitpress.mit.edu/collectivewisdom As research results accrue over the next few years, we hope that the project will yield AI-specific criteria for curators, funders, and ultimately the public, as they seek to assess proposals and documentaries.
Immerse is an initiative of the MIT Open DocLab and The Fledgling Fund, and it receives funding from Just Films | Ford Foundation and the MacArthur Foundation. IFP is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing media projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.