Guidelines for ethical virtual resurrection of the “working dead”
Illustrations of figures in
various states of digital resurrection (courtesy of Whatever Inc.)For the first time in human history, the population of the “working dead” is rising. Recent advancements in technology have made it possible to combine personal data, AI, motion capture, and high resolution CGI to virtually resurrect the deceased. For example, a program can study what those who have passed said and generate an AI character, or even recreate their semblance in real time using methods such as the notorious “deepfake.” These methods can easily infringe portrait rights of the deceased and have triggered larger ethical and moral questions.
Our company Whatever Inc produced such a project, introducing us to this phenomenon and the deeper issues surrounding it. We created the acronym “D.E.A.D.” (Digital Employment After Death) and embarked on an extensive research project on the spaces where we should — or should not — integrate this phenomenon in our lives.
History of the D.E.A.D.
From left
to right: images of Virtual Tupac (courtesy of Digital Domain), Frankenstein, Roy Orbison &
Buddy Holly Rock ’N’ Roll Dream Tour (courtesy of Base Hologram), and stills from “Be Right Back”
(courtesy of Netflix 2010s anthology series Black Mirror).People have been fascinated by the idea of bringing people back from the dead for centuries. From ancient Greek myth about Orpheus trying to recover his lost wife Eurydice, to the science fiction classic Frankenstein to films in 21st century, there are many stories told around the same theme.
Take a 2013 episode from the TV show Black Mirror called “Be Right Back.” It features a woman who signs up for a service that allows you to “bring back” her lost loved one using their personal online data and a physical avatar. While this service is far from a reality, we’ve started to see many D.E.A.D. examples in the real world, especially within the realm of entertainment.
One breakthrough example was “Dr. Dre & Snoop Dogg feat. 2PAC at Coachella” in 2012, where rapper Tupac was recreated in CGI and sang on stage at Coachella. (Later on, he also starred in a Supreme ad). This project opened up the door for other musicians such as Whitney Houston, Buddy Holly, Dio, Frank Zappa to get back on stage. Not only musicians, but artists such Andy Warhol and Salvador Dali have been revived and talking about their own life and work.
From these examples, you might think the D.E.A.D. is only for celebrities. But we’re starting to see them among more ordinary people as well:
- A mother ‘reunited’ with her dead daughter in VR
- A victim of gun violence was brought back to life to send a message against guns
- A murdered football prodigy was brought back to life as a playable character on EA Sports’ FIFA 21
- And now a VR company in Czech is teasing the opportunity to stay alive forever in the metaverse
To add one more example to many, our team has also attempted to resurrect the deceased for a 2020 TV show we created called “Day of Reunion.”
Designing the D.E.A.D.
Our show helped Tetsurō Degawa, a famous comedian in Japan, reunite one last time with his deceased mother, Yasuko. As we started to design the this exchange with care and intention, we developed a set of five guidelines we should follow in our bringing back the dead:
- Make sure the client (Tetsurō Degawa & his family) was satisfied with the visuals and all context.
- Always try to root everything based on the facts we were able to collect. (Don’t manipulate or create new perspectives.)
- Never have Yasuko talk or describe anything about the “afterlife.”
- Do not speak to the audience, and keep the conversation between the two main characters.
- Try to make the technology as invisible as possible.
Principle number one is fairly straightforward, but we wanted to make sure that the surviving family members were happy about everything they saw and heard. Success for us was defined if Tetsurō felt he had fulfilled his wishes, even if the external viewers didn’t like the program. To match his Tetsurō’s vision, we experimented with many versions of Yasuko’s look, her facial expressions, her wardrobe, her habits, which age should she be represented as, etc., until we got to a place appropriate and satisfactory to Tetsurō and his family members.
Images of Tetsurō Degawa,
courtesy of NHK’s “Day of Reunion” (2020)For guideline number two, we made sure to stay true to Yasuko’s actual persona as much as we could, without adding anything new or manipulating non-existing narratives. We asked her family to provide us with any materials we could use to try and “reconstruct” her. We were able to reference numerous family photos and videos to use to replicate her looks.
Her actions and movement was also an important factor in making her look and feel like herself, which we carefully observed the videos to imitate. We learned about a few of her habits from the family, such as crossing her arms and putting one hand at her chin — a small but important action we added. From there, we had several hours of interviews with the family members using the photos as a starting point to talk about the memorable moments they had together, and any other facts they could remember about her. We used these learnings to create a tree diagram of possible dialogue and scripts based on real facts, to use as a basis of the improvised talk session.
Images of virtual production
process, courtesy of NHK’s “Day of Reunion” (2020)Additionally, we were very careful not to have Yasuko say things like: “I was watching you from heaven,” or “I came back to life,” to avoid distracting the audience with questions of the existence of an afterlife and any religious associations. We felt this distracted from the focus of the show, which was about reuniting with someone in a way you didn’t think was possible.
For guideline number four, we envisioned this to be a conversation only between the main characters, thinking that breaking the fourth wall with audience communication would make the experience feel more fake and staged. We envisioned Yasuko not as an “actress” on stage, but more so “herself” in a realistic context. Designing the experience to create a more intimate dialogue between the living and the deceased was what made everything believable.
Finally, we wanted the focus to be about the conversation, not about the technology behind it. Our main art direction was to use techniques borrowed from “Bunraku,” a traditional Japanese theater style in which men dressed in all black called “Kuroko” move the puppets to enact a story, but they are considered invisible and part of the “background” tech behind the scenes. We wanted our D.E.A.D. technologies to be akin to Kurokos.
A bunraku performance — two
kuroko hold a puppet to enact a traditional story (Image courtesy of
© coward_lion/iStock.com)So, we designed a minimalistic black stage, where the two conversationalists were the only visible elements on screen. A black-colored, high-resolution screen blended with the background and became essentially invisible. The spotlight in the real world seamlessly continued inside the screen to connect the real and virtual. We shifted the perspective of the CGI in real time based on the camera position, so it looked natural to Tetsurō and the viewers at the same time. This allowed Tetsurō to really immerse himself into the conversations with his mother without unnecessary distractions.
Image of Testuro’s conversation
with his mother, courtesy of NHK’s “Day of Reunion” (2020)As a result, we were able to create an experience so moving that the lead actor broke down into tears towards the end of the conversation. Also, there were many positive comments on social media, and surprisingly for a project like this, not much criticism. This may have been a result of the many considerations we took into account as we designed the show.
Taking the Pulse of the Public
But not all D.E.A.D. projects end with a happy note like ours. There are many projects that are getting criticized and burned. For example, news that James Dean will be brought back to life to star in “Finding Jack” sparked debate and led to an uproar of disapproval on social media. Another example was with Hibari Misora, a very famous Japanese diva, who was brought back to life and “created” a new song. This received a tremendous amount of backlash, with many questioning the authenticity of creating a new work.
This mixed criticism made us want to know what everyone really thinks about the D.E.A.D. So to begin with, we set out to conduct a research study surveying 1000 in the US and Japan. At a high level, we found:
- Respondents have strong opposition to being resurrected and an even stronger opposition to resurrecting others. ⅔ of respondents would not want to be digitally resurrected using their personal data and AI technology and ¾ would not want to resurrect others who had passed away. Not having consent from the dead and other ethical qualms are the main objections to D.E.A.D in both countries.
- Authenticity is important, but controversial. People showed more resistance to resurrections where the visual/audio resemblance became closer to reality, preferring text to audio and audio to video. They also resisted the idea of being forced to do something they haven’t done or said while they were alive.
For more information on this research study and other projects, including a template for a D.E.A.D. Declaration Form (similar to an organ-donor opt-in process, but for digital resurrection) please visit our D.E.A.D. website.
For more news, discourse, and resources on immersive and emerging forms of nonfiction media, sign up for our monthly newsletter.
Immerse is an initiative of the MIT Open DocLab and Dot Connector Studio, and receives funding from Just Films | Ford Foundation, the MacArthur Foundation, and the National Endowment for the Arts. The Gotham Film & Media Institute is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing emerging nonfiction projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.