We’re pleased to reprint this piece, which asked 23 augmented and virtual reality journalists, producers, organizers, scholars, and curators for their notable trends, highlights and inspiration in 2019. It was originally compiled and published by Matt MacVey, AR/VR Journalism Lab at Newmark Graduate School of Journalism (CUNY), and Journalism 360 (ONA).
Debra Anderson
Co-Founder & Chief Strategy Officer, Datavized @debraeanderson
Apollo 11 Moon Landing (TIME)
This compelling augmented reality experience, available on iOS and Android, immerses you in the historic Apollo 11 landing by creating a simulation of the flight path that combines interactive audio narration with 3D. The detail of the re-creation and interactive, time-based elements draw you into the voyage of the landing and exploration of the moon’s landscape in an unexpected and delightful way that demonstrates how mobile AR can be used to create an extended reality.
How to watch: Landing on the Moon is available for iOS and Android devices in the TIME Immersive app.
Airfield
Airfield is an interdimensional audio editor that lets you import audio files from your device to edit, then export as Mono/Stereo or Spatialize for VR/AR. Airfield is a web tool designed to let you record and edit in the field, to use when you can’t bring bulky or expensive equipment. Airfield was built by Datavized Technologies and made possible by the Journalism 360 Grant Award from the Online News Association.
Get started here https://app.airfield.space
Gabo Arora
Founding Director, Department of Immersive Storytelling and Emerging Technologies at Johns Hopkins University @gaboarora
These Sleepless Nights
These Sleepless Nights is a “mixed reality” documentary that’s also an installation. It merges AR within a physical space and pushes it in really interesting directions. It adapts the Pulitzer Prize-winning book “evicted” by Matthew Desmond and has music by Philip Glass. This piece tells a non-fiction story spatially and shows how using your body and immersive design and can connect us more deeply to issues we otherwise may feel numb to.
“These Sleepless Nights” premiered at Venice Film Festival in the form of a physical installation.
Nonny de la Peña
Founder, Emblematic @ImmersiveJourno
To the Moon (Laurie Anderson x Hsin-Chien Huang)
One of my favorite pieces created this year was To the Moon, created by US artist Laurie Anderson and Taiwanese new media artist Hsin-Chien Huang. While I have found movement in VR not directly connected to body movement can be nauseating, the ability to increase speed and direction using outstretched arms was completely exhilarating. I also saw the first iteration of Transplant, a new work by producers Oscar Raby and Katy Morrison of the Melbourne based VRTOV studio, and was blown away.
How to watch: “To the Moon” is currently showing at The Classroom at The Public in New York from January 8–19, 2020 and at Royal Ontario Museum in Toronto January 11–25, 2020.
Loren Hammonds
Senior Programmer, Tribeca Immersive @TheCinematic
Another Dream; Common Ground; Porton Down
This year brought some truly compelling non-fiction immersive experiences to light, many using different forms of interactivity to further immerse viewers in the documentary form. Pieces such as Tamara Shogaolu’s Another Dream, Darren Emerson’s Common Ground, and Callum Cooper’s Porton Down took non-fiction storytelling to the next level with their use of room-scale environments, intuitive interactions, and gorgeous illustrations that blurred the line between fantasy and reality, all while firmly rooted in truth.
How to watch: keep an eye on the project’s webpages for upcoming viewing opportunities.
Robert Hernandez
Digital Journalism Professor, USC Annenberg School for Communication and Journalism @webjournalist
Photogrammetry + AR
The ability to create accurate 3D models based on photos — at quick and affordable rates — unlocks so many possibilities. And make those models accessible through a variety of ways means bigger reach an engagement. To tell stories through AR is a new opportunity everyone should explore.
USA Today uses AR to get you close up access to these Hollywood costumes… using frames taken from 4k video to produce these models. Homeless Realities is also powerful because it is done by students and takes advantage of Snapchat Lens Studio to distribute the AR experience.
How to watch: To access “Oscars AR: The Costumes and Creators”:
- Download the latest version of the USA TODAY app on your Android or iOSAR-capable device.
- Look for Augmented Reality in Sections.
To access “Homeless Realities” scan the codes below using the Snapchat app.
Laura Hertzfeld & Jake Sally
Hertzfeld is Director XR Partner Program, Yahoo! News @laurahertzfeld. Sally is Head of Development, RYOT
Rebuilding Paradise (Yahoo News)
Yahoo News created an immersive news experience that brings users into the lives of people living in places devastated by the 2018 wildfires. In Rebuilding Paradise, we follow Jo and Bert Clement, who lost their home in the Paradise, Calif. Camp Fire and are in the process of rebuilding. The XR narrative takes the user through a photogrammetry 3-D model of the Clements’ house, interspersed with family photos and a rich audio story about their lives before and after the fire. Yahoo News collaborated with RYOT on the production and concept of Rebuilding Paradise. Delivered in-app and coming in at <10mb file size, users embark on a full, robust experience, with the Clements’ house recreated to-scale, without having to download additional software.
How to watch: Download the Yahoo News app to launch the full experience. Augmented reality is currently available to iPhone users (iPhone 8 and later) with the latest version of iOS.
Inside the Amazon AR Experience (TIME)
This piece had the eerie timing of launching just as a wave of fires threatened to devastate the area even further this fall. Mixing drone photography and 3-D photogrammetry, the team created a mapped experience that showed the vastness of the destruction of the Amazon rainforest in a tangible way. And you can’t beat that they got Jane Goodall to narrate the project!
How to watch: Google Play App Store
Trends worth following: The next wave of motion capture technology is hitting the mainstream for consumers (and quickly!). Google releases real-time hand tracking API for Android; Apple brings full-body motion capture with ARKit 3; and Verizon Media launches the first 5G powered motion capture studio in Playa Vista.
Marcelle Hopkins
Executive Producer, The New York Times @marcellehopkins
Apollo 11 VR and AR (The New York Times)
Fifty years ago, Neil Armstrong, Buzz Aldrin and Michael Collins returned to Earth with one of the grandest travelogues ever made by humans. With their ever-present cameras, the astronauts showed us the moon landing through their eyes. The New York Times produced an augmented reality story and a virtual reality experience that transport you to the surface of the moon where a virtual gallery of photographs reveals a new perspective on the iconic images.
How to watch: View an enhanced version of the article in augmented reality by using the NYTimes app on newer iPhones and iPads.
Watch a video of the Times event commemorating the 50th anniversary of Apollo 11 here: https://youtu.be/qDrtWeFRBt8
Steve Johnson
Founder SeeBoundless @steveboundless
The 2050 Project (Quartz)
In this project, which we produced with Quartz and the Emerging Technologies Lab at UNC-Chapel Hill, we had to figure out how to photograph, scan, stitch and build AR models from just an iPhone and a consumer drone.
In this project, which we produced with Quartz and the Emerging Technologies Lab at UNC-Chapel Hill, we had to figure out how to photograph, scan, stitch and build AR models from just an iPhone and a consumer drone.
The goal was to make a product using tools available to newsrooms to help push not only Quartz but the industry forward into adopting their best story ideas into AR experiences.
Apple’s ARKit 2 allows us a new way to showcase AR objects. We worked with the phenomenal development team at Quartz to build our models into their existing app and website (using iOS 12 you will be able to view the models in browser).
How to watch: View “THE 2050 PROJECT” on Quartz’s website and IOS app
Roger Kenny
Dow Jones Innovation Tech Lead @RogerWallstreet
Greenland Melting (Frontline/Nova)
I’m excited to see more spatial data viz experimentation.If I had to pick one piece I loved it’s the Greenland Melting piece by Emblematic for Frontline and Nova. It shows the power to immersive media to create empathy with the abstract.
How to watch: Download for Oculus or Vive from the Steam store Greenland Melting.
Henry Keyser
Senior XR Project Manager, Yahoo News @hwkeyser
Apollo 11 Moon Landing (TIME)
The UX of TIME’s Apollo 11 AR interactive merged form and function to encourage interaction and exploration. Their on-screen UI elements that mimic iconic 1960s flight instruments provided context and control without taking away from the holistic design. Plus, the risky choice to position “The Eagle” off-screen in the first moments encourages first-time AR users to go beyond their scanned surface and take full advantage of world space.
How to watch: Landing on the Moon is available for iOS and Android devices in the TIME Immersive app.
Ben Kreimer
Journalism technologist
After the Fallout
After the Fallout, a 360° video documentary by Sam Wolson and Dominic Nahr, is a remarkable example of understated 360° storytelling mastery. The early cut of the film I saw had no narration, no on-screen text, no graphics, nothing to distract you from being absorbed into the vivid and at times disorienting and surreal realities of life in the aftermath of the Fukushima nuclear disaster. An immersive journalism trend in 2019 was to shift away from 360° video to augmented reality storytelling, but After the Fallout is an important reminder that it takes years of persistent exploration to develop a new storytelling medium’s native language, and to have the technological novelty element fade away so the focus is only on the story.
Matthew Niederhauser
Co-founder of Sensorium: sensorium.works
Common Ground
I thought Common Ground was quite good at Tribeca, especially as an exploration of how interactivity can be incorporated within an immersive documentary narrative.
Rebecca Poulson
Senior Developer, Emerging Platforms Lead, Northwestern University Knight Lab @RebeccaPoulson
National Geographic Explore VR
I’m excited to see journalists explore non-documentary immersive techniques. The first episode of National Geographic’s Explore VR helps viewers understand factual information by placing them in an environment that’s entirely computer-generated. Both the initial Antarctica episode and the recently released Machu Picchu experience also have unique techniques for encouraging the viewer to explore the space in a structured way, allowing the creator to have more input into the story the viewer experiences, without detracting from the feeling of autonomy and presence that’s unique to VR.
How to watch: The National Geographic Explore VR app is available for Oculus Quest.
The Immersive Web!
2019 was a huge year for building the foundations of an open, decentralized, connected, spatial ecosystem. The WebXR Device API now ships by default in Chrome and Oculus browsers. A-Frame had its 1.0 release and Torch AR gave creators the ability to publish interactive AR experiences to the web directly from their iPhone. WebAssembly Core 1.0 is an official W3C spec. These tools make it possible for a much broader group of people than ever before to create and consume VR/AR experiences, regardless of their hardware.
How to watch: take a look at VR in the web browser made with WebVR and A-Frame.
Theresa Poulson
Product Manager for News, McClatchy @poulsont
Lowering Barriers to Entry
This year, we saw AR/VR become a little more accessible — for creators and audiences. New tools are lowering the barriers to entry for producers, and consumers can see more nonfiction projects via WebXR and embedded 3D and AR. At McClatchy, we trained journalists in Raleigh, Kansas City, Fort Worth and Sacramento to produce photogrammetry, publishing several projects that include Sketchfab embeds. There’s still a long way to go, but you can see the beginning of truly immersive experiences becoming more readily available for the communities we serve.
How to watch: Actual Reality app on Google Play or the McClatchyNVL SketchFab library.
Clàudia Prat
Immersive Producer, TIME @webDOCC
Paradise VR (YLE)
Paradise VR, by the Finnish Broadcasting Corporation, brings viewers to the Marshall Islands to understand the consequences of nuclear bomb testing there. The piece has interesting graphics and interactivity, which allows the user’s curiosity to guide the story.
I would also highlight the 20-min piece Traveling While Black, for its great fusion of archival footage and cinematic storytelling. The film has one of the most moving interviews filmed to date in VR. It’s a must-see.
Last, I would point out that 2019 is the year of having AR available in Google search, a straight-ahead step to put and share 3D volumes into our lives.
How to watch: Paradise VR is available in the Steam store or watch a 360° version on YouTube.
Traveling While Black is available for Oculus Go, Rift and Quest and on YouTube. Search for an octopus or emperor penguin (or any of the animals on this list) in Google on a mobile phone and tap “View in 3D” to see a 3D animation of the animal.
Linda Rath
CEO & Founder Vragments @Lynda420
The Bright Side of Immersive
VR is going through a rough patch right now. With the BBC closing its VR Hub and Google stopping sales of the Daydream View headset, you would think that everyone’s turning their backs on Virtual Reality. Sure, putting on VR headsets can cause friction and the demand for some headsets is fading but I do believe that we have to look at the bigger picture: immersive storytelling is here to stay!
Immersive stories allow viewers to explore the environment, grant access to unknown surroundings and provide viewers with a different perspective. The market penetration of VR headsets might not have reached its peak yet, but it’s important to continue to experiment on the content side so that journalists can learn what works and what doesn’t. Only by observing users and seeing how they react to immersive stories can they become better at telling these stories.
While the future of the VR hardware market is still unknown, I am certain that spatial storytelling will remain relevant.
Frank Rose
Senior Fellow, Columbia University School of the Arts @frankrose
The Privacy Project (New York Times)
An extremely newsworthy series of essays, some of them involving original research, exploring and explaining a topic that is very poorly understood. By linking them together as a series, the Times has created an important online resource — something digital journalism is uniquely qualified to do.
How to watch: Take a look at some of the interactive sections of the Privacy Project series.
Mandy Rose
Co-Convenor i-Docs Symposium Professor — Documentary & Digital Cultures, University of the West of England @collabdocs
Stealing Ur Feelings (Noah Levenson)
Porton Down(Callum Cooper)
In Event of Moon Disaster (Francesca Panetta, Halsey Burgund)
Some of the most memorable interactive and immersive documentary works I saw this year tackle the ethics of emerging media technologies and platforms. Stealing Ur Feelings by Noah Levenson / Mozilla explored how Facebook, Snapchat, and Instagram use facial emotion recognition technology to make decisions about users. Porton Down, by Callum Cooper and Don Webb, explored the potential of Virtual Reality for psychological manipulation and data extraction. Perhaps the standout was Halsey Burgen and Francesca Panetta’s In Event of Moon Disaster, which I saw in competition at IDFA’s Doc Lab in November. On a TV screen in a 1960s living room, footage of the 1969 moonshot unfolds, then abruptly cuts to black. Next Richard Nixon appears, with a sombre announcement of the demise of the mission. Using easily available deep fake video technology to rewrite a central event of 20th Century history; In Event of Moon Disaster, which won IDFA’s digital storytelling award, asks us to consider what becomes of trust in our post-truth era and what kind of media literacies we need now.
In Event of Moon Disaster” premiered at IDFA DocLab on November 22, 2019, in the form of a physical installation designed to re-create a 1960s era American living room.
Lakshmi Sarah
Creative Producer, Tiny World Productions @lakitalki
Beyond Screens
Going beyond the screen and integrating AR with different elements is a trend I have been intrigued by. For example, the USA Today podcast The City coupled with an AR experience. But also combining physical locations with elements of audio and visual. Using brick and mortar locations with experiences like Meow Wolf, Area 15 in Vegas, and Gray Area in San Francisco create curiosity and add a lived experience advancing the singular VR experience to a more collective one.
Thomas Seymat
Euronews, RJI fellow 2018–19 @tseymat
News Orgs tell the Apollo 11 story in AR
- In a year during which it felt like lots of publishers were pulling back from the immersive space, Apollo 11th’s 50th anniversary in July 2019 showed how media can use Augmented Reality for editorial projects around events that resonate with the audience. By my count, at least four newsrooms (the New York Times, TIME, the Washington Post and USA Today via its 321 Launch app) covered the anniversary using AR and/or photogrammetry. I have found other, more recent examples (like Yahoo’s Camp Fire AR story, the NYT’s AR air pollution experience), to be quite engaging as well.
- As they announced their shutting down, the BBC VR Hub released a couple of very informative blogposts and reports, including some based on the feedback they gathered from audience members through pop-up VR installations in 175 local British libraries where visitors could try BBC VR experiences. Through surveys and interviews, this research project — conducted on a scale unheard of for non-fiction immersive content — provides valuable insights from the people we work for at the end of the day: our audience. Their work also echoes some of the ideas findings coming from my own research I conducted this year as a RJI fellow on why listening to the audience is key to improving immersive 360/VR experiences.
Kamal Sinclair
Executive Director, Guild of Future Architects @KamalSinclair
The Dial
This was an outstanding example of how AR can achieve beauty, story, and wonder.
Traveling While Black
In terms of emotional resonance, this one tops my list for 2019.
How to watch: Traveling While Black is available for Oculus Go, Rift and Questand on YouTube.
Ray Soto
Director of Emerging Tech, USA TODAY NETWORK @raysototech
The industry is finally starting to get it!
I’m encouraged by what I’ve seen throughout the journalism industry in 2019. We’ve experienced diverse stories built by diverse teams, technology and narrative experimentation, and honest transparency as a means to elevate immersive storytelling beyond a gimmick. 2019 was the year that everyone (including audiences) finally understands the value and power of immersive technology.
Immerse is an initiative of the MIT Open DocLab and The Fledgling Fund, and it receives funding from Just Films | Ford Foundation and the MacArthur Foundation. IFP is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing media projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.