Field Notes: XR Journalism at SXSW

Marcus Bösch on 2018-04-13

XR Journalism at SXSW

Instagram celebrity lilmiquela is digitally created. To a lot of people, including nearly one million followers, that does not matter at all. How does the blurring line between the real and the virtual affect journalism?

Let’s coin a new phrase — XR journalism — because it is needed and might prove useful in the decade to come. It could and should have been discussed in Austin, Texas this year. Because if you want to get pumped up with the latest buzzwords, where do you go? There are gazillions of panels at SXSW with all kinds of ideas about disruption, future talk, etc. Below, I compile some news snippets from this year’s SXSW that led me to believe that we should start talking more about XR journalism in 2018.

And, of course, journalists interested in shaping the near-future landscape in meaningful ways should focus not just on the talking but on the doing, inventing, evaluating, and re-doing.

Impress your colleagues by zooming into a stuffed folder with your bare hands

The expression “XR journalism” cannot be found on the SXSW website. It is not even used much on Twitter yet, except by my colleague Stephan, who reserved the twitter handle while we were talking lately. So, what does the “X” in “XR journalism” stand for, and why do we need to talk about it?

It’s because there’s yet another technological shift in the making. Amy Webb of the Future Today Institute’s presented her 235 Emerging Tech Trends For 2018 at SXSW. The very first key finding goes like this:

2018 marks the beginning of the end of traditional smartphones. During the next decade, we will start to transition to the next era of computing and connected devices, which we will wear and will command using our voices, gesture and touch. The transition from smartphones to smart wearables and invisible interfaces — earbuds that have biometric sensors and speakers; rings and bracelets that sense motion; smart glasses that record and display information — will forever change how we experience the physical world. This doesn’t necessarily signal a post-screen existence. We anticipate foldable and scrollable screens for portable, longer-form reading and writing.

If you start digging deeper into the world of existing AR glasses or prototypes you get the idea. They look “normal” like the Vaunt, they make use of voice and sound like Bose AR and they will make actual AR apps limited to the screen size of a smartphone way more useful.

Okay, now let us talk about XR

If you read Immerse, you’ve heard about VR (virtual reality), AR (augmented reality), and maybe even MR (mixed reality). In 1994 Paul Milgram and Fumio Kishino defined mixed reality as “anywhere between the extrema of the virtuality continuum,” which extends from the completely real through to the completely virtual environment, with augmented reality and augmented virtuality ranging between.

So why do we need another term?

As Sivaramakirshnan Somasegar and Linda Lian write in TechCrunch, “To bet that one specific point on the continuum will ‘win’ is a rather narrow vision of what the future may look like. To believe in XR — that a fundamental shift is occurring in the way people view, share, create and use media data — is far less risky.”

So:

I especially like the cross-reference to James Bridle’s concept of a New Aesthetic and to Gilles Deleuze’s concept of virtuality in the XR Wikipedia article:

XR can be viewed as a manifestation of “The New Aesthetic,” the term coined by James Bridle that references “the increasing appearance of the visual language of digital technology and the Internet in the physical world, and the blending of virtual and physical.” Closely related to Gilles Deleuze’s concept of virtuality, XR positions “virtual” as not opposed to “real.” Within the framework of XR, virtuality is not the opposite of reality and digital is not the opposite of biological. Rather, XR envisions a more complex relationship in which virtuality actualizes real effects to the extent that our perception of virtual objects becomes fully real.

XR technologies have applications in almost every industry, including: architecture, automotive industry, sports training, real estate, mental health, medicine, health care, retail, space travel, design, engineering, interior design, television and film, media, advertising, marketing, libraries, education, news, music, and travel.

So, what about journalism then?

Journalism is the production and distribution of reports on recent events. The word “journalism” applies to the occupation (professional or not), the methods of gathering information, and related literary styles. The role and status of journalism, along with that of the mass media, has undergone many changes over the last two decades, together with the advancement of digital technology, publication of news on the Internet, and the rise of social media.

How are you supposed to write that article when those doubtful characters brawl on your desk?

During the last ten years we could see and feel the development of data-driven mobile-optimized visual journalism. For instance check out this Medium post: “How the BBC and the Guardian tell stories on mobile.”

Going back to Amy Webb: During the next decade, we will start to transition to the next era of computing and connected devices, which we will wear and will command using our voices, gesture, and touch.

So, obviously, there is a new challenge, which some panels during this year’s SXSW were tackling. Check out @Webjournalist’s Etherpad listing of relevant sessions, including:

Let’s directly jump into a description of one of the panels: Reality But Better: Augmenting the World with News, written by multimedia journalism professor Steven King. Even in their “infancy augmented and mixed reality tools and techniques have the potential to radically reshape the news business,” he writes. “Deeply immersive stories can be made more interactive and compelling with AR techniques and tools. But in the near future, contextually relevant news — determined by time and location — will be delivered by AR/MR on new hardware systems that seamlessly blend virtual and physical worlds.”

The short video HYPER-REALITY by Keiichi Matsuda been widely shown and shared and allows a glimpse into a dystopian XR future

The AR/MR revolution he describes — which I’d rather call “XR journalism” — will change news delivery and consumption. And I would further argue that it will change way more — such as the role of journalists and the topics covered, or should I say the services provided by future journalists in an ever-changing infrastructure and a changing news paradigm.

These are early days and I strongly recommend that you check out the very first examples of AR journalism that King lists here. You can directly download them and make use of them on your mobile phone. Also, learn more about AR journalism on my twitter account, ARJournalismus.

Three things you can do right now to prepare for the near future XR Journalism wave

These are early days. To better understand, predict, plan, and make the future, here are three the things that you can do right now:

Andrew Hart is working on an AR Location toolkit

Look at the world and tell me about it: First of all, watch this talk by Benedict Evans: 10 Year Futures (vs. What’s Happening Now). Evans not only sets things straight by arguing that the future is always boring until we get there, he distinguishes two different scenarios when making use of AR, MR or XR:

1) Add something to the world — basically this is what current applications in the context of journalism do. You might want to ask yourself why you need to watch virtual Olympic events on the floor while sitting in a cafe.

2) Look at the world and tell me about it — this is where things are heading. Make sure to check out Andrew Hart’s work.

Benjamin Bratton — amongst other things — is Professor of Visual Arts and Director of the Center for Design and Geopolitics at the University of California, San Diego

Understand “The Stack” — our accidental megastructure: It is crucial to get a better understanding of the world that surrounds us and the technical implications or layers we inhabit. I recommend making yourself familiar with Benjamin Bratton’s book The Stack.

Bratton proposes that different genres of computation — smart grids, cloud platforms, mobile apps, smart cities, the Internet of Things, automation — can be seen not as so many species evolving on their own but as forming a coherent whole: an “accidental megastructure” called “The Stack” that is both a computational apparatus and a new governing architecture.

In an account that is both theoretical and technical, drawing on political philosophy, architectural theory, and software studies, Bratton explores six layers of The Stack:

The Stack is an interdisciplinary design brief for a new geopolitics that works with and for planetary-scale computation. It shows how we can better build, dwell within, communicate with, and govern our worlds.

You can only begin to implement fruitful XR applications in the context of journalism when you understand the bigger picture. I recommend digging deeper into the concept of Vision Driven Design Research.

Just as we are now amazed by experiencing mixed realities such as this , people in the 19th century totally freaked out while being physically present in a panorama

Take a media archeology approach:

Last but not least, it is a good idea to dive into academia a bit to get a better understanding of changes in the historical context.

I think that Rebecca Rouse is doing a good job here with articles such as her 2016 “Media of Attraction: A Media Archeology Approach to Panoramas, Kinematography, Mixed Reality and Beyond,” and her 2017 “Design for Emerging Media: How MR Designers Think about Storytelling, Process, and Defining the Field,” both published in Springer Press International’s Interactive Storytelling: Lecture Notes in Computer Science.

TL;DR

We might not use smartphones for news consumption in the near future, but glasses and wearables. There is already a lot of stuff happening in the fields of VR, AR and MR journalism. While we are still in the early days, I propose to make use of the term “XR journalism” instead of all these abbreviations. It better frames and channels and helps when designing experiences for emerging media to come.

Marcus is Head of Business Development at Vragments — a Berlin based VR/AR Studio currently funded by the Google News Initiative.

Immerse is an initiative of MIT Open DocLab and The Fledgling Fund. Learn more about our vision for the project here.