Augmenting Reality — Or distracting from it?

Lessons for documentary storytellers from the Minnesota Twins’ New Experiment with AR

By Jessica Fiala and Sarah Lutman

Augmented Reality (AR) has become increasingly common across retail, gaming and the arts, and yet it still frequently falls into the “emerging technology” category. Though it has been widely tested, AR still has an aura of novelty and still faces a range of kinks to be worked out as it proliferates in new contexts and attracts fresh users.

In late summer 2022, the Minnesota Twins baseball team joined the world of augmented reality with the launch of the ARound Stadium app. Developed for use during a baseball game and inside the stadium, the app includes a handful of short games as well as interactive features. Three of us from 8 Bridges Workshop attended a Twins game in September to try out the app and see whether it holds promise as a model for live performing arts. We explored what kinds of experiences AR could bring to sports audiences, and what elements might translate to other live entertainment venues.

ARound joins a portfolio of apps in which Major League Baseball has invested heavily, MLB and Ballpark. MLB is a subscription-based app that has all teams’ schedules and results, deep information about players and their stats, links to access local TV and radio broadcasts and a sophisticated play-by-play simulation called GameDay that you can watch live for any baseball game at any time. The Ballpark app allows users to pre-order food and drink, purchase tickets, and find parking options, among other features.

What did we find? With the ARound app installed, and your phone’s camera pointed toward the playing field, games dominate. You can knock down towers of blocks, go fishing or try to hit a baseball. Other interactive elements allow users to share emojis, see icons indicating that other app users are in the stands and throw virtual items such as donuts, hot dogs, and beach balls onto the virtual field for no particular reason other than fun.

The lighthearted cartoon graphics and brief games seem geared toward an audience of kids needing a distraction from the at-times slow-moving pace of baseball. While spokespersons have ruminated about expanded uses, such as showcasing player stats, the current app is largely unrelated to action on the field. In fact, ARound’s interactives obscure the field and it is even easy to forget that a game is underway. Such features might be useful for entertaining bored children (or adults), serving a similar purpose to the many major league play-pens and interactive installations, but the disconnect seems strange, as opportunities to teach about baseball or draw fans and new audiences closer to the action are lost when the game on the field becomes a secondary background for the app’s antics.

Ahead of and behind the times?

This disconnect may be just a matter of getting something new up and running. The app has been touted as likely the “first shared augmented reality application for live sports.” Yet there looks to be a need for stadiums and teams to catch up to the kind of technology integration that baseball fans and gamers will expect.

Also, if users want to be active on the app throughout an entire baseball game, they will need a way to charge their phones. (Our batteries quickly drained.) This also begs the question of how to develop content that could hold a fan’s interest throughout the game, rather than quickly running its course. The app does not gather user data, but next stages are likely to include ways to monetize interaction, potentially through in-app purchases or featured sponsors.

ARound’s launch comes in the midst of an overall drop in MLB ticket sales. The pandemic has dealt a major blow to habits of going out, and bolstered entertainment options from home. Axios linked the app to other efforts by Major League Baseball and teams to both reenergize existing fans and draw in new audiences. But in the context of this pandemic slump, aren’t the fans and families who still show up already engaged?

It seems that those who physically make the trek to the stadium may really want to follow the action on the field, particularly on fall weeknights (like the game we attended) with school or work the next day. This raises the question of what baseball pilgrims still devoted to in-person attendance may want from an app. The answer could be many things: a chance to connect with fellow fans, detailed information about players and history, or, when kids lose interest, possibly a game that totally blots out the field.

How does an augmented reality feel?

So, when does AR live up to its name? When does it truly augment experience by enriching a connection to place or to other people? When does it draw us in — facilitating a deeper experience of an unfolding moment — and when does it distract or disconnect us from the world that surrounds us?

In its possibilities and pitfalls, ARound offers a number of lessons and questions for documentarians to ponder. When devising an AR experience, how might it draw people into a deeper engagement with place and context? How can it connect people, both those who know each other and passersby who find themselves in a shared experience? How can it help users to better notice, across senses and emotions, what is happening or present in a place?

This is not to suggest that AR needs to take on a serious tone. Different forms of attention are not inherently wrong. Storytellers regularly guide the flow of audiences’ experiences, orchestrating style, media, and momentum to convey narrative and elicit emotion. In terms of AR, some places are marked by painful histories that resonate today. Grappling with these pasts can require a range of methods of curating attention, at times leaning into hard truths, at times stepping back. What matters is therefore not a hierarchy of forms of attention but rather a question of goals and audience. What are you trying to achieve and who is it for?

Sometimes, distraction can be a worthwhile goal. But, if distraction is the aim, why would someone engage with particular AR content rather than any number of other apps or options? ARound will need more features to keep us distracted in more satisfying ways or more content related to the game-at-hand to keep us engaged. Still, knowing that sports teams are working on such projects should cause artists and cultural organizations to take note. How might the cultural sector show a way forward instead of trailing commercial experiments? How might we engage and immerse, rather than distract our audiences?

For more news, discourse, and resources on immersive and emerging forms of nonfiction media, sign up for our monthly newsletter.

Immerse is an initiative of the MIT Open DocLab and Dot Connector Studio, and receives funding from Just Films | Ford Foundation, the MacArthur Foundation, and the National Endowment for the Arts. The Gotham Film & Media Institute is our fiscal sponsor. Learn more here. We are committed to exploring and showcasing emerging nonfiction projects that push the boundaries of media and tackle issues of social justice — and rely on friends like you to sustain ourselves and grow. Join us by making a gift today.