A team at NPR is the winner of a grant to develop virtual-reality stories that will transport listeners to audio-rich soundscapes.
The NPR project is among 11 winners of the Journalism 360 Challenge awards announced Tuesday. Presented by the Knight Foundation, Google News Lab and the Online News Association, the grants of $15,000–$30,000 support the use of immersive storytelling in news.
Other winners include efforts to make immersive storytelling more accessible to community and ethnic media and to help journalists and others create location-based data visualizations in a virtual-reality format.
NPR’s video team includes Bronson Arcuri, CJ Riculan, Maia Stern and Nick Michael, who worked on the Knight Foundation grant application.
NPR’s team is receiving $15,000 to develop “best practices for immersive storytelling audio by producing two virtual reality stories with a particular focus on sound-rich scenes,” according to a statement from the grantors.
Place is important in 360-degree video, NPR Video Producer Nick Michael told Current. And audio is also crucial for understanding; spatial audio allows users to see and hear themselves in space. 360 experiences tend to focus on digital immersion, Michael said, but audio immersion has lagged behind. That gives NPR and the “world class audio experience in our building” an opportunity, Michael said.
NPR has already been dabbling in 360 experiments. Last year, NPR Music partnered with immersive media company RYOT to produce a 360-degree Tiny Desk concert featuring the band Wilco.
A not-yet-published experience for the program Jazz Night in America captured an immersive field-recording performance in Glendora, Miss., with trumpeter Wadada Leo Smith playing at the site where Emmett Till’s murderers dropped the youth’s body into a river. Smith, a Mississippi native, played a selection from a song he had written in Till’s honor. The trumpeter floated down the river in a canoe at sunset.
Los Angeles NPR station KCRW has also experimented with 360 music performances.
Most virtual reality experiences don’t include audio that changes, Michael said. A little over a year ago, YouTube introduced spatial audio for 360-degree and virtual reality videos.
“That’s really valuable for locating people in a scene,” Michael said. But a work of journalism may want to add a host or reporter’s voice. Unless that person is physically in the environment talking to the viewer, it doesn’t make sense for their voice to be in the space, Michael said.
Producers can work with both spatialized and non-spatialized audio fields, according to Michael. One challenge they face is how to set up audio and video in editing software when one field is spatialized and the other is not.
“We’ve been interested in trying to figure out lower-touch ways to do 360, and as an audio organization, we’re particularly interested in how these different spatial and traditional audio components might mix together,” Michael said.
NPR’s team isn’t the first to experiment with dual audio fields, Michael said, but it aims to figure out how to easily gather audio and process it via dual fields. The team plans to share its findings through NPR training team sites and other mediums.
NPR is also considering the level of resources to devote to 360 presentations. “It’s very cool, but even as a team internally, we’ve been asking ourselves, in theory, ‘How many stories have we seen that can only be told through 360, as opposed to flat video?’” Michael said.
They’ve concluded most stories don’t need 360 but do benefit from the technology. Stories about energy lend themselves to 360 — one could transport listeners to the installation and maintenance of wind turbines. But the team has yet to settle on stories to experiment with.
NPR’s team also wants to discover and standardize production practices so the work feels less experimental.
Michael expects the team will include three producers, possibly one reporter, and a larger crew that reviews and troubleshoots. Audio engineers may also consult and travel.
Grantees will build on their ideas over the next six to 12 months, attend the ONA conference in Washington, D.C., in October and share their findings at a demo day early next year.