Public TV begins counting ‘invisible audiences’ for education content

WASHINGTON — As public television stations expand their work in education, they need a coherent, standardized system for measuring how people use the various services they offer.

That’s what audience researchers, station leaders, education specialists and CPB officials said during a Wednesday session at the National Educational Telecommunications Association conference. Speakers and attendees agreed that a shared approach to education analytics would strengthen the case for public and private funding of public TV but acknowledged challenges as they work toward that goal.

“If you look across the industry, programming and content people have metrics that are widely agreed on, and development people have theirs,” said Tom Axtell, GM of Vegas PBS. “We’re just beginning to do counting in education that is consistent across the system, and we don’t have a common yardstick. It is essential for our business that the education side of the house starts to do this.”

Vegas PBS and Georgia Public Broadcasting are among the stations leading the way. Both have developed methods for collecting data, and their top education staffers presented overviews of their progress.

GPB worked with Georgia Tech to develop its method, which tracks “interactions” with its education services across several platforms, said Andrew MacCartney, VP of education and digital media. GPB counted 8.3 million interactions with its education services in 2017. More than half of those were through the state network’s education website.

The numbers demonstrate that education “is not a fringe service within a station,” MacCartney said. “It is part of the core foundation of why we do what we do.”

MacCartney said he’s convinced that GPB Education’s reach in 2017 actually exceeded the 8.3 million interactions measured. For example, when a teacher registers for a video field trip, GPB can only estimate the number of students in the classroom who watch it.

Education-focused stations and researchers are trying to measure what David LeRoy of TRAC Media describes as public TV’s “invisible audience” — those who interact with stations in ways that aren’t captured in Nielsen ratings. They attend a parenting workshop, for example, or complete an online teacher training class.

Through TRAC Locale, an effort to identify and measure public TV’s underreported audiences, TRAC began working with Vegas PBS and other clients to develop and advocate for a standardized method for analyzing data from public TV’s education services. TRAC Media co-founders David and Judith LeRoy, who produced the NETA session, are leading the project.

“We have to start counting every interaction in the same way, like the way Google Analytics counts clicks,” said Judith LeRoy. Since public TV stations haven’t been collecting and analyzing data in this way, they can expect “revelations from just counting,” she said.

The approach developed at Vegas PBS gathers analytics including streaming data, digital usage and social media interactions, with data collected from “encounters” with its preschool, K-12 and workforce education content. Vegas PBS tracks 27 education-related data points, including registrations for online workforce training courses, the number of free books distributed through Ready to Learn workshops, and online views of educational videos.

Across all the data collected in 2017, Vegas PBS estimates 15.5 million encounters last year, said Niki Bates, education services director. “That’s a big number when your [Designated Market Area] is 2.2 million. Do you see how having numbers like this can have congressional impact and add to the stories of your station?”

Audience ratings collected through Nielsen are “missing a huge part of what we do,” Bates said.

By collecting this data, Vegas PBS now has a “true picture of our impact and footprint.” Her staff uses the data to analyze usage trends and make decisions, she said, but the metrics are most valuable as proof of performance to station boards, funders and legislators.

Bates recalled a conversation with a CPB legislative staffer who said Hill testimony on education services would be more compelling if public TV had “hard data” to demonstrate reach and impact.

“I’ll acknowledge it is a struggle,” said Michael Fragale, CPB VP of education. When CPB President Pat Harrison testified before a House Appropriations subcommittee last year, “most of the questions we got were about the education work. We can tell stories, but we haven’t got the numbers.”

CPB’s Station Activities Benchmarking Survey, which all CPB-funded stations complete annually, is woefully out of date on education services, Fragale said. “Honestly, we are not asking the right questions.” The survey asks stations about the satellite-delivered Annenberg/CPB Channel, which delivered instructional TV programs through the 1990s. His staff is looking at updating SABS questions about education. Fragale endorsed the effort to standardize how stations measure engagement with educational content.

A question from the audience underscored the challenge of getting every station on board. Pointing to the methodologies of the two stations, an attendee asked why clicks on email newsletters count the same as attending a daylong training session. “There needs to be some weighting,” he said.

“Right now, we are stressing the importance of counting before we get into the nuance,” said Judith LeRoy. “We can’t let the fact that it’s hard to do stop us from doing it.”

“How do we all arrive at common definitions?” Fragale asked attendees. “As hard as we try, we realize that we are leaving some things out. We need all of you to help us figure out what we can agree on.”

CPB VP Deborah Carr, who oversees SABS data collection, said CPB’s staff is updating the SABS survey, including questions about stations’ work in education. She anticipates a “series of conversations” with stations and other stakeholders to determine how to move forward on a systemwide approach.

An earlier version of this story mistakenly reported that CPB is working with officials at the U.S. Department of Education to revise its SABS questions about education. It is working internally to update the questions.