Inconsistent Nielsen data vexes public radio stations and inspires a commercial solution

A new audio processing unit seeks to answer long-lingering concerns about the accuracy of Nielsen Audio’s Portable People Meters, an audience-measurement system that is puzzling pubcasters with wildly fluctuating data.

The new device, Voltair, debuted publicly at the NAB Show April 11-16 in Las Vegas. Manufacturer 25-Seven Systems, a division of Telos Alliance, has been quietly testing the Voltair in markets around the country for almost a year, says company president Geoff Steadman.

But its debut in Las Vegas hit a nerve among both commercial and noncommercial broadcasters who doubt whether PPM is accurately measuring their listenership and are seeking solutions.

Arbitron, later acquired by Nielsen, launched the PPM system a decade ago with the promise that it would eliminate inaccuracies of paper diaries, which relied on diary-keepers to accurately remember and record a week’s worth of listening.

The PPM system equips stations with encoder boxes that add digital codes to broadcast audio. Listeners can’t hear these tones, but they’re detected by pager-like meters that send data to Nielsen.

As the system began rolling out, broadcasters found quirks in its data. At pubcaster KNPR in Las Vegas, ratings showed bizarre spikes in the summers of 2013 and 2014, said Program Director Dave Becker. Its weekly cume, usually between 70,000 and 100,000 listeners, began to jump randomly.

“In August and September 2013, it jumped to 180,000 and then 175,000 the following month, and then the month after that it’s back down to 75,000 listeners,” Becker said.

Closer examination found that the spikes came from a handful of quarter-hours on the weekend, when KNPR’s reported listenership inexplicably leaped from 500 to more than 38,000 during The Splendid Table.

“In both cases, that quarter-hour was far and away the highest of any station in the Vegas market by a factor of three,” Becker said.

Meanwhile, “we had a couple of months last fall where the bottom just fell out — audience numbers dropped to the lowest point in 12 years,” though they quickly recovered, Becker said.

Other pubcasters are less willing to discuss ratings swings publicly. But concern about the validity of PPM ratings has been a frequent topic of discussion at the NAB Show and other conferences in recent years, especially as research has questioned how well the PPM system can inject its inaudible codes into some of public radio’s staple programming.

In a presentation at the NAB Show in Las Vegas, 25-Seven Technical Director Dr. Barry Blesser, a former MIT associate professor with degrees in electrical engineering, laid out the results of his investigation into the workings of PPM. To keep PPM encoding inaudible to human ears, it must be masked under audio content in the same frequency range.

Blesser’s tests showed that not all audio content has the required long-duration audio energy, leaving gaps during which the PPM system is unable to inject masked coding and thus can’t accurately register listenership. Even a change in voices or a long pause between words can disrupt PPM’s encoding, Blesser said.

Blesser’s research led to 25-Seven’s development of its Voltair system, which the company claims produces a bigger blanket of processed audio under which coding can better operate. It also gives stations additional information about how their encoding is working.

A swing in PPM ratings in commercial radio can be a matter of financial success or failure. So the promise of a magic box that could improve PPM encoding or even provide better monitoring of its workings has meant brisk business for 25-Seven, even before Voltair’s commercial announcement. Before the NAB Show started, about 300 units were in the field, according to Steadman.

“We put a brand-new box into the hands of stations, and back came purchase orders,” he says of demand for the Voltair unit. In an environment in which ratings can be as much a matter of superstition as science, stations that already have Voltair in place are reluctant to be identified, though several have at least confirmed that they paid about $15,000 each for the devices.

Stations that have been testing Voltair are often so guarded with their ratings results that they have declined to share data even with 25-Seven itself, Steadman said.

Closer monitoring of PPM is already changing commercial radio. At the NAB Show, broadcasters told of air talent being moved off shifts because their voices didn’t encode well and of songs being pulled from playlists for not being “meter-friendly.”

Frustrating lack of clarity

Because public broadcasters can also study pledge drive results to gauge their success, stations are unlikely to make such drastic changes based on concerns about PPM’s measurements. But Voltair’s NAB Show debut did prompt interest among pubcasters in PPM markets, which include most of the 50 largest markets in the U.S.

At least one pubcaster in a top-10 market was planning to install a Voltair for testing after the NAB Show, though its staff requested anonymity for competitive reasons.

At others, the jury is still out, especially as rival audio processing companies raise questions about whether Voltair’s processing distorts station audio.

“My GM asked me about it the other day, but we are not ready to spend money for one just yet,” said Lowell Kiesow, senior engineer at KPLU in the Seattle market.

Like KNPR, KPLU has seen ratings swings that are hard to explain other than as possible hiccups in the PPM system.

“When the content of our station and that of our competitors hasn’t really changed, it is difficult to find reasonable explanations,” Kiesow said. “Our content managers here chalk it up to a change in the PPM panel group. With a sample size that is very small, one or two new panelists can really change the results.”

Even before the concerns about coding began to surface, panel size has been an issue with PPM. The Media Ratings Council, the board that certifies ratings systems, withdrew an early accreditation of PPM in some markets due to small sample sizes, though it later reinstated that accreditation after Arbitron, then PPM’s owner, adjusted panel sizes.

For station managers who try to maintain tight control over all aspects of their on-air presentation, questions about PPM’s coding have added to frustration over what they say amounts to a “black box” in their audio chain that they are required to use but that provides little detail about how it works. Whether sample sizes, encoding issues or something else entirely is causing PPM ratings to swing, the lack of transparency frustrates pubcasters —  especially at stations such as KNPR that depend on ratings as an important tool to help sell underwriting.

“The biggest concern I’ve got personally is that as a community licensee, our underwriting is pretty important to us,” Becker said. “And when numbers are wildly going up and down like that, it’s hard to get any sense of stability for the underwriting department, because they’re just not sure what’s going on.”

“I would like to see some independent, scientific testing on both the PPM system and the Voltair,” Kiesow said. “The industry deserves more that vague assurances from Nielsen. Perhaps that would be something for NPR Labs to do.”

PPM’s credibility is vital to Nielsen’s business. The company withdrew the old diary-based system in its largest markets after introducing PPM, and its acquisition of Arbitron accompanied the rollout of a PPM-like system for television ratings as well. Nielsen has not commented publicly about Voltair and did not answer requests from Current. But in a letter to stations a week before the NAB Show, it said using the Voltair was “not recommended” in order “to ensure a transparent measurement environment.”

Nielsen lawyers attended the NAB Show and negotiated a statement that Blesser read before his white paper. He said both companies looked forward to further research and the involvement of the Media Ratings Council, which has been provided with a Voltair unit for further testing.

UPDATE: Nielsen provided this statement to Current Thursday:

Nielsen is evaluating and testing the Voltair product. We have had dialogue with the MRC about its role in validating our testing results and about working with the MRC’s independent CPAs to execute a series of lab and real-world tests of the Voltair product. These discussions are ongoing. Additionally, Nielsen is working directly with Telos, the maker of Voltair to evaluate the effect of the Voltair product. As we complete our thorough assessment, we will provide additional updates. Until we’ve completed our analysis, Nielsen cannot endorse use of the Voltair product.