Inconsistent Nielsen data vexes public radio stations and inspires a commercial solution

Print More
The Voltair box on display at the NAB Show in Las Vegas. (Photo: Scott Fybush, Current)

The Voltair box on display at the NAB Show in Las Vegas. (Photo: Scott Fybush, Current)

A new audio processing unit seeks to answer long-lingering concerns about the accuracy of Nielsen Audio’s Portable People Meters, an audience-measurement system that is puzzling pubcasters with wildly fluctuating data.

The new device, Voltair, debuted publicly at the NAB Show April 11-16 in Las Vegas. Manufacturer 25-Seven Systems, a division of Telos Alliance, has been quietly testing the Voltair in markets around the country for almost a year, says company president Geoff Steadman.

But its debut in Las Vegas hit a nerve among both commercial and noncommercial broadcasters who doubt whether PPM is accurately measuring their listenership and are seeking solutions.

Arbitron, later acquired by Nielsen, launched the PPM system a decade ago with the promise that it would eliminate inaccuracies of paper diaries, which relied on diary-keepers to accurately remember and record a week’s worth of listening.

The PPM system equips stations with encoder boxes that add digital codes to broadcast audio. Listeners can’t hear these tones, but they’re detected by pager-like meters that send data to Nielsen.

As the system began rolling out, broadcasters found quirks in its data. At pubcaster KNPR in Las Vegas, ratings showed bizarre spikes in the summers of 2013 and 2014, said Program Director Dave Becker. Its weekly cume, usually between 70,000 and 100,000 listeners, began to jump randomly.

“In August and September 2013, it jumped to 180,000 and then 175,000 the following month, and then the month after that it’s back down to 75,000 listeners,” Becker said.

Closer examination found that the spikes came from a handful of quarter-hours on the weekend, when KNPR’s reported listenership inexplicably leaped from 500 to more than 38,000 during The Splendid Table.

“In both cases, that quarter-hour was far and away the highest of any station in the Vegas market by a factor of three,” Becker said.

Meanwhile, “we had a couple of months last fall where the bottom just fell out — audience numbers dropped to the lowest point in 12 years,” though they quickly recovered, Becker said.

Other pubcasters are less willing to discuss ratings swings publicly. But concern about the validity of PPM ratings has been a frequent topic of discussion at the NAB Show and other conferences in recent years, especially as research has questioned how well the PPM system can inject its inaudible codes into some of public radio’s staple programming.

In a presentation at the NAB Show in Las Vegas, 25-Seven Technical Director Dr. Barry Blesser, a former MIT associate professor with degrees in electrical engineering, laid out the results of his investigation into the workings of PPM. To keep PPM encoding inaudible to human ears, it must be masked under audio content in the same frequency range.

Blesser’s tests showed that not all audio content has the required long-duration audio energy, leaving gaps during which the PPM system is unable to inject masked coding and thus can’t accurately register listenership. Even a change in voices or a long pause between words can disrupt PPM’s encoding, Blesser said.

Blesser’s research led to 25-Seven’s development of its Voltair system, which the company claims produces a bigger blanket of processed audio under which coding can better operate. It also gives stations additional information about how their encoding is working.

A swing in PPM ratings in commercial radio can be a matter of financial success or failure. So the promise of a magic box that could improve PPM encoding or even provide better monitoring of its workings has meant brisk business for 25-Seven, even before Voltair’s commercial announcement. Before the NAB Show started, about 300 units were in the field, according to Steadman.

“We put a brand-new box into the hands of stations, and back came purchase orders,” he says of demand for the Voltair unit. In an environment in which ratings can be as much a matter of superstition as science, stations that already have Voltair in place are reluctant to be identified, though several have at least confirmed that they paid about $15,000 each for the devices.

Stations that have been testing Voltair are often so guarded with their ratings results that they have declined to share data even with 25-Seven itself, Steadman said.

Closer monitoring of PPM is already changing commercial radio. At the NAB Show, broadcasters told of air talent being moved off shifts because their voices didn’t encode well and of songs being pulled from playlists for not being “meter-friendly.”

Frustrating lack of clarity

Because public broadcasters can also study pledge drive results to gauge their success, stations are unlikely to make such drastic changes based on concerns about PPM’s measurements. But Voltair’s NAB Show debut did prompt interest among pubcasters in PPM markets, which include most of the 50 largest markets in the U.S.

At least one pubcaster in a top-10 market was planning to install a Voltair for testing after the NAB Show, though its staff requested anonymity for competitive reasons.

At others, the jury is still out, especially as rival audio processing companies raise questions about whether Voltair’s processing distorts station audio.

“My GM asked me about it the other day, but we are not ready to spend money for one just yet,” said Lowell Kiesow, senior engineer at KPLU in the Seattle market.

Like KNPR, KPLU has seen ratings swings that are hard to explain other than as possible hiccups in the PPM system.

“When the content of our station and that of our competitors hasn’t really changed, it is difficult to find reasonable explanations,” Kiesow said. “Our content managers here chalk it up to a change in the PPM panel group. With a sample size that is very small, one or two new panelists can really change the results.”

Even before the concerns about coding began to surface, panel size has been an issue with PPM. The Media Ratings Council, the board that certifies ratings systems, withdrew an early accreditation of PPM in some markets due to small sample sizes, though it later reinstated that accreditation after Arbitron, then PPM’s owner, adjusted panel sizes.

For station managers who try to maintain tight control over all aspects of their on-air presentation, questions about PPM’s coding have added to frustration over what they say amounts to a “black box” in their audio chain that they are required to use but that provides little detail about how it works. Whether sample sizes, encoding issues or something else entirely is causing PPM ratings to swing, the lack of transparency frustrates pubcasters —  especially at stations such as KNPR that depend on ratings as an important tool to help sell underwriting.

“The biggest concern I’ve got personally is that as a community licensee, our underwriting is pretty important to us,” Becker said. “And when numbers are wildly going up and down like that, it’s hard to get any sense of stability for the underwriting department, because they’re just not sure what’s going on.”

“I would like to see some independent, scientific testing on both the PPM system and the Voltair,” Kiesow said. “The industry deserves more that vague assurances from Nielsen. Perhaps that would be something for NPR Labs to do.”

PPM’s credibility is vital to Nielsen’s business. The company withdrew the old diary-based system in its largest markets after introducing PPM, and its acquisition of Arbitron accompanied the rollout of a PPM-like system for television ratings as well. Nielsen has not commented publicly about Voltair and did not answer requests from Current. But in a letter to stations a week before the NAB Show, it said using the Voltair was “not recommended” in order “to ensure a transparent measurement environment.”

Nielsen lawyers attended the NAB Show and negotiated a statement that Blesser read before his white paper. He said both companies looked forward to further research and the involvement of the Media Ratings Council, which has been provided with a Voltair unit for further testing.

UPDATE: Nielsen provided this statement to Current Thursday:

Nielsen is evaluating and testing the Voltair product. We have had dialogue with the MRC about its role in validating our testing results and about working with the MRC’s independent CPAs to execute a series of lab and real-world tests of the Voltair product. These discussions are ongoing. Additionally, Nielsen is working directly with Telos, the maker of Voltair to evaluate the effect of the Voltair product. As we complete our thorough assessment, we will provide additional updates. Until we’ve completed our analysis, Nielsen cannot endorse use of the Voltair product.

  • Dave Mason

    I know of several stations that are using the Voltair, and in most cases it does make a difference. I also know of one AM station that has an audible “ringing” on voices which shows the presence of Voltair. But it must me emphasized that Voltair doesn’t ADD to a statoins PPM numbers. It does appear to increase the audio level of the PPM pulses that may otherwise be missed by a panelists PPM. This only means a station prior to Voltair might just have a number of those pulses missing due to low level insertion. Some would flinch at the $15k investment, others would believe that getting a few more tenths of a share would make the potential revenue worth it. Using any type of “audio” device to capture listening is going to be shaky at best. Acoustics, environment, audio levels – PPM in pockets/purses -all will have an impact. Couple that with smaller sample sizes than with the diary system brings up a whole new set of issues. Voltair isn’t a way to cheat. It’s a way to get credit.

    • Gregg McVicar

      Next, the Voltair Apple Watch app!

  • John Sutton

    Great article on Voltair. Thanks for reporting it.

    For the record, it is inaccurate to say that stations use pledge dries to evaluate their success. It hasn’t been that way for decades at most stations. Most stations bring in the majority of their listener donations outside of pledge drives. That’s even more true now that so many listeners give as monthly sustainers.

  • Aaron Read

    Dave, I have a hypothesis/concern that Voltair could, in fact, be a method of cheating. We don’t know for sure and I want to emphasize that. But it has to do with the “black box” between the raw data that Nielsen collects from the PPM meters, and the resulting “numbers reports” that subscribers get.

    We don’t really have any idea what goes on in that black box, except to say that Nielsen is doing SOME statistical analysis and weighting to interpret the numbers.

    Ergo, it’s possible…likely, even…that the methodology in that black box could be set up to inherently assume that some percentage of PPM listeners to a given station are present but for whatever reason the PPM meter isn’t reading them. As in, lets say there are 100 people with PPM’s listening to station WXYZ, but since Nielsen knows their own system, they know that at any given point, only 90 of them will normally be counted. The reasons could be anything and most, if not all, could be quite valid; their stereos could be turned down too quiet. Their car interior could be really loud. Who knows?

    Whatever the reason, if it happens to be a reason that Voltair can compensate for; if Voltair makes it so those “uncounted” 10 listeners are suddenly being counted? Now the methodology in the black box is going to OVERCOUNT those 10 listeners and make it seem like the station has 110 listeners when there’s really only 100.

    This is a drastic oversimplification to prove a point, of course. And the point is: we just don’t know what the heck is really going on here, because in large part Nielsen has been so secretive about its methods. All we know is that stations who install Voltairs SEEM to see an increase in ratings, and without any other attributable reason.

    So the bottom line here is that the system CAN be rigged. Since the ratings depend on the perception of integrity…the perception that Nielsen is justified in saying “just trust us, these numbers are correct”…this is a HUGE problem.