Last week, Variety (and multiple others) published a report on a new study from Cowan & Co. on Netflix use, and it’s hard to decide at whom to get grouchy. At Variety, for writing up an article with no context, or Cowan for dropping survey results without publishing any details about their study.

Let’s look at the headline first – “Netflix Is No. 1 Choice for TV Viewing, Beating Broadcast, Cable and YouTube (Study)”. What, according to the article, did the survey results actually say? That people self-reported they used Netflix (27%) “more often” to view than cable TV (20%) or broadcast TV (18%).

Let’s parse this out a bit. First, consider that Nielsen reported in Q1 2017 that 90% of viewing time is still on traditional TV networks. Sure, there are issues with Nielsen but even so it is reasonable to assume that it’s not too far off. This means that in terms of actual viewing time among the total population, Netflix is nowhere near the most-watched platform despite what people may say they “use most often.”

Second is the rather subjective decision to compare broadcast and cable separately against Netflix. It’s been my experience that people with a streaming agenda tend to also be the ones who say viewers can’t tell or don’t care about cable vs broadcast. But that would ruin the headline, because it would change to “Legacy TV Networks Are No. 1 Choice for TV Viewing [38%], Beating Netflix [27%] and YouTube.”

This point is emphasized further when the data for homes with pay TV are shown. Most trusted studies show that a majority of Netflix homes still have pay TV in some form, and here the difference is even more pronounced, with 45% choosing legacy broadcast or cable and 24% Netflix. No attention-getting, disruptive headline from that.

The Frowns are Awarded

Thus a big Research Grouch frown is aimed at Variety (and other sites) for publishing these data without any context at all – context one would hope the beat writers in this area would know enough to include.

Cowan doesn’t escape without a frown either, for my pet peeve – promoting a study without publishing anything about it on their own site. I could not find anything on their website or a press release with which to follow-up. I understand that we don’t need a dissertation, but if you’re going to promote research, then at least have some basic details available to read outside the lens of the press, who (from experience) are notoriously fast-and-loose with their interpretation of research results. What age was the sample? When was it fielded? How was it weighted?

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Please note the TiceVision blog is on a reduced publication schedule through Jan 2.

Yesterday, there were a raft of stories about a new PwC report that claimed Netflix now equaled pay TV subscribers. This seemed a bit off to me, based on my own research done earlier this year, and my recall of Netflix’s own numbers. For Q3 2017, Netflix reported 53 million US subscribers – roughly 20% of the 250 million adults in the US or, if you assume one sub per home, 40% of US households. Even accounting for churn over a survey period and people sharing passwords, how did, as these articles reported, PwC end up with 73% of Americans being Netflix users?

I decided to dig a little deeper, which many of the reporters seem unwilling – or not aware enough – to do. The digging showed a different picture than painted in many of the articles.

Sample

The first big consideration is that on the PwC page describing the study, it does clearly state that the sample are people age 18-59 who have a household income over $40,000/year – far from a representative sample of all Americans or all households. Perhaps some reporters may have mentioned that, but none of the half dozen articles I read had made that distinction. This is important, since that age range and income bracket would be much more likely to subscribe to have internet in the home and thus able to subscribe to Netflix.

Method

Another aspect not discussed anywhere in PwC’s landing page for the report or the downloaded report itself is the methodology or sample used. I’ll presume it was an online sample. Again, this would not be representative of Americans as a whole and since by definition online sample consists of only internet users, this sample would of course more likely to subscribe to or use Netflix. And no mention anywhere if Spanish-dominant persons were interviewed to be inclusive of “all” Americans.

Terminology

Another issue in the press articles is that the terms Netflix “subscribers” and “users” were used indiscriminately – some used one of those terms, some the other. The PwC report specifically has the term “users” – an important distinction the reporters for some articles missed, as users may or may not be subscribers (in past research I found about 15% of Netflix users say they use other people’s logins). But by using “subscribers,” some of the reporters add to the misinterpretation of the data. And in neither the articles nor the PwC supporting material is how “users” are defined – are these Netflix people “ever” users, regular users, or so forth? Or what is the difference in time spent for Netflix versus pay TV channels?

Always consider the limitations of any report

Words have meaning. Methods have consequences. Firms that publish – and the press that covers – reports such as these should keep that in mind. While this PwC study may show some interesting trends for the slice of the US public to which it applies, it should not be confused as a definitive profile of Americans’ access or usage, which is how many of the press articles presented it.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.