Warning: A non-numeric value encountered in /home/customer/www/ticevision.com/public_html/wp-content/themes/Builder-Cohen/lib/builder-core/lib/layout-engine/modules/class-layout-module.php on line 505

Archive for Research business – Page 2

Label Surveys As Well As Data

It was with great interest I read of the new “data transparency label.” This label is being released for comment by several of the media alphabet associations – the AMA, ARF, and CIMM.

data transparency label example


In the manner of the nutrition labels mandated by the FDA, these labels are hoped to increase clarification about the torrent of data being aimed at big data applications in media, particularly advertising targeting. Adopting a very brief but standard reporting structure, the labels will give users of data a high-level assessment of the quality of the numbers being injected into their algorithmic black boxes. (And by the way, notice there is no equivalent transparency effort about those black boxes; but that’s another story.)

Survey Nutrition Too?

This is important news in that corner of the research, data, and analytics world. What would I like to see? An equivalent nutrition label for publicly released surveys, perhaps sponsored by the Insights Association (the 2017 amalgamation of CASRO and the MRA). The label would provide a required minimal level of information to release with research conducted by its members. This would include items such as:

  • Who paid or sponsored the poll
  • A description of the sample
  • Mode of collection
  • Probability or non-probability sample
  • Dates for fielding
  • Standard error for probability samples, or some “equivalent” for non-probability samples

This information should be enough to quickly evaluate the bias and relative level of quality of a publicly released survey. In fact, some of this information may already be required, but in reality is rarely available in press articles or from the entity releasing the survey.

Too Busy to Process

The Press is too inundated with press releases and too busy filling a 24/7 demand for content to bother to evaluate PR surveys anymore (read MediaPost‘s disclaimer on their Research Intelligencer newsletter). It’s all just grist for the content mill. But maybe with a very simple label, they will be tempted to think once in a while. At the least, we could do the thinking with the right information.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Read his new book, “The Genius Box” – details here
Get notifications of new posts – sign up at right or at bottom of this page.

Dave the Research Grouch: Variety and Cowan

Variety logoLast week, Variety (and multiple others) published a report on a new study from Cowan & Co. on Netflix use, and it’s hard to decide at whom to get grouchy. At Variety, for writing up an article with no context, or Cowan for dropping survey results without publishing any details about their study.

Let’s look at the headline first – “Netflix Is No. 1 Choice for TV Viewing, Beating Broadcast, Cable and YouTube (Study)”. What, according to the article, did the survey results actually say? That people self-reported they used Netflix (27%) “more often” to view than cable TV (20%) or broadcast TV (18%).

Let’s parse this out a bit. First, consider that Nielsen reported in Q1 2017 that 90% of viewing time is still on traditional TV networks. Sure, there are issues with Nielsen but even so it is reasonable to assume that it’s not too far off. This means that in terms of actual viewing time among the total population, Netflix is nowhere near the most-watched platform despite what people may say they “use most often.”

Second is the rather subjective decision to compare broadcast and cable separately against Netflix. It’s been my experience that people with a streaming agenda tend to also be the ones who say viewers can’t tell or don’t care about cable vs broadcast. But that would ruin the headline, because it would change to “Legacy TV Networks Are No. 1 Choice for TV Viewing [38%], Beating Netflix [27%] and YouTube.”

This point is emphasized further when the data for homes with pay TV are shown. Most trusted studies show that a majority of Netflix homes still have pay TV in some form, and here the difference is even more pronounced, with 45% choosing legacy broadcast or cable and 24% Netflix. No attention-getting, disruptive headline from that.

The Frowns are Awarded

Thus a big Research Grouch frown is aimed at Variety (and other sites) for publishing these data without any context at all – context one would hope the beat writers in this area would know enough to include.

Cowan doesn’t escape without a frown either, for my pet peeve – promoting a study without publishing anything about it on their own site. I could not find anything on their website or a press release with which to follow-up. I understand that we don’t need a dissertation, but if you’re going to promote research, then at least have some basic details available to read outside the lens of the press, who (from experience) are notoriously fast-and-loose with their interpretation of research results. What age was the sample? When was it fielded? How was it weighted?

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Quick takes from ARF 2018 AudienceXScience conference

ARF LogoLast week, I attended the ARF’s AudienceXScience (AxS) conference in Jersey City. A new incarnation of the former Audience Measurement conference, it’s one of my two favorite conferences of the year. Here are some highlights from the main keynotes, plus an observation about the conference itself.

The Future of TV’s Currency and Measurement
Linda Yaccarino (NBCU), Dave Morgan (Simulmedia)
Sound bite: The opening salvo in the recurring theme of traditional television being a better value versus digital. Traditional television needs to counter the narrative being driven by digital. The TV industry should not be tied to legacy audience measures or back office systems if they are not in step with the changes occurring in the industry.

Nielsen’s View on Media Measurement
Megan Clarken (Nielsen),  Scott McDonald (ARF)
Sound bite: Continued Nielsen’s recent themes of being open to new approaches and new partnerships to deal with business issues – without necessarily committing to either.

The Future of Media: An Epic Battle
Laura Martin (Needham & Company)

Sound bite: Despite advantages in many business KPIs, the FAANGs will not “defeat” legacy television because of the need for storytelling excellence and creating emotional connections. Their only path to success is acquiring a legacy media company and let it do its thing.

An Evolving Arena: Program Currency and Measurement
Lisa Heimann (NBCU), ‎James Petretti (Sony), Will Kreth (EIDR)

Sound bite: Cross-platform is important for program development and promotion just as it is for advertising sales. But a big part of the puzzle continues to be missing due to some streaming partners refusing to cooperate and share data.

An Agency Perspective: The State of Media
Lyle Schwartz (GroupM), Joe Mandese (MediaPost)

Sound bite: GroupM is asking more from the data but extracting less; research has not evolved to match consumer and business changes. Need to get to a seamless “device-agnostic” measurement, but to do so, marketers will have to reformulate their device-centric planning and viewpoints.

Reach & Frequency Balance: How to Get it Right in the World of Advanced Television
David F. Poltrack (CBS), Radha Subramanyam (CBS)
Sound bite: Ad buyers have always traded off between reach and frequency, but recent years have moved from a focus on reach to a focus on efficiency. This leads to less reach and more frequency, and undervaluing of television. Excess exposures have no value, so advertisers should be converting those exposures to new exposures via TV.

How to Use 6’s – ARF 2018 Primary Research
Paul Donato (ARF), Dan Schiffman (TVision)

Sound bite: Linear TV 6 second ads seem to work at present, but have had some advantages such as being placed in premium primetime content, in short pods, or in favorable pod positions. There are a lot of areas still to explore as these 6s become more prevalent.

Consumers, Cross-Platform & Trust
Bryan Wiener (comScore), Jason Lynch (Adweek)

Sound bite: Wiener sees the audience measurement space as open for innovation, but he did not seem to want to be held back by legacy expectations of quality or accreditation before releasing new products.

Oxford Style Debate: Has Marketing Taken Targeting Too Far?
Arguing for: Gian Fulgoni (former comScore), Radha Subramanyam, (CBS)

Arguing against: Dave Morgan (Simulmedia), Yin Woon Rani (Campbell Soup)
Sound bite: A new format for the conference, and it went pretty well – although it seemed too long since many of the same arguments kept being made. Those arguing “For” focused on the bad experience retargeting gives consumers, and that narrowing the base of consumers exposed to your brand can have a negative long-term impact. Those arguing “Against” said not to condemn a whole concept because of bad implementation; in the long-term, targeting is the right way to go as problems get corrected. As for the result of the debate, a fairly even split in the audience before the debate was converted into a clear win for the “Against” team after the debate.

How Far Can You Go? (before government steps in, that is)
Tim Wu (Columbia Law School)

Sound bite: The European model of GDPR with regards to data privacy is unlikely to gain traction in the USA. Instead, we may see more stringent fiduciary duty defined for those who collect or use consumer data.

Thoughts on the conference itself

The ARF AxS (or AM) conference series has been fighting a battle to maintain relevance over the past few years. Perhaps most notably, its usual dates are now used by both the TV of Tomorrow conference in San Francisco and the VideoNuze Online Ad Summit, siphoning off potential speakers from important digital media firms. Even among the core group of past AM participants (media companies built around TV networks), it seems attendance is down. Media consolidation doesn’t help, and neither do changes in budget or leadership priorities, but…

I often joke about seeing the “same cast of characters” at ARF conferences over the 20+ years I’ve been going to them, but perhaps it’s not so funny. I am a firm believer in the ARF and its mission, but I’m sure the ARF can appear to younger people as a crotchety old uncle trying to enforce dedication to methodological rigor and quality established by the generation of researchers who came up to lead the industry in the 1960s and 1970s. Not that rigor and quality are bad things, but I’m sure it sounds preachy at times, especially to younger industry members who are in companies founded on disruption.

The ARF has made strides in recent years in outreach to younger generations of researchers (Young Pros, for example). Under new leadership, it has refocused its overall efforts in the past year to be more relevant. And I’m sure the ARF tries to get its digital members to participate in AxS.

I Love You ARF, But…

But judging by AxS, the ARF still has a way to go to reflect the diversity in our industry – not only by demographics but also in types of firms that show up for its conferences. If it continues to be mostly older white guys talking about television-centric topics (I’m guilty on all counts, by the way), AxS will continue to lose ground to competitive conferences in the near term — and the ARF its relevance to the next generation of advertising researchers in the long term.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Dave the Research Grouch: Recruiting isn’t Representativeness

Bloomberg View logoThe Research Grouch got a double whammy Monday morning. First, having seen the headline “We Are Finally Getting Better at Polls” on Bloomberg.com, I clicked right to the piece. Once there, however, I discovered it was a self-promoting opinion piece by a UK polling company. 

Already grouchy from feeling somewhat misled, the content of the piece ratcheted up the grouch level. The author discussed what he considered to be new, innovative ways to reach “representative” samples for polling. These new ways included using IM instead of email to reach out to respondents (OK, that makes sense); showing respondents how their responses sit against others taking the poll (debatable, especially if it’s raw instantaneous data); giving people surveys on topics they like in order to maintain interest (again, debatable); emotional testing rather than direct questions (not sure how this solves a sample issue); recruiting from non-political or non-news websites, or from social media (diversification of online recruiting does not create representativeness).

About the only indisputable point made in the piece is politically active people need to be recruited in their correct proportions to get the best data. If this is considered news to political pollsters in the UK, then no wonder they had issues predicting recent elections.

I may be a grouch but I’m not against innovation. These suggestions are certainly ways to potentially increase respondent engagement and diversify online sample sources. But, despite claims from online research firms everywhere, a volunteer opt-in online sample is by definition not representative regardless of panel size, recruitment techniques, weighting, or other manipulations.

Such samples may often give the same answers as a truly representative sample, but that doesn’t make them representative. Neither does the use of the terminology, such as response rates and error margins, lifted from traditional probability-based research and which really don’t apply to volunteer samples of any kind. A serious problem for this industry is that there is a whole generation of researchers that don’t realize this.

There really are probability-based online panels

I do recognize it’s a different world today than 20 years ago. Opt-in online samples are generally “good enough” for many applications, and I’ve used them myself many times. But, at least in the USA, there are sources of online access panels recruited using traditional probability techniques (including GfK’s KnowledgePanel, NORC’s AmeriSpeak, and RAND’s American Life panel) which are available for important research, whether political polls or key business decisions.

Yes, these panels are expensive compared to opt-in sample, but you get what you pay for – and as many pollsters, candidates, and businesses have found over the years, the most expensive research is bad research.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Reflections on the MIE Media State of the Union session

family watching TVIn the Industry State of the Union session at the 2018 Media Insights and Engagement Conference, panelists were asked to share with the audience one takeaway from the current circumstances of the industry.

Click on this link to see my summaries of the takeaways along with some commentary, via one of my guest blog posts for the conference – Reflections on The Media Insights State of the Union

Other MIE conference posts

Please note these posts were originally to be posted during the MIE conference itself but are being published on delay by the conference organizers. Other of my guest blog posts from this year’s conference include…

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Ad Experiments Sing for NBC

Click on this link to see one of my guest blog posts from this year’s Media Insights & Entertainment conference – Ad Experiments in Musical Sing for NBC.

Please note these posts are delayed due to issues with the conference website; they were originally to be posted during the conference itself. Look for several more posts from the 2018 conference over the next few days – updates via my twitter feed or via the conference twitter.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Will Control of Data Control the Future of Media?

Please click on this link to see my second guest blog for this year’s Media Insights & Entertainment conference – Will Control of Data Control the Future of Media?

MIE conference logo

I will be blogging on-site with session recaps from the 2018 conference February 6th through 8th… look for my updates via my twitter feed or via the conference twitter.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Scenes from the 2018 CIMM Summit

CIMM logoThe following are some quick notes and comments I jotted down from CIMM’s Cross-Platform Media Measurement & Data Summit (CIMM = Coalition for Innovative Media Measurement). The CIMM Summit was held February 1st at the Time Warner Center in New York City, and was well attended by the movers and shakers of the media research world.

Presentations and videos of some or all of the presentations should be available soon on CIMM’s website, cimm-us.org.

Notes and Comments

Opening with Jane Clarke, CEO of CIMM
Jane reviewed CIMM’s manifesto and progress made in the past year. [I think how actual much progress will depend on one’s viewpoint (buyer, seller, vendor, or MRC (Media Ratings Council)]

Fireside Chat with Rishad Tobaccowala, Publicis
– he believes there will be a 20-30% reduction in impression-based advertising in the next few years
– “data connecting to data” is the next level of connectivity
– advertising has been in the segmentation business (breaking up large audiences); it’s now entering the aggregation business (collecting smaller, fragmented audiences to create reach)

Buyers and Sellers Speak Out (panel)
– highly entertaining panel!
– Cost per rating point (getting a program in front of a viewer) for ad-supported content is skyrocketing (Joe Marchese, FOX)
– Data about viewers of an ad impression are quickly becoming almost more valuable than the impact of the impression itself (Lou Paskalis, Bank of America)
– You cannot create the same reach of a Super Bowl ad with digital, even if you had a year [assuming full viewing of ad] (Marchese)
– We are now being forced to create valuable, engaging marketing content (Paskalis)

CIMM Attribution Provider Comparison Study (panel)
– Attribution models are like Christmas trees; you can turn them to look good and hide the bare spots (Newcombe Clark, AIG)

Disney-ABC Multiplatform TV Attribution, Phase 2
– There are three main drivers to multiplatform ROI: Audience size, consumer commitment to content, and consumer perception of quality (Cindy Davis, Disney/ABC). [this was highly quantitative but very reminiscent of survey-based work we did at Knowledge Networks in early/mid 2000s]
– “Smarts”, “edge”, and “relatability” are the three of the eight Magid Emotional DNA attributes that are best indicators of multiplatform ROI
– Report supposedly available at ABCAllAccess.com

Creating a Data Relationship with TV Viewers (Channel 4, UK)
– An audience analysis that was interesting if not particularly groundbreaking
– Did show some very cool personalized ads served on digital streaming
– Trying to sell on “ABC”: Audience demos, Behavioral info, Content/Context (building ads or pods related to the content being viewed)

Coffee Break

Industry Associations Speak Out (panel)
– “Muscle memory” [mentioned several times earlier in the day as a reason why various stakeholders don’t adopt new methods] is a good thing, because we need to consider both legacy standards and all viewers [eg, measurement needs to include all viewers, even those still watching VCRs, not just who consumes digital content] (George Ivie, Media Ratings Council)
– We as an industry need “TED Talks” to discuss marketing successes, not just continual talk about the challenges we are facing. (Bob Liodice, ANA)
– We eventually will need MRC audit and accreditation of sales or brand lift providers. If we are validating the data going in, then the loop should be closed by accrediting the lift calculations (Ivie)

Who’s Getting It Right? (panel)
– We need progress not perfection (Kate Sirkin, Publicis)
– Gaps that need to be filled:
— Complete multiplatform system, both pipes and data (Brian Hughes, MAGNA)
— Vendors need to take the time to understand our business to know what the business questions are (Lisa Heimann, NBC)
— Measuring attention or engagement, Magid’s Emotional DNA doesn’t scale. (Howard Shimmel, Turner)
— Be prepared to validate your methods (Daniel Slotwiner, Facebook)
— Transparency and validation. Measurement is now a team sport (Elissa Lee, Google)

Programmatic TV (panel)
– It still takes a long time to evaluate a campaign, up to six months (Dan Aversano, Turner)
– We could use a quick read on campaigns using proxy data (Greg Pharo, Coca-Cola)
– If you make one [national] ad addressable, then the whole program can’t be C3 rated by Nielsen’s rules (Aversano)
– There are too many layers, each with their hand out for a piece of the pie; this can force us to do what we CAN rather than what we would LIKE (Mike Bologna, One2One Media)
– Cycle times are becoming more and more compressed between pitch, sale, and execution (Aversano)

Is the TV Industry Ready for Ad Ratings?
– Results of 27 interviews with industry leaders by Artie Bulgrin
– In 1987, the PeopleMeter came on; 2009 C3 ratings; 2017 separate measures of content and of ads
– Having a standard cross-platform currency is seen as important but NOT critical
– Having an accurate measure of net reach and duplication IS seen as critical but doesn’t have to be “currency quality”

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Dave the research grouch: another viewing data fluff piece

Inscape logoTime to call out another example of “publicity research” that seems to say something but proves very little. Today’s fickle finger gets pointed at Inscape, Vizio’s division that is trying to monetize their TV set data, and data released to Deadline and Broadcasting & Cable about 2017 viewing.

Just quickly looking at the data in the article, I suspected that any measure that had “Teen Mom 2” in the top five programs viewed via DVR, VOD, and OTT may be somewhat skewed. I decided to look deeper and see if there was any definition of the sample – who was measured, etc.

The short version is there is such a paucity of supporting information that it’s hard to tell anything about these reported data. The article has some of the usual phrases thrown around to impress those who don’t know any better about audience measurement – “second-by-second viewing,” “7.7 million households,” “can go granular and capture more precise information on viewing.” As the old saying goes, one can be very precise but not very accurate. Large samples and second-by-second measures don’t guarantee accurate, reliable measures – only a lot of data points.

What we can glean from the article is that these data represent sets in 7.7 million homes, or roughly six percent of all TV households. But we don’t know what proportion of sets these Vizio sets represent (out of the roughly 2.7 sets per TV home), which rooms these sets are in, or who is viewing them. Plus, we don’t know the profiles of those who have given opt-in permission to be tracked by Inscape – how different are they from Vizio owners overall, or the general population? This all informs the value of the information.

With their ACR measurement, Vizio/Inscape claims to be able to measure broadcast, DVR, OTT, and VOD viewing on a set. Again, no explanation of how this is accomplished (eg, how is the ACR match attributed to a source) or what other limitations there may be in the measurement. Are there any gaps that would be helpful to know to assess the data?

I did go further afield to try to find further information via a press release or other material aside from what was published on Deadline or B&C – but there is no other information I could find at the Vizio or Inscape websites.

Free your data! (at least the basics)

Thus the bottom line is there is little supporting data or context, and regular readers know I don’t think too highly of data published without any context. Obviously a press release doesn’t have to be a dissertation but there should be some basic information available.

As a result, in my opinion these viewing data only really represent some measure of some viewing by some unknown population – not exactly the kind of information that can be used “as is” for meaningful extension of industry knowledge. Presumably Vizio/Inscape has more detailed and useful data they share with their clients and partners – and it would be nice to see some of these basics also shared when pursuing press attention.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

What those PwC-Netflix stories yesterday didn’t mention

Please note the TiceVision blog is on a reduced publication schedule through Jan 2.

Yesterday, there were a raft of stories about a new PwC report that claimed Netflix now equaled pay TV subscribers. This seemed a bit off to me, based on my own research done earlier this year, and my recall of Netflix’s own numbers. For Q3 2017, Netflix reported 53 million US subscribers – roughly 20% of the 250 million adults in the US or, if you assume one sub per home, 40% of US households. Even accounting for churn over a survey period and people sharing passwords, how did, as these articles reported, PwC end up with 73% of Americans being Netflix users?

I decided to dig a little deeper, which many of the reporters seem unwilling – or not aware enough – to do. The digging showed a different picture than painted in many of the articles.


The first big consideration is that on the PwC page describing the study, it does clearly state that the sample are people age 18-59 who have a household income over $40,000/year – far from a representative sample of all Americans or all households. Perhaps some reporters may have mentioned that, but none of the half dozen articles I read had made that distinction. This is important, since that age range and income bracket would be much more likely to subscribe to have internet in the home and thus able to subscribe to Netflix.


Another aspect not discussed anywhere in PwC’s landing page for the report or the downloaded report itself is the methodology or sample used. I’ll presume it was an online sample. Again, this would not be representative of Americans as a whole and since by definition online sample consists of only internet users, this sample would of course more likely to subscribe to or use Netflix. And no mention anywhere if Spanish-dominant persons were interviewed to be inclusive of “all” Americans.


Another issue in the press articles is that the terms Netflix “subscribers” and “users” were used indiscriminately – some used one of those terms, some the other. The PwC report specifically has the term “users” – an important distinction the reporters for some articles missed, as users may or may not be subscribers (in past research I found about 15% of Netflix users say they use other people’s logins). But by using “subscribers,” some of the reporters add to the misinterpretation of the data. And in neither the articles nor the PwC supporting material is how “users” are defined – are these Netflix people “ever” users, regular users, or so forth? Or what is the difference in time spent for Netflix versus pay TV channels?

Always consider the limitations of any report

Words have meaning. Methods have consequences. Firms that publish – and the press that covers – reports such as these should keep that in mind. While this PwC study may show some interesting trends for the slice of the US public to which it applies, it should not be confused as a definitive profile of Americans’ access or usage, which is how many of the press articles presented it.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.