Archive for Research business

Scenes from the 2019 CIMM Summit

CIMM logoThe eighth annual CIMM Cross-Platform Video Measurement & Data Summit was held on February 7th at the Time Warner Center in New York. As always, this annual fixture in the media research industry provided an interesting discussion about the state of media measurement.

Among the recurrent themes were:

  • C-3 and C-7 measures, meant to be temporary, are now 12 years old and do not seem to be going anywhere – despite not reflecting today’s viewers
  • Greater transparency is still needed at all levels
  • The need for “ground truth panels” seems to be making a comeback
  • Attribution continues to be the hot topic in measurement

In something of a change from previous editions, no-one from Nielsen or Comscore (or any start-up measurement service for that matter) presented or was part of a panel.

The hand-outs, press releases, and deck from the summit are available on the CIMM website, as are materials from earlier summits.

This was the first CIMM Summit since CIMM was acquired by the ARF back in October. I hope that CIMM and the ARF will continue to offer this summit, and to keep it free so that all those with an interest are able to attend.

Detailed Notes

Below are notes from each of the panels/presentations. These are by necessity distilled down based on how quickly I could take notes, so they do not reflect the totality of the discussions.

After a short kick-off by CIMM CEO and Managing Director Jane Clarke, the first session featured an interview of Krishan Bhatia of NBCUniversal.

  • C-3 and C-7 are outdated by today’s viewing habits
  • C-Flight introduction by NBCU came with little pushback. There is some friction around the work but not about the concept
  • They are working on attribution, campaign measurement, and how to prove performance across all NBCU media
  • He is skeptical that there will ever again be a one-size-fits-all solution
  • 34% of NBCU consumption is now on digital – expect it to be up to 50% very soon

The next session was a panel featuring Rob Master of Unilever, David Cohen of MAGNA, and Laura Nathanson of Disney to discuss business needs for cross-measurement and metrics.

  • RM: There is no common solution. Industry needs to develop a common vernacular to discuss. Can’t be perfect – what is now? near? next?
  • LN: Disney adjusted by moving all media sales under one group. The “plumbing” is an issue – need to plumb and test
  • DC: C-3 and C-7 are no longer sufficient. Need to move to exact commercial minute measurement. In the mid-/long-term, need to look at audible and visual measures across all platforms.
  • RM: Unilever doesn’t care so much about addressability – they have broad markets
  • LN: But then Unilever should use addressability to send different creative to various segments within a broad demo
  • One key thought to close:
    • RM: Transparency and dialog around counting
    • DC: Let’s “start by starting” – need to get moving
    • LN: Just because it’s hard doesn’t mean we shouldn’t do it – it’s the reason we should do it

Next, an overview of this year’s update of the CIMM TV attribution whitepaper was presented by Jim Spaeth and Alice Sylvester of Sequent Partners. Attribution then discussed by Claudio Marcus of Freewheel and Lisa Giacosa of Spark Foundry.

  • What is the state of the art of attribution?
    • LG: I’m excited and hungry [for more]
    • CM: Like in the UK train stations, “Watch the Gap”. There are gaps in cross-platform attribution, and brand/longer-term effects
  • CM: Biggest effect so far on automotive. Auto had moved money from TV to digital – but attribution showed TV drove the digital exposures. Moving back to TV. Media & Entertainment another area – TV program promotion
  • LG: Need to understand content effects. Can’t just follow short-term ROI over a cliff.
  • JS: Need to use baseline sales as a basis for calculating incremental effects of attribution media

Following a break, there were brief updates of the Taxi Complete (AD-ID and EIDR) and Data Label initiatives.

Another panel discussed Deduplicating Reach for Content and Ads, featuring Radha Subramanyam of CBS, Eric Cavanaugh of Publicis, Beth Rockwood of Turner, and Ed Gaffney of GroupM and moderated by Scott McDonald of the ARF.

  • EC: A good quality attribution should be getting deduplication as a byproduct
  • BR: how things fit together is a big issue
  • RS: need both counting and outcome measures. But we need to up-level the conversation: There are lots of products and data, but are we any closer to making sense of media and marketing together? Need a commonsense playbook at a high level.
  • EG: Need dedup in place before this years upfront – or 2020 upfront.
  • RS: Vendors need to listen closely to needs. Their solutions are not necessarily addressing the needs.
  • EC: We also need to know about content to be able to place ads in context.
  • EG: Blindspots are getting smaller but there are new ones popping up every day
  • EC: We are getting one-off fixes to blindspots but need integrated response
  • RS: Integrating projectable and non-projectable samples is doable but needs more investment
  • BR: The technical issues of integration are easier than making the theory work
  • RS: In terms of privacy, one-to-many is less threatening than true one-to-one marketing

Is there One Metric to Rule Them All? Kavita Vazirani of NBCU, Brian Hughes of MAGNA, George Ivie of the MRC, and Sheryl Feldinger of Google discussed this topic.

  • BH: Need exact minute commercial ratings
  • SF: Need equitable (with TV) transparency at exposure and second-by-second ratings
  • KV: Need to measure effort vs return. Shouldn’t we be focusing on cross-platform measures rather than arguing about TV measures?
  • BH: already does second-by-second with MediaOcean, which is an old platform – so it can be done today
  • GI: MRC is working on standard definitions with partners and industry, aiming for impression-based duration-weighted data by 2021. Measures to include exposure, viewability, duration-weighting, complete exposure to an ad.
  • SF: Wants absolute exposure. His work shows that a 5 or 10 second exposure elicits a similar response, regardless of the total length of an ad
  • KV: Disagrees. She claims the only time a 6 second ad worked was as part of a larger integrated campaign
  • GI: There is a big gap in content measurement in digital. For content measurement in a cross-platform world, customer journey analysis is something that should be syndicated (eg, third party)
  • All: agree audio status needs to be known (muted vs non-muted)

The last panel talked about Audience-Based Buying Platforms for TV/Video. This panel included Bryson Gordon of Viacom, Mike Law of Dentsu Aegis, Bob Ivins of NCC Media, and Mike Welch of Xandr.

  • BI: Inertia is real. Need to get marketers to “cross the bridge” and not turn back halfway across. We need standards and transparency.
  • MW: Can help reach low incidence/low viewing HHs
  • BI: Need an automated platform like Google and Facebook. Still too much manual transfers between different applications
  • BG: users on OpenAP have already created 1,872 segments
  • Opportunities in 2019
    • BI: More inventory and optimization
    • ML: Platform, optimization, interactivity
    • BG: Automated workflows, cross-platform delivery, unified posting
    • MW: Platform, true cross-platform delivery

To wrap up the afternoon, Jack Smith of GroupM told us about what he saw at the 2019 CES conference.

  • The three areas to pay most attention to are Assistants (Alexa, etc); Autonomy (self-driving cars); and Simulation (VR/AR).
  • It is important to understand how algorithms work – what products are suggested when Alexa is asked to buy something. Should brands have an avatar to speak for themselves, rather than relying on Amazon etc.
  • Most everything will still be on screens. How are these to be measured?
  • Top takeaways: 1) Interface revolution. 2) Immersion environments. 3) The ethics of tech in general.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

2019 MIE Conference Summaries

MIE Conference logoAs guest-blogger for the 2019 edition of the Media Insights & Engagement Conference (which is put on by knect365), I wrote up summaries of the keynotes and the break-out sessions I attended. You can find the daily summaries on the knect365 website:

Day 1 of the 2019 MIE conference: Day 1 (Jan 29 2019)
Day 2 of the 2019 MIE conference: Day 2 (Jan 30 2019)
Day 3 of the 2019 MIE conference: Day 3 (Jan 31 2019)

Also, read my three pre-conference posts here:

2019’s New SVOD Services: Blitzkrieg or War of Attrition?

Connected TVs: Corporate Connections as Important as Internet Connections

Does AVOD News Reveal a New Phase of SVOD?


David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Most Popular Posts of 2018

2018 is coming to a close and it’s time to take a look back. Which TiceVision blog posts have had the most interest in the past year?

Third Place

In a virtual tie for third place are two posts:

3a. Quick Takes from the ARF AudienceXScience Conference – as the name implies, in this June post I share some of my thoughts on the 2018 edition of this long-running conference, the good (as always, some interesting sessions) and the bad (its lack of diversity in companies and presenters).

3b. Drake vs The Beatles: Let it Be – A post from July, I take issue with press comparisons that claim Drake outdoes The Beatles. These comparisons don’t take into account differences in how the Hot 100 is calculated now vs the 1960s.

Second Place

2. In second place for the year is Dave the Research Grouch: Another Data Fluff Piece. This post, one of the generally popular “Dave the Research Grouch” series, takes exception to press coverage of a data release by Inscape, Vizio’s division which monetizes their TV set viewing data.

First Place

My most popular post of the year, by a margin of almost 2-to-1 over the runners-up, is Foreverspin Tops? More Like Forever Annoying Ads. This post has the longest legs of my 2018 posts, with at least a reader or two every week since being published last February. In the post, I take issue with the bad side of digital advertising, exemplified by the Foreverspin Tops ads that followed me for years.

Happy Holidays!

Whether you observe Christmas, Hanukkah, Kwanzaa, or another winter holiday, I hope all my readers have – or have had – an enjoyable holiday season. And best wishes for your happiness and success in 2019!

  • Don’t miss any of my 2019 posts by signing up for email notifications here
  • Haven’t read my new book about TV, The Genius Box? It’s available in paperback and e-book formats. Book details and ordering info here

David Tice is the principal of TiceVision LLC, a media research consultancy.

ARF-CIMM is good news, but let’s get CREative

The ARF logoThere was interesting news in the audience measurement business yesterday. Several outlets covered the announcement that the Advertising Research Foundation (ARF) will acquire the Coalition for Innovative Media Measurement (CIMM). As a couple of articles noted, this is a continuation of the trend in consolidation in many sectors of the media business.

CIMM logoI’ve been on the sharp end of trying to sell syndicated research studies to a decreasing pool of clients because of consolidation. I can imagine that CIMM was dealing with a similar issues among its membership in the wake of the Disney-Fox, Discovery-Scripps, and other recent deals. The ARF, facing an increased battle to be relevant, gets a high-profile, major initiative “off the shelf.” It seems to be a win-win situation for both sides.

CRE logoLet’s Get CREative

But let’s be adventurous and go for a trifecta. There are also the assets of the Council for Research Excellence (CRE) sitting out there, in the wake of its defunding by Nielsen at the end of 2017. These would be a nice complement to the CIMM’s body of work. In my own viewpoint, I tended to think of the CRE as dealing more with the micro issues of audience measurement while CIMM took much broader, macro brushstrokes. At the least, the CRE’s work deserves an archival home if (when?) the plug is finally pulled on the CRE website.

In any case, congratulations to the ARF and CIMM on their new marriage. Let’s hope this blended family adds some new audience research to its existing initiatives.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Read his new book, “The Genius Box” – details here
Get notifications of new posts – sign up at right or at bottom of this page.

Label Surveys As Well As Data

It was with great interest I read of the new “data transparency label.” This label is being released for comment by several of the media alphabet associations – the AMA, ARF, and CIMM.

data transparency label example

In the manner of the nutrition labels mandated by the FDA, these labels are hoped to increase clarification about the torrent of data being aimed at big data applications in media, particularly advertising targeting. Adopting a very brief but standard reporting structure, the labels will give users of data a high-level assessment of the quality of the numbers being injected into their algorithmic black boxes. (And by the way, notice there is no equivalent transparency effort about those black boxes; but that’s another story.)

Survey Nutrition Too?

This is important news in that corner of the research, data, and analytics world. What would I like to see? An equivalent nutrition label for publicly released surveys, perhaps sponsored by the Insights Association (the 2017 amalgamation of CASRO and the MRA). The label would provide a required minimal level of information to release with research conducted by its members. This would include items such as:

  • Who paid or sponsored the poll
  • A description of the sample
  • Mode of collection
  • Probability or non-probability sample
  • Dates for fielding
  • Standard error for probability samples, or some “equivalent” for non-probability samples

This information should be enough to quickly evaluate the bias and relative level of quality of a publicly released survey. In fact, some of this information may already be required, but in reality is rarely available in press articles or from the entity releasing the survey.

Too Busy to Process

The Press is too inundated with press releases and too busy filling a 24/7 demand for content to bother to evaluate PR surveys anymore (read MediaPost‘s disclaimer on their Research Intelligencer newsletter). It’s all just grist for the content mill. But maybe with a very simple label, they will be tempted to think once in a while. At the least, we could do the thinking with the right information.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Read his new book, “The Genius Box” – details here
Get notifications of new posts – sign up at right or at bottom of this page.

Dave the Research Grouch: Variety and Cowan

Variety logoLast week, Variety (and multiple others) published a report on a new study from Cowan & Co. on Netflix use, and it’s hard to decide at whom to get grouchy. At Variety, for writing up an article with no context, or Cowan for dropping survey results without publishing any details about their study.

Let’s look at the headline first – “Netflix Is No. 1 Choice for TV Viewing, Beating Broadcast, Cable and YouTube (Study)”. What, according to the article, did the survey results actually say? That people self-reported they used Netflix (27%) “more often” to view than cable TV (20%) or broadcast TV (18%).

Let’s parse this out a bit. First, consider that Nielsen reported in Q1 2017 that 90% of viewing time is still on traditional TV networks. Sure, there are issues with Nielsen but even so it is reasonable to assume that it’s not too far off. This means that in terms of actual viewing time among the total population, Netflix is nowhere near the most-watched platform despite what people may say they “use most often.”

Second is the rather subjective decision to compare broadcast and cable separately against Netflix. It’s been my experience that people with a streaming agenda tend to also be the ones who say viewers can’t tell or don’t care about cable vs broadcast. But that would ruin the headline, because it would change to “Legacy TV Networks Are No. 1 Choice for TV Viewing [38%], Beating Netflix [27%] and YouTube.”

This point is emphasized further when the data for homes with pay TV are shown. Most trusted studies show that a majority of Netflix homes still have pay TV in some form, and here the difference is even more pronounced, with 45% choosing legacy broadcast or cable and 24% Netflix. No attention-getting, disruptive headline from that.

The Frowns are Awarded

Thus a big Research Grouch frown is aimed at Variety (and other sites) for publishing these data without any context at all – context one would hope the beat writers in this area would know enough to include.

Cowan doesn’t escape without a frown either, for my pet peeve – promoting a study without publishing anything about it on their own site. I could not find anything on their website or a press release with which to follow-up. I understand that we don’t need a dissertation, but if you’re going to promote research, then at least have some basic details available to read outside the lens of the press, who (from experience) are notoriously fast-and-loose with their interpretation of research results. What age was the sample? When was it fielded? How was it weighted?

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Quick takes from ARF AudienceXScience conference

ARF LogoLast week, I attended the ARF’s AudienceXScience (AxS) conference in Jersey City. A new incarnation of the former Audience Measurement conference, it’s one of my two favorite conferences of the year. Here are some highlights from the main keynotes, plus an observation about the conference itself.

The Future of TV’s Currency and Measurement
Linda Yaccarino (NBCU), Dave Morgan (Simulmedia)
Sound bite: The opening salvo in the recurring theme of traditional television being a better value versus digital. Traditional television needs to counter the narrative being driven by digital. The TV industry should not be tied to legacy audience measures or back office systems if they are not in step with the changes occurring in the industry.

Nielsen’s View on Media Measurement
Megan Clarken (Nielsen),  Scott McDonald (ARF)
Sound bite: Continued Nielsen’s recent themes of being open to new approaches and new partnerships to deal with business issues – without necessarily committing to either.

The Future of Media: An Epic Battle
Laura Martin (Needham & Company)

Sound bite: Despite advantages in many business KPIs, the FAANGs will not “defeat” legacy television because of the need for storytelling excellence and creating emotional connections. Their only path to success is acquiring a legacy media company and let it do its thing.

An Evolving Arena: Program Currency and Measurement
Lisa Heimann (NBCU), ‎James Petretti (Sony), Will Kreth (EIDR)

Sound bite: Cross-platform is important for program development and promotion just as it is for advertising sales. But a big part of the puzzle continues to be missing due to some streaming partners refusing to cooperate and share data.

An Agency Perspective: The State of Media
Lyle Schwartz (GroupM), Joe Mandese (MediaPost)

Sound bite: GroupM is asking more from the data but extracting less; research has not evolved to match consumer and business changes. Need to get to a seamless “device-agnostic” measurement, but to do so, marketers will have to reformulate their device-centric planning and viewpoints.

Reach & Frequency Balance: How to Get it Right in the World of Advanced Television
David F. Poltrack (CBS), Radha Subramanyam (CBS)
Sound bite: Ad buyers have always traded off between reach and frequency, but recent years have moved from a focus on reach to a focus on efficiency. This leads to less reach and more frequency, and undervaluing of television. Excess exposures have no value, so advertisers should be converting those exposures to new exposures via TV.

How to Use 6’s – ARF 2018 Primary Research
Paul Donato (ARF), Dan Schiffman (TVision)

Sound bite: Linear TV 6 second ads seem to work at present, but have had some advantages such as being placed in premium primetime content, in short pods, or in favorable pod positions. There are a lot of areas still to explore as these 6s become more prevalent.

Consumers, Cross-Platform & Trust
Bryan Wiener (comScore), Jason Lynch (Adweek)

Sound bite: Wiener sees the audience measurement space as open for innovation, but he did not seem to want to be held back by legacy expectations of quality or accreditation before releasing new products.

Oxford Style Debate: Has Marketing Taken Targeting Too Far?
Arguing for: Gian Fulgoni (former comScore), Radha Subramanyam, (CBS)

Arguing against: Dave Morgan (Simulmedia), Yin Woon Rani (Campbell Soup)
Sound bite: A new format for the conference, and it went pretty well – although it seemed too long since many of the same arguments kept being made. Those arguing “For” focused on the bad experience retargeting gives consumers, and that narrowing the base of consumers exposed to your brand can have a negative long-term impact. Those arguing “Against” said not to condemn a whole concept because of bad implementation; in the long-term, targeting is the right way to go as problems get corrected. As for the result of the debate, a fairly even split in the audience before the debate was converted into a clear win for the “Against” team after the debate.

How Far Can You Go? (before government steps in, that is)
Tim Wu (Columbia Law School)

Sound bite: The European model of GDPR with regards to data privacy is unlikely to gain traction in the USA. Instead, we may see more stringent fiduciary duty defined for those who collect or use consumer data.

Thoughts on the conference itself

The ARF AxS (or AM) conference series has been fighting a battle to maintain relevance over the past few years. Perhaps most notably, its usual dates are now used by both the TV of Tomorrow conference in San Francisco and the VideoNuze Online Ad Summit, siphoning off potential speakers from important digital media firms. Even among the core group of past AM participants (media companies built around TV networks), it seems attendance is down. Media consolidation doesn’t help, and neither do changes in budget or leadership priorities, but…

I often joke about seeing the “same cast of characters” at ARF conferences over the 20+ years I’ve been going to them, but perhaps it’s not so funny. I am a firm believer in the ARF and its mission, but I’m sure the ARF can appear to younger people as a crotchety old uncle trying to enforce dedication to methodological rigor and quality established by the generation of researchers who came up to lead the industry in the 1960s and 1970s. Not that rigor and quality are bad things, but I’m sure it sounds preachy at times, especially to younger industry members who are in companies founded on disruption.

The ARF has made strides in recent years in outreach to younger generations of researchers (Young Pros, for example). Under new leadership, it has refocused its overall efforts in the past year to be more relevant. And I’m sure the ARF tries to get its digital members to participate in AxS.

I Love You ARF, But…

But judging by AxS, the ARF still has a way to go to reflect the diversity in our industry – not only by demographics but also in types of firms that show up for its conferences. If it continues to be mostly older white guys talking about television-centric topics (I’m guilty on all counts, by the way), AxS will continue to lose ground to competitive conferences in the near term — and the ARF its relevance to the next generation of advertising researchers in the long term.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Dave the Research Grouch: Recruiting isn’t Representativeness

Bloomberg View logoThe Research Grouch got a double whammy Monday morning. First, having seen the headline “We Are Finally Getting Better at Polls” on, I clicked right to the piece. Once there, however, I discovered it was a self-promoting opinion piece by a UK polling company. 

Already grouchy from feeling somewhat misled, the content of the piece ratcheted up the grouch level. The author discussed what he considered to be new, innovative ways to reach “representative” samples for polling. These new ways included using IM instead of email to reach out to respondents (OK, that makes sense); showing respondents how their responses sit against others taking the poll (debatable, especially if it’s raw instantaneous data); giving people surveys on topics they like in order to maintain interest (again, debatable); emotional testing rather than direct questions (not sure how this solves a sample issue); recruiting from non-political or non-news websites, or from social media (diversification of online recruiting does not create representativeness).

About the only indisputable point made in the piece is politically active people need to be recruited in their correct proportions to get the best data. If this is considered news to political pollsters in the UK, then no wonder they had issues predicting recent elections.

I may be a grouch but I’m not against innovation. These suggestions are certainly ways to potentially increase respondent engagement and diversify online sample sources. But, despite claims from online research firms everywhere, a volunteer opt-in online sample is by definition not representative regardless of panel size, recruitment techniques, weighting, or other manipulations.

Such samples may often give the same answers as a truly representative sample, but that doesn’t make them representative. Neither does the use of the terminology, such as response rates and error margins, lifted from traditional probability-based research and which really don’t apply to volunteer samples of any kind. A serious problem for this industry is that there is a whole generation of researchers that don’t realize this.

There really are probability-based online panels

I do recognize it’s a different world today than 20 years ago. Opt-in online samples are generally “good enough” for many applications, and I’ve used them myself many times. But, at least in the USA, there are sources of online access panels recruited using traditional probability techniques (including GfK’s KnowledgePanel, NORC’s AmeriSpeak, and RAND’s American Life panel) which are available for important research, whether political polls or key business decisions.

Yes, these panels are expensive compared to opt-in sample, but you get what you pay for – and as many pollsters, candidates, and businesses have found over the years, the most expensive research is bad research.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Reflections on the MIE Media State of the Union session

family watching TVIn the Industry State of the Union session at the 2018 Media Insights and Engagement Conference, panelists were asked to share with the audience one takeaway from the current circumstances of the industry.

Click on this link to see my summaries of the takeaways along with some commentary, via one of my guest blog posts for the conference – Reflections on The Media Insights State of the Union

Other MIE conference posts

Please note these posts were originally to be posted during the MIE conference itself but are being published on delay by the conference organizers. Other of my guest blog posts from this year’s conference include…

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.

Ad Experiments Sing for NBC

Click on this link to see one of my guest blog posts from this year’s Media Insights & Entertainment conference – Ad Experiments in Musical Sing for NBC.

Please note these posts are delayed due to issues with the conference website; they were originally to be posted during the conference itself. Look for several more posts from the 2018 conference over the next few days – updates via my twitter feed or via the conference twitter.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Get notifications of new posts – sign up at right or at bottom of this page.