Warning: A non-numeric value encountered in /home/customer/www/ticevision.com/public_html/wp-content/themes/Builder-Cohen/lib/builder-core/lib/layout-engine/modules/class-layout-module.php on line 505

Archive for Research business

Scenes from the ARF 2019 AudiencexScience Conference – Day 2

AxS logoDay 2 of The 2019 AudienceXScience conference from the Advertising Research Foundation (ARF) was held April 16th in Jersey City. This annual fixture in the media research industry calendar – a rebrand of the ARF’s 2006-2018 Audience Measurement conference – again brought together many luminaries to shed light on the current state of media measurement.

ARF logo

Detailed Notes for Day 2
See Day 1 Notes here

Below are notes from each of the panels/presentations I attended. These notes are by necessity distilled down based on how quickly I could take notes, so they do not reflect the totality of the presentations or discussions. I apologize in advance to any presenters who feel short-changed, misinterpreted, or misquoted.

Marketing Effectiveness in the Digital Era – Les Binet, adam&eveDDB

Presented results from a long-term analysis of UK data, which has head to four books. Fundamental principle: Brand (building) vs activation (sales). Activation can be high efficiency and high ROI – but while it creates sales blips, it does not build growth. Brand building creates long-term memories – broader reach, different attention, more memorable activities; its decay is slower which leads to long-term build.

How to maximize effectiveness

  • Penetration is always the main driver of growth; reach is king
  • Maximize mental availability; brand awareness/salience/fame
  • Messages vs emotions; rational is used for activation, emotional for brand effects

Invest in share-of-voice; if share-of-voice is greater than share-of-market, then growth. An optimal budget should be 60% brand, 40% activation. The fundamental rules haven’t changed with the emergence of digital. Digital increases efficiency and makes activation easier, but brand building is still more important in the long run.

The Race to Own the Future of TV – Julie DeTraglia, Hulu; Natasha Hritzuk, WarnerMedia; Ali Rana, Snapchat

— AT&T’s sale of Hulu?
JD: no change in near term at Hulu
NH: WarnerMedia are treating their upcoming DTC service as a CPG product, not a tech product. Doing UX research for consumer features. Content discovery and personalization are key attributes.
— Snapchat issues?
AR: they have Discover for storytelling from respected partners. Both scripted and unscripted shows. Content for mobile is very difference from regular TV.
— Future of appointment TV?
NH: Appointment TV now is when people get together to watch, not a set time based on broadcast schedule
JD: viewers want their content on every screen. Most of Hulu viewing is a connected TV in the living room
— Ads?
JD: Hulu wants to offer choice and flexibility like they do for viewing their programs. The ad load is less than regular TV; viewers can choose ads or have interactivity. All this leads to more effective ad environment. Their choice to place ads in “pause” screens was another space they could use without interrupting viewing (since viewing was already paused)

— Measurement?
NH: are we putting the cart before the horse by focusing on developing current measures, when they are working on ad experiences that bypass traditional ads; shouldn’t the measurement match the new experiences, rather than trying to fit new experiences into the old measures?
JD: Measurement needs to be part of the ecosystem. She has to do attribution with different vendors depending on measure needed. Hulu does some attribution now directly so to bypass.
AR: In 5 years, all advertising will be “performance” advertisers (eg, only pays on results, not exposures or impressions)

— Importance of diversity
NH: It’s a given. May need to offer multiple service options to serve all consumers.
JD: Same. Their research covers all types of persons.
AR: Snap has diverse user base and staff.

Seeking a Framework for Measurement – Radha Subramanyam, CBS

Media measurement has historically been about counting, in the future it will need to add outcomes as well. The current state of attribution research is that there are no consistent outcome measures or standards; and the impact of linear TV is underestimated.

The state of counting is it is too complex and still siloed. She wants simplification: a total audience count across all devices that gives total program and commercial audiences.

Philosophy for the future

  • Data comes in all sizes
  • Consumer analytics need to be aligned (survey and passive measures)
  • There is an art and a science to interpret meaning of data – the art focus on storytelling

And apparently, if she’s on your team and she yells at you, it’s a sign she cares.

Exploring the Multiple Dimensions of Attention – MediaScience & Google

What is attention? Desk analysis of existing literature revealed there is attention (in a continuum from Passive to Active) and inattention. There is much academic research on attention but little on inattention.

Attention is the absence of inattention, and inattention can be accurately measured. In the lab, blink duration and eye fixations per second had the highest accuracy in measuring attention/inattention

Within attention the best measures may be dependent on the content viewed, or the intended outcome of the stimulus.

Next steps are 1) a pilot to see if measures of attention translate to ads and 2) confirm the best measures for ad attention.

The Future of Audience-Based Buying  – Comscore

This session was really just a review of OpenAP without any new insights. It was also somewhat ironic as WarnerMedia (Turner), one of the founders of OpenAP, announced three days later it was dropping out of the OpenAP system.

OpenAP is helping network sales teams and their buying partners utilize new datasets. These can be used for planning, buying, posting, and auditing.

Demand for OpenAP has been “limited” but expanding. Despite the free access, the presenters quoted there are about 1,000 individual users signed up.

Consistent segment definitions can be used across network groups with secure segment sharing. It also allows independent 3rd party posting.

From Proxy-Based Optimization to People-Based Optimization – Survata

The problem of proxies. Today optimization is typically against viewability, CPMs, and reach but not against outcomes (such as brand lift).

To enable auto-optimization, need to move from campaign level to persons-level reporting (the latter being modeled). Also need single KPI to optimize against (such as funnel impact).

Can’t use traditional survey research, need “programmatic scale”
Can’t use tradition panel accuracy, need superior data accuracy
Don’t use look-a-like respondents, need causal AI

Cross-Platform Insights Every Influencer Will Cite This Year – Nielsen

This was pretty much a recitation of relevant results from the latest Total Audience Report from Nielsen.

There has been a 182% year-over-year increase in connected TV (CTV) impressions
There is currently about 10 billion(!) hours per month viewing time of CTV in the USA, translating to about 75 hours/month of CTV time among CTV users.
CTV adds about a 16% increment to a P18-49 audience.

Erwin Ephron Demystification Award

Congrats to Leslie Wood!

Brand Purpose and Cinema – NCM, ScreenVision, MESH

Many brand experiences are perceived as neutral, whereas consumers and brands both want “purpose”. This study used Real-Time Experience Tracking (RET), a one week brand experience diary.

Paid brand touchpoints are seen as less engaging and persuasive than owned or earned touchpoints. But paid can be a first step to drive people to the better-received owned/earned experiences.

Cinema cuts through neutrality [as one would expect from an NCM/ScreenVision presentation]. Two thirds of cinema brand experiences were positive, more than any other touchpoint, and was particularly helpful among 18-24 demo. TV & cinema together work even better.

A Levi’s case study was presented. Cinema exposures were 2x more engaged than TV alone; 93% found cinema memorable compared with 71% of TV.

Can Data Privacy Be Good for Brands? – Dan Linton, W2O Group

The risk of harm is real. Examples are physical (such as when FitBit jogging data revealed secret military/CIA bases) and emotional (such as when a woman miscarried but still was followed by baby advertising online).

The California Consumer Privacy Act will have a large impact, and is being followed by similar laws in WA, VT, OR, CT, IL, and TX. GDPR is already impacting the EU.

But GDPR did not kill off digital advertising in the EU. In fact, privacy ethnics are not detrimental but can be a positive differentiator for a brand/ad tech service. There are many positives that can result. These include:

  • Getting ahead of the curve in terms of what data are collected and how – and if any will fall foul of new laws
  • Becoming aware of, and organizing, data streams. Where are they from? Why do we use them? Are they really needed? Where are they stored? Is there PII to worry about?
  • Being transparent will build trust
  • Give consumers a reason to engage and share their data

Presenting the ARF Code of Conduct – Paul Donato, CRO of the ARF

Donato discussed the recently announced ARF Code of Conduct. What makes it different?

  • A focus on research not activation-type data
  • A commitment requires research of terms and agreement
  • A chain of trust between elements of the research process
  • Includes automated, location, and AI-driven research
  • There are monitoring KPIs; the ARF can see how many times the terms have been read and agreed to
  • There is a required annual compliance report

Companies can apply online and it is voluntary. It was made voluntary to avoid company lawyers resisting a more structured commitment.

[Donato completely sidestepped the whole issue of compliance. The code is a nice idea but it has no teeth – there is no active enforcement by the ARF and it’s dependent on someone being a whistle-blower. And the penalty of having the ARF seal rescinded may likely have no effect other than temporary embarrassment]

Too Much Math, Too Little Meaning – Rishad Tobaccowala, Publicis

We are in the 3rd connected age (1st = initial computer/browser based; 2nd = computer + smartphone; 3rd = internet of things, all is connected)


  • Erosion of trust
  • Close-mindedness – we need to do “A/B testing” in our own beliefs, ie consider other viewpoints
  • Rising inequality
  • These are all the dark side of the first two connected ages

Data isn’t missing about what to do to solve many of these issues, it’s the will to implement solutions

Purpose – what are we doing all this for?
Poetry – where is the art/beauty in what is being done?
People – you need to change people or keep them and change their mindset

See Day 1 Notes here

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Scenes from the ARF 2019 AudiencexScience Conference – Day 1

AxS logoDay 1 of The 2019 AudienceXScience conference from the Advertising Research Foundation (ARF) was held April 15th in Jersey City. This annual fixture in the media research industry calendar – a rebrand of the ARF’s 2006-2018 Audience Measurement conference – again brought together many luminaries to shed light on the current state of media measurement.

ARF logoA surprisingly large 550 registrations were announced for AudienceXScience, indicating that the conference is in good health. However, part of that may be due to the ARF killing off its long-running annual conference this year (called re:think for many years before being rebranded ConsumerXScience in 2018).

Among the recurrent themes this year are:

  • Attribution and its issues continues to be the hot topic in measurement
  • One to segment may be a better targeting approach than one-to-one, especially given future developments re privacy
  • Data quality and the need for “ground truth panels” continues to make a comeback

Detailed Notes for Day 1
See Day 2 Notes here

Below are notes from each of the panels/presentations I attended. These notes are by necessity distilled down based on how quickly I could take notes, so they do not reflect the totality of the presentations or discussions. I apologize in advance to any presenters who feel short-changed, misinterpreted, or misquoted.

Opening Remarks – Scott McDonald, ARF President

McDonald feels there had been an improvement in measuring video in the past year, at least in coverage. But there are still blind spots and there are still uncooperative sellers who won’t open their walled gardens.  Advertisers need to pressure Amazon and others to open their systems to measurement. But there is no consensus yet on a cross-platform video measurement that takes into account both TV and digital. McDonald repeatedly called out “parochial concerns” as roadblocks – companies wanting to keep their data walled up to gain a competitive advantage.

Advertising in a Modern Media Company – Rick Welday, Xandr Media (A&T)

Welday spend some time on the advertising structure within AT&T: WarnerMedia with premium advertising opportunities, AT&T with ability to serve addressable ads across multiple channels, and Xandr being AT&T’s adtech solution. Key trends include 1) addressability scaling; 2) addressable is becoming easier to buy; 3) addressable expanding into other areas; 4) advertisers are committing to always-on budgets enabling digital optimization.

Frequency capping continues to be an issue. Example showed 70% of impressions were served to 28% of targets. However, using Xandr increases efficiency and allows advertisers to reach the “gold” light TV viewer. But Xandr right now only works with the 2 min of local avail time given to MVPDs.

The future includes improvement of frequency delivery and also sequencing ads. Local avails converge with national ads. Format innovations via AR, MR, and 5G. Very bullish on 5G and on its potential ability to bridge rural & digital divides.

Transforming Measurement – Megan Clarken, Nielsen

Overall media use has increased from 50 hours/week in 2003 to 75 hours/week in 2018 – an increase of 50%. Targeted advertising has increased from 2017 to 2019 from $2.4B to $6.8B for linear TV ads, and from $47B to $73B for digital TV ads.

Is there a problem with “measurement”? No, measurement is being done (by Nielsen, of course). There are issues with the overall system

  • alignment on comparability
  • everything should be measured and available, like for TV – all see all
  • how to avoid fraud
  • improvements in the ecosystem to support this goal

Many people are unaware of what Nielsen can do with de-duping audiences and with measurement within walled gardens.

Planning in an AI World – Brad Smallwood, Facebook

82% of display ads are bought using automated systems. Agencies, advertisers, and platforms need to think differently – “liquidity” and “signals”.

Liquidity allows each $1 to be spent on the next most valuable impression. An automated system selects the most valuable impression and creative, and serves it to the right person in real time.

Signals are behavioral data that machine learning uses to make predictions. They drive improvement in ROI for advertisers.

Automated systems like these are only as good as the data passed into them. And do the signals align with the end goal of a campaign? E.g., advertising ROI and optimization are two different things.

He feels that the implication for Nielsen and measurement is how can Nielsen make marketing better? It should be a marketing improvement company, not a counting company. It should add value rather than being a cost center.

Counting the Right Viewers in OTT Measurement – Nielsen

We should be measuring people not devices for both linear and digital.

  • Connected TV audiences are different from both linear TV and digital audiences
  • Should be measured at the persons level
  • This will assist dynamic ad insertion (DAI)

More Than Impressions: OTT in the TV Daypart Model – Roku & TVision

How does attention (measured by eyes-on-screen) and OTT translate into TV’s traditional daypart model? OTT has similar co-viewing levels as linear TV but attention to commercials is 50% higher for OTT. Why?

  • Intentional viewing
  • Can’t skip ads
  • Captive audience – channel surfing is much more difficult than in the past

These OTT advantages persist across the total day. Final points: 1) OTT is TV – mostly same viewing habits; 2) OTT has higher attention; 3) OTT breaks the linear daypart model.

Quantifying and Aligning Emotion – Magid & Warner Bros Entertainment

This paper discussed efforts by WB to help their affiliates align the local news promos shown in syndicated Warner Bros programs with the content in those programs, allowing greater synergy in brand image and increasing audience flow into local news.

For Ellen, 99% of affiliates use it to lead into local news; high levels also for Warner Bros programs Dr Phil and Judge Judy. Particularly for the feel-good Ellen, the typical “if it bleeds it leads” style of news promotion can cause cognitive dissonance and actually decrease intent to view the news.

A series of surveys and focus groups, the former making use of Magid’s Emotional DNA metric, showed that the more tonal the news promo is to the program, the better the tune-in rate. A key point is to use a positive spin in the promo, even if it’s a serious story. An example would be “Suspects identified and being pursued by police” rather than “Killers on the run!”.

The findings are being shared with news directors at the the affiliates.

In or Out? – WarnerMedia

Advanced TV includes data-driven linear TV. Audience Now is WM’s (nee Turner’s) own targeting system. Has been proven to drive outcomes – example showed 1.6x ROAS target among campaign using Audience Now vs not using it.

Uses three components: 1) Spot level measurement via EDO; 2) Nielsen Catalina data; 3) Kantor surveys.

Audio and Video at the Intersections of Digital Video and Linear TV – Omnicom & Tunity

This paper discussed out-of-home (OOH) measurement. There is a gap for OOH measures where audio cannot be heard. This is addressed by the Tunity app, which apparently streams the audio of muted programs to a user through their smartphone. The Tunity data was analyzed to look at OOH viewing behaviors.

Key takeaways:

  • Tunity app did indeed capture OOH viewing
  • A substantial amount of use of the app was “in home” as well as OOH
  • Location of viewing was a substantial influence on viewing behavior
  • Need to think about how OOH viewing can contribute to the TV audience
  • Consider including OOH into cross-platform measures

How a Truth Set Can Power Data Accuracy Verification – Ericsson Emodo

Emodo is the digital advertising arm of Ericsson. There is so much focus on media quality but so little on how we decide to buy. Segments, build requests metadata, attribution studies all dependent on data.

Raw data can be 46% inaccurate, even filtered data can be 34% inaccurate. Emodo can use Ericsson’s cell-tower-level data from all mobile service providers to validate GPS location data (their data not dependent on device, OS, carrier).

When questioned further, the presenter had difficulty articulating why Emodo’s data are a truth set: “It’s hard to explain;” “Scale and completeness”.

Takeaways: 1) Carve out data quality from media quality; 2) seek proof of data quality not just indicators; 3) recognize the key role that “truth sets” should play in scaling data

Calibrating Bias in Online Samples for High Quality Surveys at Scale – MRI/Simmons

This presentation made some very on-point points, mainly reminding people that online panels and surveys are not representative in the same way traditional probability sample are. This is a key point that from experience I know that people ignore, forget, or are not even aware of.

Sample bias tends to be narrow; in other words, most of a survey using a non-probability sample can be perfectly fine but then a few points are not representative of the real world. Analysis of data using Simmons’ National Consumer Sample showed some deviations in topic areas such as:

  • Online shopping
  • Communications
  • Video streaming
  • Use of tech
  • Numerous psychographic attributes

Use of demo weighting does not address these differences, only moderates them a little. Bottom line is do not ask questions about online uses or attitudes to a non-probability online sample.

[personal note: this argument was made for years by Knowledge Networks in support of its probability-based panel called KnowledgePanel (now part of Ipsos). Unfortunately, these arguments typically fell on deaf ears; researchers acknowledged the numerous papers put out by KN on the topic, but getting them to actually spend the extra money for KnowledgePanel sample was a much more difficult task. I wish MRI/Simmons better success than we had!]

A Segments Journey – clypd, Acxiom, MRI/Simmons

This presentation discussed taking segments from MRI to other environments. The issue: audience consistency. Offline and digital measures represent identities and attitudes differently.

They followed five segments from MRI to the Nielsen-MRI fusion, and also MRI to Acxiom to DMPs, publishers, etc.

For the segments, they evaluated the segment sizes and how well the profiles compared (using 47 variables). As for the Nielsen-MRI fusion, there was good matching. With the digital fusion, the matching was (as expected) less good. Issues included ID fuzziness, loss of scale, drop off, and impact.

Correlations for digital segments were in the range of 0.62 to 0.71 compared with the Nielsen-MRI segments which were 0.89 to 0.97. But due to the inherent differences in the datasets, it should not be expected that digital segments match the correlation of the two probability-based datasets.

Standards, Research and Rationale – George Ivie, Media Ratings Council

Need to move from gross impressions to targeted characteristics. Need to increase the quality of the digital side of measurement to that of TV. The standard is based on consistency for video exposures. Provides stronger content focus for digital, stronger ad focus for TV.

There are rules for granularity and comparability, durations and completions, practices for appending audience characteristics. Because of its establishment in current agency systems, the 30 second base is being used.

Is it for planning or currency? Both, but mainly as a currency. Planning tools, which are not the basis of sales, don’t require same rigor. Duration weighted video impressions (DWVI) is getting almost all the debate and comment, despite taking up only 4 of the 70 pages of the draft document.

Going Beyond :30s, :15s or :06s – Vas Bakopoulos, Mobile Marketing Association

This was the first study to pass the new ARF Certification Program and dealt with attention and cognitive load. Mobile ads do more in one second than we think. Attention is almost always similar and cognition follows closely.

Focus on creative in the first second. Ads that fail, fail in the first second. For longer exposures, are you overpaying for unneeded exposure if key effects are almost immediate?

Advance Toward Digital Audience Quality – Robin Opie, Oracle

Poor audience quality results from several factors:

  • Bad actors
  • Weak ID graphs
  • Over-extension of data
  • Quality of source data
  • Bad modeling

Oracle employs a number of different processes to combat bad quality, including:

  • Audience health
  • Model diagnostics
  • Ecosystem diagnostics
  • Real-world validation
  • ID graph accuracy

Grow Your Brand With Better Audience Targeting – Nishat Mehta, IRI

Top tips for targeting:

  • Quality @ scale (what is the highest quality at the highest scale?)
  • Recency of data
  • Future proofing (getting ahead of regulations – is data collected now in a way that will be legal in the future? Example – he feels traceable tender will not survive in the future)

Should a big brand be microtargeting? Does that defeat the purpose of building a big-umbrella brand? Plus he feels microtargeting is too creepy.

Paving the Way for News Organizations – Lisa Ryan Howard, NY Times

[note: This might have been the worst-presented session of the entire conference, with Ms Howard spending most of her time standing in one place, hand on hip, looking down to read the teleprompter… not the type of dynamic presenter needed at 5PM in the afternoon.]

This presentation basically reviewed the NY Times’ advertising assets, and how they have adjusted to the current digital era. A brand needs to matter… and consumers need to know what matters. The NYT has expanded into audio with podcasts, and into TV with an upcoming series on the FX network.

The NYT ReaderScope application gives advertisers insights into what topics are being read by their targets, and insights into contextual advertising.

CampaignScope is an advertising tool that profiles content and what each impression was exposed to/read. They are currently still mostly audience buys, but want to move more advertising to contextual, which they feel is more advantageous both in terms of effectiveness and the reader experience.

See Day 2 Notes here

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Is There An Elon Musk For Media Measurement?

Nielsen ratings boxNews in recent weeks called out the troublesome business situation in the media measurement space. Both Nielsen (which is rumored to be finding it difficult to find a buyer) and Comscore (which forced out its CEO and president after less than a year) highlight the difficulties even the key companies in this space are experiencing, quite apart from the difficulty of measuring today’s media use.

[The following post is adopted from the recently published book “The Genius Box: How the “Idiot Box” Got Smart & Is Changing the Television Business”. “The Genius Box” is available in paperback or digital format from Amazon, Barnes & Noble, Apple iBooks, and most major online booksellers. A short term discount is available at the BookBaby store, thru April 17th. Go to se code ARF2019PRINT for paperback, ARF2019EBOOK for ebooks.]

In most industries, the seller delivers a discrete product or service to the buyer – but in TV and media, buyers and sellers transact their business based on market research results (audience estimates, also called “ratings”). Because the audience measures account for billions of dollars in spending, media research has traditionally been subject to high levels of scrutiny, an important consideration to keep in mind when considering the future of audience measurement.

Disruption Isn’t As Easy As Some Might Think

It would seem that, in today’s world, a business such as audience measurement of electronic media – led by a near-monopolist for half a century – would be a ripe target for disruption and new entrants. But it is not that easy. There are numerous “structural” issues that stand in the way of progress, separate from developing a holistic, cross-platform solution.

These obstacles include:

  • Nielsen exploiting its monopoly power in terms of revenue and agreements, and generally implementing improvements only when faced with potential competitors
  • On the TV network side, a reluctance to fund two parallel measurements – most past models of Nielsen competitor roll-outs assume that the new entrant would have to run parallel with Nielsen for at least some period
  • TV network sales people preferring to sell a “Nielsen” currency because of the prestige of the name itself
  • Getting agencies to buy into an audience measurement system developed or led by TV networks, since the assumption is that a method led by the sellers will disadvantage the buyers.

Despite its protestations to the contrary, Nielsen wields the power of a monopoly – one that US courts said was OK, even before Nielsen gobbled up one of it only potential competitors, Arbitron, in 2013. Being the sole arbiter of the national television currency for decades, and of local television since 1993, Nielsen has been a perennial lightning rod for critics, with some good reason. It is expensive and seemingly slow to innovate unless it perceives a competitive threat.

In Defense of Nielsen

The ratings giant does have a difficult mission – trying to keep up with the constant change in media while still maintaining the strict quality its clients demand (or at least the previous generation of research heads used to demand). Media researchers have been bashing Nielsen for the three decades I have been in the industry, but no one yet has been willing to fully fund an alternative. For many in the industry, to paraphrase Churchill’s comment about democracy: Nielsen is seemingly the worst form of audience measure, except for all the others.

Despite calls for disruptive entrants, what I perceive from many in the industry is resignation to Nielsen’s dominance. As with the Borg from Star Trek: The Next Generation, “resistance is futile,” given that Nielsen has faced down about a dozen potential competitors as well as an antitrust suit over the past 50 years.

Who Could Step Up?

Only the most deep-pocketed, risk-tolerant firms would even be tempted to enter this space as the barriers to entry for a new currency-quality measure are now so high.  Alphabet, Amazon, and Facebook all have the money and would

likely have a great deal of interest in the viewer data stream; but their positioning as competitors in this space – both between themselves and with regular television – would almost certainly prevent any one of them from creating a widely accepted advanced measurement.

Perhaps someone could interest Elon Musk once he gets a man on Mars – that might be the easier task!

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Dave the Research Grouch: Pew Goes Online

Pew Research logoThe Grouch was actually happy last week. The Pew Research Center announced it was moving away from telephone-based research to an online research panel recruited using a traditional, representative probability-based sample.

Pew is home to the Pew Internet Project and multiple other political and social research centers. It has long done research to a standard that The Grouch would tell anyone to emulate. But its one drawback was reliance on RDD telephone samples (even if gussied up with cell phone supplements).

There is another aspect of this move that makes The Grouch happy. It is another example that exonerates his belief in representative probability-based online research panels. This is because the Pew panel was developed using the same concepts and team as KnowledgePanel, the probability-based panel used by The Grouch for 15 years during his time with Knowledge Networks and GfK.

Now part of Ipsos after its acquisition of much of GfK, KnowledgePanel is almost unique in the world as the only large-scale implementation of an access panel of its type. Pew is not the first client to have used KN/GfK to recruit and maintain a proprietary panel using similar methods to KnowledgePanel (names you would know but I can’t share).

What’s the Big Deal?

The distinctive aspect of the recruitment of these panels, compared with opt-in internet panels, is people can’t volunteer to join the panel. An address-based sample from the US Postal Service is used to recruit the panel. Basically, you are eligible to be selected in a recruitment batch if you have a valid mailing address. And to enable a cross-section of all US homes, offline homes are given a netbook and internet access.

In this way, a true random selection can be made and response rates can be calculated, unlike with opt-in samples. This is because it is known exactly how many have been asked and how many cooperated. It was – and still may be – the only online research panel accepted for peer-reviewed academic research.

I won’t dive much more into this whole topic. But there are clearly applications where a truly representative panel is a superior choice. These would include trying to nail down high-quality estimates for a population or for making important business decisions.  There are certainly uses for opt-in samples as well. These would be where the level of data quality needed may not justify the added research expense that results from the costs of recruiting and maintaining a probability-based research panel.

The Grouch Emerges

To get grouchy at least once in this post, too many experienced researchers today have no idea that a random sample doesn’t just mean a random pick from any sample source. The sample has to originate from a probability-based panel to be truly representative in the classical research sense. They also don’t realize that more sample doesn’t mean better data. Or that just because an opt-in survey’s demos equal Census distributions makes it truly representative.

The use of an expensive recruited panel is never an easy sell in these days of procurement departments driving down costs and where awareness of traditional measures of quality are quickly disappearing from the research gene pool. It is encouraging to see Pew step up and make the investment in quality sample. This should result in furthering their tradition of quality research.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Entering the Gen Z Zone

As guest-blogger for the 2019 Media Insights & Engagement Conference (staged by knect365), I am putting some of the overarching themes I heard at the conference in perspective. I discuss about what was said at the conference about Gen Z, the rising group of young adults, in my second post-conference piece.

“A number of presentations at the 2019 Media Insights & Engagement Conference talked about the newest generation for us to worry about: Gen Z. Presentations or keynotes touching on Gen Z were given by Viacom, Freeform, ABC, TiVo, BBC America, and Zebra Intelligence/Ipsy…”

Read the rest of the post at the knect365 website here.

MIE Conference logo
The MIE conference was held in Los Angeles between January 29-31. Details about the conference can be found here.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, The Genius Box. Details here . 

Advice for Future Researchers

As a guest-blogger for the 2019 edition of the Media Insights & Engagement Conference (which is put on by knect365), I have put some of the themes I heard at the conference in perspective. In this first post, I discuss about what was said at the conference about what future – or up-and-coming – researchers should know.

“Up-and-coming or future researchers were on the minds of several presentations at the 2019 Media Insights & Engagement Conference, which took place January 29-31 in Los Angeles. These included a panel of high-level research execs, a session from Viacom, and a tech perspective. And, at least two of the “Off the Record Industry Conversations” discussed future researchers, or researchers now vs. then.

There seemed to be three main themes that I took away from these sessions…”

Read the rest of the post at the knect365 website here.

MIE Conference logo
Between January 29-31, the MIE conference was held in Los Angeles. Details about the conference can be found here.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, The Genius Box. Details here . 

Scenes from the 2019 CIMM Summit

CIMM logoThe eighth annual CIMM Cross-Platform Video Measurement & Data Summit was held on February 7th at the Time Warner Center in New York. As always, this annual fixture in the media research industry provided an interesting discussion about the state of media measurement.

Among the recurrent themes were:

  • C-3 and C-7 measures, meant to be temporary, are now 12 years old and do not seem to be going anywhere – despite not reflecting today’s viewers
  • Greater transparency is still needed at all levels
  • The need for “ground truth panels” seems to be making a comeback
  • Attribution continues to be the hot topic in measurement

In something of a change from previous editions, no-one from Nielsen or Comscore (or any start-up measurement service for that matter) presented or was part of a panel.

The hand-outs, press releases, and deck from the summit are available on the CIMM website, as are materials from earlier summits.

This was the first CIMM Summit since CIMM was acquired by the ARF back in October. I hope that CIMM and the ARF will continue to offer this summit, and to keep it free so that all those with an interest are able to attend.

Detailed Notes

Below are notes from each of the panels/presentations. These are by necessity distilled down based on how quickly I could take notes, so they do not reflect the totality of the discussions.

After a short kick-off by CIMM CEO and Managing Director Jane Clarke, the first session featured an interview of Krishan Bhatia of NBCUniversal.

  • C-3 and C-7 are outdated by today’s viewing habits
  • C-Flight introduction by NBCU came with little pushback. There is some friction around the work but not about the concept
  • They are working on attribution, campaign measurement, and how to prove performance across all NBCU media
  • He is skeptical that there will ever again be a one-size-fits-all solution
  • 34% of NBCU consumption is now on digital – expect it to be up to 50% very soon

The next session was a panel featuring Rob Master of Unilever, David Cohen of MAGNA, and Laura Nathanson of Disney to discuss business needs for cross-measurement and metrics.

  • RM: There is no common solution. Industry needs to develop a common vernacular to discuss. Can’t be perfect – what is now? near? next?
  • LN: Disney adjusted by moving all media sales under one group. The “plumbing” is an issue – need to plumb and test
  • DC: C-3 and C-7 are no longer sufficient. Need to move to exact commercial minute measurement. In the mid-/long-term, need to look at audible and visual measures across all platforms.
  • RM: Unilever doesn’t care so much about addressability – they have broad markets
  • LN: But then Unilever should use addressability to send different creative to various segments within a broad demo
  • One key thought to close:
    • RM: Transparency and dialog around counting
    • DC: Let’s “start by starting” – need to get moving
    • LN: Just because it’s hard doesn’t mean we shouldn’t do it – it’s the reason we should do it

Next, an overview of this year’s update of the CIMM TV attribution whitepaper was presented by Jim Spaeth and Alice Sylvester of Sequent Partners. Attribution then discussed by Claudio Marcus of Freewheel and Lisa Giacosa of Spark Foundry.

  • What is the state of the art of attribution?
    • LG: I’m excited and hungry [for more]
    • CM: Like in the UK train stations, “Watch the Gap”. There are gaps in cross-platform attribution, and brand/longer-term effects
  • CM: Biggest effect so far on automotive. Auto had moved money from TV to digital – but attribution showed TV drove the digital exposures. Moving back to TV. Media & Entertainment another area – TV program promotion
  • LG: Need to understand content effects. Can’t just follow short-term ROI over a cliff.
  • JS: Need to use baseline sales as a basis for calculating incremental effects of attribution media

Following a break, there were brief updates of the Taxi Complete (AD-ID and EIDR) and Data Label initiatives.

Another panel discussed Deduplicating Reach for Content and Ads, featuring Radha Subramanyam of CBS, Eric Cavanaugh of Publicis, Beth Rockwood of Turner, and Ed Gaffney of GroupM and moderated by Scott McDonald of the ARF.

  • EC: A good quality attribution should be getting deduplication as a byproduct
  • BR: how things fit together is a big issue
  • RS: need both counting and outcome measures. But we need to up-level the conversation: There are lots of products and data, but are we any closer to making sense of media and marketing together? Need a commonsense playbook at a high level.
  • EG: Need dedup in place before this years upfront – or 2020 upfront.
  • RS: Vendors need to listen closely to needs. Their solutions are not necessarily addressing the needs.
  • EC: We also need to know about content to be able to place ads in context.
  • EG: Blindspots are getting smaller but there are new ones popping up every day
  • EC: We are getting one-off fixes to blindspots but need integrated response
  • RS: Integrating projectable and non-projectable samples is doable but needs more investment
  • BR: The technical issues of integration are easier than making the theory work
  • RS: In terms of privacy, one-to-many is less threatening than true one-to-one marketing

Is there One Metric to Rule Them All? Kavita Vazirani of NBCU, Brian Hughes of MAGNA, George Ivie of the MRC, and Sheryl Feldinger of Google discussed this topic.

  • BH: Need exact minute commercial ratings
  • SF: Need equitable (with TV) transparency at exposure and second-by-second ratings
  • KV: Need to measure effort vs return. Shouldn’t we be focusing on cross-platform measures rather than arguing about TV measures?
  • BH: already does second-by-second with MediaOcean, which is an old platform – so it can be done today
  • GI: MRC is working on standard definitions with partners and industry, aiming for impression-based duration-weighted data by 2021. Measures to include exposure, viewability, duration-weighting, complete exposure to an ad.
  • SF: Wants absolute exposure. His work shows that a 5 or 10 second exposure elicits a similar response, regardless of the total length of an ad
  • KV: Disagrees. She claims the only time a 6 second ad worked was as part of a larger integrated campaign
  • GI: There is a big gap in content measurement in digital. For content measurement in a cross-platform world, customer journey analysis is something that should be syndicated (eg, third party)
  • All: agree audio status needs to be known (muted vs non-muted)

The last panel talked about Audience-Based Buying Platforms for TV/Video. This panel included Bryson Gordon of Viacom, Mike Law of Dentsu Aegis, Bob Ivins of NCC Media, and Mike Welch of Xandr.

  • BI: Inertia is real. Need to get marketers to “cross the bridge” and not turn back halfway across. We need standards and transparency.
  • MW: Can help reach low incidence/low viewing HHs
  • BI: Need an automated platform like Google and Facebook. Still too much manual transfers between different applications
  • BG: users on OpenAP have already created 1,872 segments
  • Opportunities in 2019
    • BI: More inventory and optimization
    • ML: Platform, optimization, interactivity
    • BG: Automated workflows, cross-platform delivery, unified posting
    • MW: Platform, true cross-platform delivery

To wrap up the afternoon, Jack Smith of GroupM told us about what he saw at the 2019 CES conference.

  • The three areas to pay most attention to are Assistants (Alexa, etc); Autonomy (self-driving cars); and Simulation (VR/AR).
  • It is important to understand how algorithms work – what products are suggested when Alexa is asked to buy something. Should brands have an avatar to speak for themselves, rather than relying on Amazon etc.
  • Most everything will still be on screens. How are these to be measured?
  • Top takeaways: 1) Interface revolution. 2) Immersion environments. 3) The ethics of tech in general.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

2019 MIE Conference Summaries

MIE Conference logoAs guest-blogger for the 2019 edition of the Media Insights & Engagement Conference (which is put on by knect365), I wrote up summaries of the keynotes and the break-out sessions I attended. You can find the daily summaries on the knect365 website:

Day 1 of the 2019 MIE conference: Day 1 (Jan 29 2019)
Day 2 of the 2019 MIE conference: Day 2 (Jan 30 2019)
Day 3 of the 2019 MIE conference: Day 3 (Jan 31 2019)

Also, read my three pre-conference posts here:

2019’s New SVOD Services: Blitzkrieg or War of Attrition?

Connected TVs: Corporate Connections as Important as Internet Connections

Does AVOD News Reveal a New Phase of SVOD?


David Tice is the principal of TiceVision LLC, a media research consultancy.
Don’t miss future posts by signing up for email notifications here .  
– Read my new book about TV, “The Genius Box”. Details here . 

Most Popular Posts of 2018

2018 is coming to a close and it’s time to take a look back. Which TiceVision blog posts have had the most interest in the past year?

Third Place

In a virtual tie for third place are two posts:

3a. Quick Takes from the ARF AudienceXScience Conference – as the name implies, in this June post I share some of my thoughts on the 2018 edition of this long-running conference, the good (as always, some interesting sessions) and the bad (its lack of diversity in companies and presenters).

3b. Drake vs The Beatles: Let it Be – A post from July, I take issue with press comparisons that claim Drake outdoes The Beatles. These comparisons don’t take into account differences in how the Hot 100 is calculated now vs the 1960s.

Second Place

2. In second place for the year is Dave the Research Grouch: Another Data Fluff Piece. This post, one of the generally popular “Dave the Research Grouch” series, takes exception to press coverage of a data release by Inscape, Vizio’s division which monetizes their TV set viewing data.

First Place

My most popular post of the year, by a margin of almost 2-to-1 over the runners-up, is Foreverspin Tops? More Like Forever Annoying Ads. This post has the longest legs of my 2018 posts, with at least a reader or two every week since being published last February. In the post, I take issue with the bad side of digital advertising, exemplified by the Foreverspin Tops ads that followed me for years.

Happy Holidays!

Whether you observe Christmas, Hanukkah, Kwanzaa, or another winter holiday, I hope all my readers have – or have had – an enjoyable holiday season. And best wishes for your happiness and success in 2019!

  • Don’t miss any of my 2019 posts by signing up for email notifications here
  • Haven’t read my new book about TV, The Genius Box? It’s available in paperback and e-book formats. Book details and ordering info here

David Tice is the principal of TiceVision LLC, a media research consultancy.

ARF-CIMM is good news, but let’s get CREative

The ARF logoThere was interesting news in the audience measurement business yesterday. Several outlets covered the announcement that the Advertising Research Foundation (ARF) will acquire the Coalition for Innovative Media Measurement (CIMM). As a couple of articles noted, this is a continuation of the trend in consolidation in many sectors of the media business.

CIMM logoI’ve been on the sharp end of trying to sell syndicated research studies to a decreasing pool of clients because of consolidation. I can imagine that CIMM was dealing with a similar issues among its membership in the wake of the Disney-Fox, Discovery-Scripps, and other recent deals. The ARF, facing an increased battle to be relevant, gets a high-profile, major initiative “off the shelf.” It seems to be a win-win situation for both sides.

CRE logoLet’s Get CREative

But let’s be adventurous and go for a trifecta. There are also the assets of the Council for Research Excellence (CRE) sitting out there, in the wake of its defunding by Nielsen at the end of 2017. These would be a nice complement to the CIMM’s body of work. In my own viewpoint, I tended to think of the CRE as dealing more with the micro issues of audience measurement while CIMM took much broader, macro brushstrokes. At the least, the CRE’s work deserves an archival home if (when?) the plug is finally pulled on the CRE website.

In any case, congratulations to the ARF and CIMM on their new marriage. Let’s hope this blended family adds some new audience research to its existing initiatives.

David Tice is the principal of TiceVision LLC, a media research consultancy.
Read his new book, “The Genius Box” – details here
Get notifications of new posts – sign up at right or at bottom of this page.