The Grouch was actually happy last week. The Pew Research Center announced it was moving away from telephone-based research to an online research panel recruited using a traditional, representative probability-based sample.
Pew is home to the Pew Internet Project and multiple other political and social research centers. It has long done research to a standard that The Grouch would tell anyone to emulate. But its one drawback was reliance on RDD telephone samples (even if gussied up with cell phone supplements).
There is another aspect of this move that makes The Grouch happy. It is another example that exonerates his belief in representative probability-based online research panels. This is because the Pew panel was developed using the same concepts and team as KnowledgePanel, the probability-based panel used by The Grouch for 15 years during his time with Knowledge Networks and GfK.
Now part of Ipsos after its acquisition of much of GfK, KnowledgePanel is almost unique in the world as the only large-scale implementation of an access panel of its type. Pew is not the first client to have used KN/GfK to recruit and maintain a proprietary panel using similar methods to KnowledgePanel (names you would know but I can’t share).
What’s the Big Deal?
The distinctive aspect of the recruitment of these panels, compared with opt-in internet panels, is people can’t volunteer to join the panel. An address-based sample from the US Postal Service is used to recruit the panel. Basically, you are eligible to be selected in a recruitment batch if you have a valid mailing address. And to enable a cross-section of all US homes, offline homes are given a netbook and internet access.
In this way, a true random selection can be made and response rates can be calculated, unlike with opt-in samples. This is because it is known exactly how many have been asked and how many cooperated. It was – and still may be – the only online research panel accepted for peer-reviewed academic research.
I won’t dive much more into this whole topic. But there are clearly applications where a truly representative panel is a superior choice. These would include trying to nail down high-quality estimates for a population or for making important business decisions. There are certainly uses for opt-in samples as well. These would be where the level of data quality needed may not justify the added research expense that results from the costs of recruiting and maintaining a probability-based research panel.
The Grouch Emerges
To get grouchy at least once in this post, too many experienced researchers today have no idea that a random sample doesn’t just mean a random pick from any sample source. The sample has to originate from a probability-based panel to be truly representative in the classical research sense. They also don’t realize that more sample doesn’t mean better data. Or that just because an opt-in survey’s demos equal Census distributions makes it truly representative.
The use of an expensive recruited panel is never an easy sell in these days of procurement departments driving down costs and where awareness of traditional measures of quality are quickly disappearing from the research gene pool. It is encouraging to see Pew step up and make the investment in quality sample. This should result in furthering their tradition of quality research.
David Tice is the principal of TiceVision LLC, a media research consultancy.
– Don’t miss future posts by signing up for email notifications here .
– Read my new book about TV, “The Genius Box”. Details here .