All Sample Sources Are Not Created Equal: How to Select the Right Sample Source

August 20, 2019

Panel and river samples have become very popular, and we are now seeing growth in panel aggregators. None of these sample sources are inherently good or bad, but it is important to understand their differences – and how to choose the best sample source for your project.

What's The Difference

  • Panel Sample.Panels are databases of contactable survey respondents who have completed a rigorous registration process and have agreed to participate in survey research. A panel can be a general population or a specialty panels consisting of one or more types of lower-incidence populations, such as physicians, IT decision-makers, or people employed in the construction industry. Using a specialty panel can make recruiting respondents more efficient, although they are more expensive than general-population panels.
  • River Sample.Also known as “Dynamically Sourced Sample”. River Sample respondents come into contact with a survey opportunity through digital marketing. The sample comes to you in a constant flow, which is why it is called river sample. Respondents are generally routed randomly to a survey opportunity.
  • Aggregated Sample.Aggregators may or may not have a proprietary sample source, but they create a unique sample solution based on your study requirements by working with multiple sample sources to combine qualified respondents. Because aggregation can minimize some of the biases inherent in single-source panels, this method is becoming more popular with researchers. Additionally, aggregate sample can be a solution to very low-incidence-population surveys.

The Good and The Bad

Panel Sample River Sample Aggregated Sample

PROS

  • Recruited through multiple vehicles, including social media, advertising, and emails
  • Extensive demographic profiles
  • Double opt-in for secure respondent identification
  • Able to recontact respondents (for some panels)
  • Actively managed and maintained for accuracy
  • Potential to integrate with other data streams (such as geo-coding, sales data, etc.)
  • Predictable response rate
  • Usually faster than panel samples
  • Usually less expensive than panel samples
  • Can include all the pros of panel samples.

CONS

  • “Professional respondents” may develop behaviors to gain access to more surveys, reducing dataquality
  • Panel attrition must be actively managed by the panel vendor to ensure the panel is viable
  • No ability to recontact respondents
  • Less-secure respondent identification without double opt-in
  • Not able to integrate with other data streams to produce enhanced insights
  • No demographic profile
  • Requires router to match potential respondent to survey opportunities.
  • Not actively managed
  • Dependent on internet traffic at the moment
  • Difficult to predict feasibility and response rates
  • May have duplicate respondents if the aggregator does not have appropriate sample-cleaning procedures
  • Only as good as the panels being aggregated

Choosing the Right Sample Source: Key Questions

The key to choosing the right sample source is to understand the needs of your project. Are you looking for a very unique sample or respondents who typically do not participate in surveys? Or is your qualified respondent fairly ubiquitous in the population? Think about what is most important to your project: speed, replicability, a nationally representative sample, or cost? Here are key questions to ask your sample provider to make sure you’re getting the right sample for your project:

  1. How do you recruit and incentivize respondents?
  2. What registration process do you use to ensure respondent-identification accuracy and security?
  3. What variables are available on potential respondents?
  4. Will I be able to recontact my respondents should that become necessary in future research?
  5. What quality checks do you implement during fielding or after data collection to ensure high-quality data?
  6. How do you de-dupe respondents in the sample and then later post-data collection?

Understanding how your sample is sourced is the key to getting the right sample, and having the right sample is the key to data quality. Don’t assume that all samples are the same – they aren’t! And those differences can have a huge impact on the success – or failure – of your marketing research project.

Contact us to talk about your sampling needs: info@symmetricsampling.com