One of the riskiest moves in a consumer Marketing Research project is to start with a new sample
provider. How do you know they will deliver the number of completes they say they will? How do you
know that their sample will be representative of your population? Will the new source blend in with
your existing sample partners? And what about data quality?
These days, there seems to be a challenge in vendors’ “say/do rates,” which can leave you holding the
bag. Here are five ways to build a sample safety net for your project:
Spread the risk. Don’t give all of your sample requests to one new partner. Always split the
sample with a known provider to have a source to fall back on. That way, you can see how the
new partner performs without risking the success of your entire project.
Vet your partners continually. Don’t assume that the panel partner who has always performed
well for you will continue to do so. Panels change over time: they lose and gain members and try
new recruiting methods. Lately, many sample companies are being acquired by other companies
resulting in their panels getting integrated into other panels. That will all affect sample
performance. Continuously monitor response rates, data quality, rate of deletions, and other
performance metrics to make sure you should continue working with that partner.
Be safe and reduce quoted sample feasibility. When you receive feasibility numbers from your
sample partner, especially a new one, play it on the safe side and reduce their number by 10%-
20% just in case they overestimated, or your IR is not as high as you think it is. If they are not
fully feasible with the reduced estimate, you know that you will need to have a backup provider
ready to go to meet your dates before the project launches.
Plan even further ahead. In addition to planning for a backup sample source when launching a
project, get ahead of the curve and do your due diligence early. As mentioned above, panel
providers are changing, so you should continually look for and vet potential sample sources –
even when you don’t have a specific project starting. Ask for a ‘feasibility test’ to see how well
the new panel meets your needs. That way, you can estimate for future projects. Or start a new
panel vendor on a smaller project to get a realistic picture of panel performance that you can
It’s not always about price. Data quality should always be more important than price, and the
old saying “You get what you pay for” holds on sample purchases. A $2 CPI sample where you
have to throw away 20% or more of your completed surveys means you will be in data collection
longer and spending more time cleaning data. Selecting a new panel provider is also about
service, professionalism, knowledge, and responsiveness. A small ‘test project’ will give you
some insight here.
At Symmetric, not only do we have our own proprietary panels, but we have been vetting sample
partners for over a decade. We understand that each project is different, and there is a science to
blending samples, executing a project properly, and completing it on time.
While you may not work with as many panels or samples as we do, you should still be sure to protect
your projects from poor quality or insufficient samples. Don’t wait until you have already launched the
project to find out you need a different sample partner!
Contact Symmetric to talk about vetting panel partners! You’ll be glad you did!