It seems that a new research methodology comes to market a few times a year. And every year,
researchers claim that the new methodology the only methodology that we will ever need. That is
simply not true. Marketing problems are so variable, and respondents differ so greatly, there cannot
possibly be a one-size-fits all methodology.
Choose your marketing research methodology based on the specific requirements of each survey
project, the available sampling frame, the timeline, and the budget. Additionally, to choose the optimal
methodology for your project, we propose evaluating each method on four categories of factors:
-
Task Factors are first. These are the specific requirements of your project that different
methodologies can meet to varying degrees. Think about the diversity of questions the
methodology can accommodate. Not all questions can be asked effectively in all settings, and
not all tools are flexible enough to ask all types of questions. Some methodologies are better or
worse for different questions (think grid questions in mobile surveys). Consider whether you
need respondents to evaluate stimuli and what form those stimuli might take. It is very difficult
to present graphic, text, or video stimuli to phone interview respondents, which would be a
significant drawback to choosing that methodology. The quantity of data needed is also a
concern (especially when combined with budgetary limitations). Sometimes, conducting as
many in-home interviews as you need for a given project is not feasible. Finally, response rate, a
concern in all methodologies, should be evaluated against your intended analysis.
-
Situational Factors are second. Does the chosen methodology give you enough control over the
data collection environment to accommodate your project? Doing an in-home interview with a
busy parent with three kids running around may not be optimal for her to focus on the task.
Additionally, do you have sufficient control of your field data-collection force? It is critical that
each survey is administered consistently to control bias. You should also consider the potential
for interviewers to inject their own bias into data collection. Finally, which methodology allows
you to bring the project in on time and on budget? Social media surveys are notoriously fast and
inexpensive, but are these the right respondents and question formats for your project?
-
Respondent Factors are third. Do the respondents feel comfortable with the methodology? Do
the respondents believe that their data will remain private and anonymous, or do they perceive
a threat of their identity being compromised? Is the methodology likely to elicit socially
desirable responses? That is, will respondents give the responses they believe to be more
socially or politically acceptable than their “true” answer? How comfortable will the
respondents be in giving honest responses to questions seeking personal or sensitive
information? Low-incidence populations are another consideration. Suppose you are trying to
sample unique or rare respondents or a small geographic area. In that case, you may be limited
in the methodology you can choose by the available sampling frame. Finally, consider how much
control for respondents is appropriate for your survey. You might like to give respondents a very
long data-collection period to accommodate their busy schedule, but your boss wanted the
answers yesterday.
-
Finally, the Environmental Factors. If the global pandemic has taught us anything, it’s that
everything can change. What’s going on in the world in general? Will that impact your research
or potential respondents in any way? If so, which methodology should we choose to meet the
research objectives in some other way? For example, consider economic upheaval, such as we
have seen with recent inflation and the potential for a recession. Then, on a micro-level, think
about the seasonality of your business and holidays. Trying to survey during the summer is more
demanding than other times since summer break, travel comes into play, and response rates are
lower. The same thing happened with holidays. Respondents have other things on their minds
than taking the time to complete a survey. You still need to make the business decisions, so not
getting the information is not an option. Consider what questions you are asking, whether the
environment might skew or bias respondents’ answers, and find ways to adjust for the
difference.
There is no perfect all-around methodology for every project. You must balance the good and the bad to
optimize your survey research. There may be times when you may need to shore up your
methodological choice to overcome its weaknesses for your project. Mobile optimization, incentives,
reminders, and slider scales are all tools that researchers use to optimize respondent participation. The
important thing is to understand how your methodology works, how it might impact your project overall
and in data collection in particular, and how those impacts can affect your analysis and reporting.
Contact Symmetric to talk about overcoming these research challenges! CLICK HERE.