Sample & Respondent Quality
Wakefield Research Uses the Highest Standards for Data Quality
Respondents from elite, high-quality panels are subject to verification protocols to confirm their identity and qualifications. They must complete a double opt-in process before they will be contacted to participate in any research. It is important to note that all Wakefield respondents are screened and re-screened multiple times during onboarding and while they are members of a panel.
Panelists matching the scoped criteria of a particular survey will be contacted using an email invitation. Once they agree to participate in the survey, each respondent must answer additional screening and classification questions during the sampling process and after entering the survey link. In other words, each respondent must answer questions on the relevant audience characteristics, such as revenue, company size, industry and type of business to qualify for the survey.
The sampling process will be informed by the scoped quota expectations for markets, most notably the revenue brackets and expected counts in each country. This way, Wakefield ensures an accurate distribution of the survey data.
In addition to identifying and interviewing the right respondents, Wakefield also allocates a significant amount of resources to check the quality of the collected data. Key elements of our data-checking procedures focus on the investigation of errors and deviations from survey protocols. Wakefield is utilizing a multi-factor-multidimensional approach to check the data for all our research studies.
During fieldwork of Wakefield Research surveys, our agency will review the technical consistency and the quality of the data using metadata as well as the survey data for the following elements:
Length
Wakefield is using traditional and statistical approaches to identify interviews that are too short and therefore indicate a problematic respondent.
A traditional approach is to set a minimum expected interview length, which will be determined once a final instrument has been developed. The minimum required length implies that all interviews shorter than X minutes will be automatically removed from the sample for quality concerns.
In addition to this less flexible approach, we also use a statistical test to identify outliers based on measures of central tendency. This is an ex-post test and helps identify interviews that are outside of the length variation, based on the information of all collected interviews. Interviews that are flagged as being too short will be removed and replaced.
Location Markers
When deploying the survey, Wakefield also collects metadata from the devices interviewees are using to complete the survey. Most notably, Wakefield collects information about browser language and IP address. This data will be matched against the data provided by the respondents regarding the market they are placed in.
These checks will be performed at multiple stages during the data collection process. Interviews for which the browser language or IP address does not align with the information provided by the respondent (language used, country of work) will be removed and replaced with new respondents.
Problematic Respondent Behavior
Wakefield also uses checks that are automatically performed during the data collection, which flag and remove cases that are showing primacy effects, such as always selecting the first answer on the screen, as well as straight-lining and flatlining of respondents, which are respondents who are selecting only one or only specific punches throughout the survey and show low cognitive efforts when answering the survey.
These tests are performed automatically during fielding and manually while reviewing the data on the case level.
Additionally, the data will be checked for consistency on the case level to identify respondents who are providing inconsistent information across the multiple topics explored in the survey instrument.
Consistency with Existing Data
Wakefield will, if available, check the trendlines of the data during the fielding process against existing data from previous years. While some deviation from trend results can always be expected, trends should not suddenly reverse without a reasonable explanation.
Trend data can therefore be a powerful instrument to identify weaknesses in the fresh data, which can also result in adjustments of the sampling process.
These checks are performed on the aggregated data throughout the fielding window, which will allow Wakefield to react quickly and adjust factors control, namely sampling and quality control procedures.
If you have any questions about Wakefield’s industry-leading practices for maintaining the highest sample and respondent quality, please fill out the form below or give us a call.
Let's Work Together
Tell us about your project. A senior member of our team will reply promptly. Every proposal includes guaranteed audience specifications, budget and timing.
Prefer to talk? Call (888) 527-9253.
Media inquiries: media@wakefieldresearch.com