Survey mode effects

Survey mode effects are systematic differences in how people answer the same question depending on how the survey is administered, by phone, online, mail, in person, or automated voice (IVR). The same questionnaire design can produce different answer patterns purely because of the mode, even when the wording is held constant.

Common sources include social desirability bias, stronger when a live interviewer is present, phone or in person, than in self-administered modes, web or mail, making respondents more likely to give a socially acceptable answer to sensitive questions; differences between visual modes, where respondents can re-read all options such as a likert scale, and aural modes, where they hold options in memory as an interviewer reads them; and satisficing, where respondents in longer or lower-engagement modes put in less effort and default to easy answers.

Mixed-mode surveys, combining two or more modes to improve coverage and response rate, are increasingly common but introduce their own complication: any difference between groups surveyed by different modes is now tangled up with genuine differences in opinion, which survey weighting alone cannot fully untangle, since weighting adjusts for demographic imbalance, not for mode-driven answer shifts. This differs from nonresponse bias, which is about systematic differences between who responds and who does not; mode effects instead concern how the same person's answer can shift depending on how they were asked.