How much is a box? The hidden cost of adding an open-ended probe to an online survey

Primary idea: adding non-mandatory open-ended questions to surveys increases perceived response burden and decreases response rates.

Excerpts

Attached to a closed-ended question in the form of an open-ended comment box, probes can solicit additional input on a respondent’s understanding of a question, their reasons for selecting an answer category and aspects not covered by the closed-ended question. Among others, Singer and Couper (2017) argue that, as long as probes are non-mandatory, they should have little adverse impact on survey response. This paper challenges this view and argues that, from the viewpoint of respondents, open-ended probes are open-ended questions and hence increase perceived response burden (Crawford, Couper & Lamias, 2001). This should in turn lead to more satisficing and higher non-response (Krosnick, 1991; Krosnick, Narayan & Smith, 1996). Unlike the majority of the literature that studies responses to probing questions themselves (see e.g. Behr et al., 2012), the present paper therefore focuses on how a probe affects survey completion and responses to a closed-ended question.

— Page 34

Reference

Malte Luebker “How much is a box? The hidden cost of adding an open-ended probe to an online survey” (2021)

@Article{luebker2021,
  title = {How much is a box? The hidden cost of adding an open-ended probe to an online survey},
  author = {Luebker, Malte},
  journal = {Methods, data, analyses: a journal for quantitative methods and survey methodology (mda)},
  volume = {15},
  number = {1},
  pages = {7--42},
  year = {2021},
  publisher = {DEU}
}