Pros and Cons of Online Survey Research

by Janet Salmons, PhD, Research Community Manager for SAGE Methodspace

Research design is the focus for the first quarter of 2023. Find the unfolding series here. Want to know more about survey research in particular? See these recent posts.

---

It seems so easy. Just put your questions into an online survey template on SurveyMonkey or Qualtrics and away you go! Or maybe there is more to it. In the research case “Lessons in quantitative and qualitative data collection using online surveys” (2022) Shane, Amada, and Niwa summarized advantages and disadvantages. If you are thinking about designing a study with an online survey component, you might benefit from their experience.

Lessons learned about using online surveys

Advantages

• Speed and ease of collecting data

• No limit to the number of responses

• Available to complete anywhere a respondent can access the internet

• Automatic transcription

Disadvantages

• Potential variability due to the device that participants use to complete the survey since it is usually easier to type longer answers on computer or laptop keyboards than on cellphones. “Answering close-ended (e.g., yes/no) questions that typify our quantitative-heavy surveys on a mobile device is simple, but answering open-ended prompts that typify our qualitative-heavy surveys on a mobile device is much more difficult” (p. 9).

• Possible distractions given that surveys can be completed anywhere, “for example, participants could have been on a subway, eating dinner with their family, watching a movie, or splitting their attention in ways that may have altered their responses” (p. 3).

Design factors to consider

• Modify survey length. Shane et al observe that participants often get tired of answering questions in long surveys, which might be exacerbated by distractions or multitasking online. “Any effort to reduce the length of a survey will often produce better data because participants will be more likely to focus on the questions” (p. 5).

• Design questions so response options can fit on the screen. Instead of long lists of responses embed them in drop-down menus.

• Re-think open-ended questions. For open-ended questions meant to generate qualitative data, Shane articulated smaller, simpler, more specific questions to increase the likelihood that participants would thoroughly answer each one. They increased the size of the answer text box to encourage respondents to type longer answers.

• Avoid repetitive questions. Look for alternatives to listing several rows of with the same responses (e.g., “strongly agree” to “strongly disagree”) so respondents don’t simply skim questions and click the same answers.

Shane, J., Reigada, L. C., Amada, N. M., & Niwa, E. Y., (2022). Lessons in quantitative and qualitative data collection using online surveys. In SAGE Research Methods: Doing Research Online. SAGE Publications, Ltd., https://dx.doi.org/10.4135/9781529601244

---

More Mehodspace posts about survey research