Video: Avoiding Bad Survey Design

“Bad questionnaire design is really the bane of my professional existence. It is immensely frustrating. It is something we see, unfortunately, a lot ...” So starts Julia Clark, senior vice president for Ipsos Public Affairs, a for-profit research agency, as she discusses bad survey design. While the flaws she identifies in this SAGE Research Methods video – viewable for free by clicking on the image below -- aren’t seen so often in top research companies’ work (like that of Ipsos) they are definitely apparent in places where he enthusiasm is abundant and the oversight lacking.Her first example is biased question design, which not only creates bad data, but then may be pushed into the public sphere as reliable survey results. Plus, she adds, bias can exist apart just ham-handed questions: “We can't just think about the question itself when we think about a biased question design. We have to think about the context.”After setting the scene, Clark then workshops a questionnaire with three other practitioners, who show the technical and ethical issues – and putative solutions – that real-life survey professionals deal with. For example, do you call possible respondents, or do you offer an online interface? The questions and context differ, and so do answers.“People simply answer questions differently,” Clark explains. “Again, this goes back to the psych principles, to some of the sort of fundamental psychology of not just questionnaire design but of how humans function. You read something on a paper, and you answer it one way. And you talk to someone on the phone, and you're maybe answering a little bit differently for a huge host of reasons.SAGE Video offerings are close captioned but also include a searchable transcript that’s clickable to that point in the video, so when O’Leary shows a chart of various reasons for a lit review, you can go directly there in the video by clicking on the text. They also include citation information and the ability to make your own clips from the longer video. You can share via e-mail or social media, save to a playlist, and speed up of slow down play. To view this particular video, for free, click here.For more about SAGE Video Research Methods Video, click here.