Avoid Scams, Imposters, and Fraud in Online Research Participation
By Janet Salmons, Ph.D. Manager, Sage Research Methods Community
Dr. Salmons is the author of Doing Qualitative Research Online, and Gather Your Data Online. Use the code COMMUNIT24 for 25% off through December 31, 2024 if you purchase research books from Sage.
Find real participants for real research online
Surely you have seen the posts on social media inviting you to participate in a study. Perhaps not surprising, this kind of approach can lead to problematic situations. In the online world people often interact under pseudonym, and may act in ways inconsistent with their real-life ethics or truthfulness. Particularly when an incentive is involved, people may come forward to simply collect the reward.
How can the researcher locate credible research participants? How cab you verify the identity and age (or other relevant criteria) of research participants recruited online? Rather than simply posting your recruitment message online and hope for the best, here are some strategies that can help:
Recruit from a known organization, membership group or society. Reach out to an organization or entity that has verified members. In essence, use an established sampling frame.
Ask for referrals. In snowball sampling we ask participants to suggest others who might be willing to participate, which can be workable. However, you can take it to the next level by asking respected leaders (employers, online community moderators, thought leaders in their fields) to suggest individuals or to reach out to their own networks.
Create an authentication system. The above strategies might fit qualitative studies with small samples sizes. If you are trying to recruit a larger group of participants, create a system that allows you to verify identifications. Triangulate information from more than one source.
Use a commercial service. Market research or survey technology companies assemble panels of vetted, paid, participants. While you will need to pay for such services, depending on the nature of the study it might be a worthwhile investment.
This collection of open-access articles explores recruitment issues qualitative and quantitative researchers face, and learn from their experiences.
Chandler, J. J., & Paolacci, G. (2017). Lie for a Dime: When Most Prescreening Responses Are Honest but Most Study Participants Are Impostors. Social Psychological and Personality Science, 8(5), 500-508. https://doi.org/10.1177/1948550617698203 and (open access link)
Abstract. The Internet has enabled recruitment of large samples with specific characteristics. However, when researchers rely on participant self-report to determine eligibility, data quality depends on participant honesty. Across four studies on Amazon Mechanical Turk, we show that a substantial number of participants misrepresent theoretically relevant characteristics (e.g., demographics, product ownership) to meet eligibility criteria explicit in the studies, inferred by a previous exclusion from the study or inferred in previous experiences with similar studies. When recruiting rare populations, a large proportion of responses can be impostors. We provide recommendations about how to ensure that ineligible participants are excluded that are applicable to a wide variety of data collection efforts, which rely on self-report.
Fronek, P., & Briggs, L. (2018). Faking participant identity: Vested interests and purposeful interference. Research Ethics, 14(2), 1-5. https://doi.org/10.1177/1747016117740177
Abstract. Misrepresentation and mischief in the research process can impact on ethical conduct, the validity of findings and deliberately change the outcome. This short report presents a scenario about deliberate interference in adoption research by one organisation seeking accreditation to deliver adoption services. Unbeknown to the researchers, fake participants completed an online survey designed to capture the post-adoption needs of adult international adoptees living in Australia. Interference was unexpected as it was naively assumed that all stakeholders involved in adoption would be concerned with meeting post-adoption needs. A definition of politically motivated fake participants, implications for the research process, predicting such interference and strategies to address the problem are discussed.
Levi R, Ridberg R, Akers M, Seligman H. Survey Fraud and the Integrity of Web-Based Survey Research. American Journal of Health Promotion. 2022;36(1):18-20. doi:10.1177/08901171211037531
Abstract. Compared to traditional paper surveys, online surveys offer a convenient, efficient, and socially distant way to conduct human subjects research. The popularity of online research has grown in recent decades. However, without proper precautions, false respondents pose a serious risk to data integrity. In this paper, we describe our research team’s own encounter with survey fraud, steps taken to preserve the integrity of our study, and implications for future public health research.
Li, X., Shi, W., & Zhu, B. (2018). The face of internet recruitment: Evaluating the labor markets of online crowdsourcing platforms in China. Research & Politics, 5(1). https://doi.org/10.1177/2053168018759127
Abstract. Zhubajie/Witmart and other online crowdsourcing platforms have proliferated in China, and researchers have increasingly used them for subject recruitment. One critical question remains, however: what is the generalizability of the findings based on these online samples? In this study, we benchmark the demography of an online sample from Zhubajie to nationally representative samples and replicate commonly asked attitudinal questions in national surveys. We find that online respondents differ from the general population in many respects. Yet, the differences become smaller when comparison is made with the internet users in benchmark surveys. Importantly, when predicting attitudes, our online sample with post-stratification weights is able to produce similar coefficients in most cases as these internet-active subsamples. Our study suggests that online crowdsourcing platforms can be a useful tool for subject recruitment, especially when researchers are interested in making inferences about Chinese netizens. We further analyze the political and social desirability issues of online subjects. Finally, we discuss caveats of using crowdsourcing samples in China.
Littler, C., & Joy, P. (2021). Queer considerations: Exploring the use of social media for research recruitment within LGBTQ communities. Research Ethics, 17(3), 267-274. https://doi.org/10.1177/17470161211003021
Abstract. The use of social media platforms (such as Facebook) for research recruitment has continued to increase, especially during the global COVID-19 pandemic. Social media enables researchers to reach diverse communities that often do not have their voices heard in research. Social media research recruitment, however, can pose risks to both potential participants and the researchers. This topic paper presents ethical considerations related to social media recruitment, and offers an example of harassment and hate speech risks when social media is used for research recruitment. We explore the implications of hate speech risks for ethical research.
Pullen Sansfaçon, A., Gravel, E., & Gelly, M. A. (2024). Dealing With Scam in Online Qualitative Research: Strategies and Ethical Considerations. International Journal of Qualitative Methods, 23. https://doi.org/10.1177/16094069231224610
Abstract. In the wake of COVID-19, numerous research projects moved to online data collection to comply with public health guidelines. Since then, many qualitative projects have continued to use online methods to collect data. While online methods facilitated research continuity, they also introduced new opportunities for deceptive behaviors, particularly misrepresentation and multiple participation. Drawing from a recent project that conducted online interviews with young people who detransition after a gender transition, this article describes how fraudulent interviews were identified and dealt with. We present 12 indicators of potential scams in qualitative interviews, including similarities between participants, the type of information provided, participants’ behaviors, and inconsistencies in the narratives. We discuss our overall experience and, in light of recent literature, present strategies to prevent and deal with scams in qualitative research.
Salinas MR. Are Your Participants Real? Dealing with Fraud in Recruiting Older Adults Online. Western Journal of Nursing Research. 2023;45(1):93-99. doi:10.1177/01939459221098468 (Subscription required)
Abstract. The internet offers exciting opportunities for quick, cost-efficient, and widespread recruitment and data collection without face-to-face contact. Previous research has demonstrated success in reaching population subgroups not typically included in traditional recruitment methods, yet challenges in data quality protection remain paramount. This article describes using Amazon Mechanical Turk, Facebook groups, and email distribution lists to recruit older adults who live alone for a quantitative study using a cross-sectional online survey. Fraudulent survey takers became a major concern in this study, and a protocol was developed to identify and exclude suspicious data. Of 738 recorded participants, 117 responses were retained in the final sample. The majority of sham responses were collected from Facebook with the fewest number of issues identified in responses collected via targeted emailing. Implications for survey design, data analysis, and future research are discussed.
Santinele Martino, A., Perrotta, A., & McGillion, B. J. (2024). Who can you trust these days?: Dealing with imposter participants during online recruitment and data collection. Qualitative Research, 0(0). https://doi.org/10.1177/14687941231224591
Abstract. The use of digital technologies in qualitative research has been found to increase access and participation by minimizing geographical, scheduling, and financial barriers. However, discussions among the qualitative research community about the challenges of conducting research online and, specifically, what steps can be taken to mitigate “imposter participants” remain limited. Anchored in a critical disability studies perspective, in this field note paper, we discuss lessons learned in conducting online qualitative research and preserving data trustworthiness.
Sefcik, J. S., Hathaway, Z., & DiMaria-Ghalili, R. A. (2023). When snowball sampling leads to an avalanche of fraudulent participants in qualitative research. International Journal of Older People Nursing, 18(6), e12572. https://doi.org/https://doi.org/10.1111/opn.12572
Abstract. The scientific literature has examples of how fraudulent respondents who gainfully access studies for financial incentives negatively affect the integrity of research results. However, there have been less examples shared of participant misrepresentation in qualitative research, the implications, and how to prevent deception. This session entails the presentation of a qualitative study where we believe participants misrepresented themselves during an interview for the financial incentive. For the study, we sought caregivers for individuals with a diagnosis of Alzheimer’s disease or Alzheimer’s disease related dementia with a history of chronic leg wounds. One of our recruitment efforts was through ResearchMatch. After one ResearchMatch interviewee completed the interview, he reached out to the project manager and asked if the study flyer could be shared with others that were “dealing with similar issues”. This resulted in an uptick of people being interested in study participation. Several of these participants were scheduled to complete individual interviews on one day with an experienced qualitative researcher. Clues indicating potentially fraudulent participants were vague and illogical responses to questions, and similar nonsensical responses shared across interviews. Once potential fraud was detected, interviews were paused for the following interview day so the study team could regroup and none of the interviews were used for data analysis. The lessons learned and steps for researchers to take to help reduce the number of fraudulent individuals participating in interviews will be shared. This includes two-step screening where the study staff interacts with the potential participants through methods other than over email.
Avoid fraud in survey research with the REAL framework: Reflect, Expect, Analyze, Label
Read this open access article by Jennifer and her co-researchers:
Lawlor, J., Thomas, C., Guhin, A. T., Kenyon, K., Lerner, M. D., & Drahota, A. (2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations, 14(3). https://doi.org/10.1177/20597991211050467
Abstract. Online survey research has significantly increased in popularity in recent years. With its use, researchers have a new set of concerns about data collection and analysis to consider, including the possibility of fraudulent survey submissions. The purpose of this article is to demonstrate to survey researchers an innovative and systematized process for addressing online survey fraud over the course of collecting survey data, especially when respondents collect incentives for participation. We provide the Reflect, Expect, Analyze, Label Framework, which includes four sets of guiding questions for use by online survey researchers to plan for addressing survey fraud and making determinations about the inclusion or exclusion of participant submissions from the dataset based on level of suspicion. We also provide a full case example utilizing the Reflect, Expect, Analyze, Label Framework as an appendix. Those wanting to apply the Reflect, Expect, Analyze, Label Framework should keep in mind several considerations as they apply it, including determining logistical needs ahead of survey implementation, considering the ethical issues related to including or excluding data in a study, and considering the issues related to providing incentives for participating in research. Future research should assess the frequency of survey fraud, investigate the reasons for its occurrence and explore the role social networks may play in fraudulent participants sharing information. We suggest that researchers consider online survey fraud as an issue over the lifespan of their survey and apply the guiding questions we present to address the issue throughout.
Sage Research Methods Community posts about online research
The wealth of material available online is irresistible to social researchers who are trying to understand contemporary experiences, perspectives, and events. The ethical collection and -use of such material is anything but straightforward. Find open-access articles that explore different approaches.