This is an open access article distributed under the terms of the Creative Commons Attribution License (Social media platforms have revolutionized research recruitment in many areas. Social media, as a means of connection and engagement, has the potential to reach a diverse group of participants on a global scale. The number of social media users worldwide surpassed five billion in 2024, with projections to reach at least six billion users by 2028 (Dixon, 2024). With the increasing number of users, social media platforms, such as Facebook, X (formerly Twitter), Instagram, and WeChat have been leveraged to advance research in the health and behavioral sciences, and to reach underrepresented and historically marginalized populations (Miller-Perusse et al., 2019; Sandhu et al., 2023).
In addition to the benefits of making research participation more accessible, social media-based recruitment (SMR) methods can also be a cost-effective or timely approach for scientists, with SMR methods often costing less than traditional recruitment methods and reaching a wider audience (Tsaltskan et al., 2023). A retrospective analysis of studies (n = 105) funded by the National Institutes of Health (NIH) identified that more than half (51.4%) of the studies’ participants were recruited via social media platforms (Nebeker et al., 2020). While it is evident that the utilization of social media as a research recruitment strategy will continue to grow, researchers can face significant challenges when designing and implementing a research study that uses social media for recruitment. Inattention to the pitfalls and risks of using social media can result in methodological issues, such as online surveys being hacked by bad actors (i.e., fraudulent participants) or internet bots contributing to the number of survey responses (Pozzar et al., 2020).
SMR plans are largely held to the same ethical standards as traditional recruitment methods with human subjects but require special consideration when applied to the context of the online environment (Harvard Catalyst Regulatory Foundations, 2017). Institutional Review Boards (IRBs) are the delegated authorities providing oversight, ensuring proposed research studies uphold ethical standards and the rights of research participants. Studies with human subjects must be reviewed and receive approval from IRBs before beginning data collection, but the reporting of ethical parameters in publications varies. A scoping review of 268 articles from 2005–2020 highlighted that ethical issues were generally mentioned in health research reports that used social media, but with limited detail (Bour et al., 2021). Similarities can be seen with the reporting practices of data quality mostly because there is scant documentation in the literature supporting how data quality is achieved or maintained throughout studies using SMR methods. For example, Darmawan et al. (2020) found inconsistencies in reporting social media recruitment and enrollment data during a review of thirty-three clinical trials. The lack of transparency and limited attention to data integrity in publications using SMR methods is a significant gap that points to the importance of educating about the risks associated with SMR and standards for reporting how data integrity was maintained during studies. This gap contributes to a lack of understanding about the issue.
The use of SMR is constantly evolving and can present a challenge outside of the typical ethical and legal requirements that most IRB members are used to (Gelinas et al., 2017). The potential pitfalls of SMR can jeopardize data integrity, especially if researchers are not keenly attentive to discrepancies arising during the recruitment period and upon the initial enrollment of participants. Thus, this paper aims to inform researchers of ways to use social media as a recruitment tool while maintaining data quality.
Method
We conducted a targeted literature search within the CINAHL and Academic Search Complete databases hosted by EBSCO using the following terms: social media, recruitment, and research. We reviewed articles that mentioned using social media as a recruitment tool for primary research, systematic reviews of social media use in research, and articles that described how to mitigate issues with social media-based recruitment (SMR). We also reviewed grey literature published by universities and other research organizations specific to SMR methods. Combined with our experience as researchers who have used SMR methods, we looked for recurring themes throughout the reviewed literature, which became the basis of the hypothetical scenarios we present and, ultimately, the concluding recommendations of this paper.
We present three hypothetical scenarios based on the real-world experiences of researchers using social media-based recruitment (SMR) methods. The scenarios serve as a discussion and learning opportunity for researchers to practice identifying data quality issues with SMR methods and postulate how data quality issues can be mitigated. Each scenario describes the scientific endeavors of Researcher X as they use social media as a recruitment tool for studies with different designs. Discussion questions after each scenario are followed by sample answers in the Supplement (see Bartmess & Abreu, 2024). These answers are not exhaustive; other responses are possible. We encourage readers to answer discussion questions independently or as a group before reviewing the answers provided in the Supplement (Bartmess & Abreu, 2024). For context, all the research activities mentioned in the scenarios received IRB approval before recruitment began. For the purposes of this paper, the term “data quality” refers to the accuracy, integrity, and reliability of data derived from human subjects. Through the scenarios presented below, we aim to educate researchers on the connections between SMR methods and data quality. Data quality issues do not end with the enrollment of participants; however, for the purposes of this paper we focus on data quality as it relates to recruitment efforts with social media and the initial enrollment of participants rather than the ongoing management of data after participants are enrolled.
SMR Scenario One: A Mixed-Methods Study Using Social Media for Recruitment
Researcher X wants to collect quantitative and qualitative data from health professionals across the United States. The study design includes a survey with open and closed-ended questions with text box responses and multiple-choice responses. To obtain a diverse sample, Researcher X plans to share a link to their survey on Facebook and X (formerly Twitter) platforms. No gift card or payment is advertised or associated with completing the survey. Before sharing the survey link online, the survey settings are reviewed. Due to the sensitive nature of the qualitative responses, Researcher X makes the survey responses anonymous, without the ability to see IP addresses; however, their survey settings are still set to let them see which countries the data are coming from. Through the survey settings, Researcher X requires all participants to complete a Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) question before starting the survey and a screening questionnaire. A response is also required for each survey question (including the qualitative questions) before the survey can be submitted for completion. After a few weeks of the survey published online, Researcher X receives 200 clicks on their survey with 50 complete responses. Upon review of the survey responses, Researcher X can discern that the completed responses came from their desired sampling location. By reading through the qualitative responses, Researcher X can determine that the survey participants were knowledgeable of the study topic since they provided specific information that only authentic study participants would know. Researcher X and their co-researchers feel confident that they have quality data.
Discussion Questions for Scenario One
What data quality vulnerabilities are present with Researcher X’s social media recruitment plan?
What data quality protection measures, either survey settings or study methods, are in place?
SMR Scenario Two: A Qualitative Study Using Social Media for Recruitment
After the success of their first study, Researcher X decided to design a qualitative study, interviewing study participants who reside in specific parts of the United States. Researcher X does not live near the population of interest (POI), so interviews take place over Zoom. To recruit participants, Researcher X creates a flyer to be shared on Facebook, X (formerly Twitter), and LinkedIn. Researcher X also offers a 15-dollar (USD) electronic gift card for those who complete an interview. They make a statement on the flyer that participants can keep their cameras off during the interview if they wish. A link on the flyer directs respondents to an online screening questionnaire, where participants self-identify whether they meet inclusion or exclusion criteria via ‘yes/no’ format questions. Researcher X requires that all participants complete a CAPTCHA question before starting the screening questionnaire.
To support participant privacy given the sensitive nature of some interview questions, Researcher X tries to limit the amount of identifying information gleaned from the screening questionnaire associated with participants. The online questionnaire settings are set not to track IP address information, but Researcher X has enabled an online survey feature that supposedly prevents multiple questionnaire attempts by the same person. The online questionnaire includes some inclusion/exclusion screening questions and ends by asking for an email address so Researcher X can contact respondents for an interview.
After just a few days of recruitment, Researcher X completed three interviews and delivered three gift cards. However, Researcher X starts to notice some oddities during the interviews and screening questionnaire process. Participants’ voices sound similar, and there are similar, unique grammatical errors in emails from respondents who are presumably different individuals. Researcher X also notices that some participants seem to be in different time zones when all participants who are screened eligible for the study should be in one time zone. Additionally, none of the participants volunteered to turn their cameras on during the interview, and the content participants discussed during interviews was generic, impersonable, or lacked detail. Researcher X prepares for the fourth interview with a supposedly different participant, but upon seeing the fourth participant enter the Zoom waiting room, they notice that the participant’s Zoom screen name is the same as the third participant’s screen name. Researcher X now believes a person, or several people, are pretending to be participants and completing interviews to receive gift cards. Researcher X does not proceed with the fourth interview. They inform their co-researchers of the events, pause recruitment, and contact their IRB for further assistance.
Researcher X did not have a data quality protocol in place for screening questionnaire responses or potential participant correspondence before starting interviews via Zoom. Because Researcher X did not have a way to objectively screen participants, they decided that they should not use the previous three interviews in their study for two reasons: 1) the information gleaned from past participants could be from bad actors who were not authentic members of the population of interest (POI) and 2) relying purely on subjective criteria from interviews to assess the validity of participants’ lived experiences could result in Researcher X invalidating the experiences of one or more authentic participant. Researcher X decides to start over and pursue a modified study protocol with IRB guidance.
Discussion Questions for Scenario Two
What vulnerabilities were present in Researcher X’s study protocol that were exploited by bad actors?
What could Researcher X have done differently to support data quality?
What did Researcher X do right in this situation to support data quality?
SMR Scenario Three: A Revised Qualitative Study Using Social Media for Recruitment
After several consultations with their co-researchers and the IRB, Researcher X develops a modified study protocol for their qualitative study. The eligibility criteria are listed in detail on the research flyer and social media posts, but gift cards are no longer offered for compensation. Researcher X has a more objective protocol in place to assess whether respondents are eligible for the study before starting the consent and interview process. In addition to the series of ‘yes/no’ format of screening questions, Researcher X includes a page where respondents provide their first name, last name, the U.S. state they work in, and an email address. Since Researcher X’s population of interest (POI) are licensed healthcare professionals, Researcher X can cross reference names with online license databases. Researcher X states on the questionnaire that respondents' names, states, and email addresses will not be shared with anyone outside the study team.
The online screening questionnaire has also been strengthened. Researcher X can see the IP address information for every person who takes the questionnaire. The questionnaire software Researcher X uses also has additional measures they can use to prevent fraudulent responses, such as preventing multiple submissions and scanning respondent computer/location information to flag potential duplicate questionnaire attempts. After receiving IRB approval for the modified study, Researcher X shares their updated recruitment flyer with the improved screening questionnaire on Facebook, X (formerly Twitter), and LinkedIn. Researcher X also uses targeted recruitment methods such as using relevant email listservs, sharing study information with organizations close to the POI, and sharing study information in private Facebook groups (with host permission) that are more likely to have members of the POI. After several months of recruitment and successful interviews, Researcher X reached data saturation for their qualitative study.
Discussion Questions for Scenario Three
What changes in online questionnaire settings helped support data quality?
What changes in study methodology helped support data quality?
What else could be done to strengthen the study’s recruitment plan? Are there any remaining vulnerabilities?
Discussion and Recommendations
As identified in the scenarios, consideration of both study methodology and use of online survey tools are necessary to maintain data quality when using SMR methods. Each research study is unique and will require context-specific considerations to maintain data quality; however, some general considerations and recommendations are presented below as researchers prepare protocols using SMR methods. Please see Table 1 for a summary of potential issues with SMR and mitigating strategies.
Table 1
Potential Issues and Mitigating Strategies
| Issue | Potential Impacts on Data | Mitigating Strategies |
|---|---|---|
| No protocol for data quality screening. | ○ Risk of analyzing and disseminating data that did not originate from authentic populations of interest (POI). | 1. Collaborate with the research team to create a data quality screening protocol with objective and subjective screening criteria. 2. Approach the social media recruitment plan with at least two goals: a) protection and respect of human subjects and b) protection of data quality. |
| ○ Risk of removing authentic participants from data sets based on subjective criteria only. | 3. Enable CAPTCHA questions to differentiate between internet bot responses and humans. 4. Utilize security features provided by the survey software platform, including the capability to view IP addresses and the prevention of repeated submissions. | |
| ○ Risk of including duplicate responses from participants or automated, artificial-intelligence-driven responses. | 5. Consult with information technology professionals to create effective security measures. 6. Consider the use of hidden survey items to deflect bot activity. This is where some survey items are hidden from respondents but are detectable by internet bots. Open-text survey questions can also help researchers identify whether a respondent is a human or a bot by looking at the quality of responses. | |
| Respondents misrepresented identities to secure eligibility in the study. This can occur via: | ○ Collecting data from ineligible participants. | 1. Carefully select messaging for online ads/flyers or postings about the study. For example, providing details of compensation for participating in a study or automating compensation for participation may increase the number of respondents, but this could also encourage bot activity or participation from individuals under false pretenses, especially if other data quality measures are not in place. |
| • Use of VPN (Virtual Private Network) servers and anonymous proxies to hide identities. | 2. Add survey or screening questions that only the POI would know or would be difficult for someone other than the POI to answer. For example, if the POI typically uses a certain tool or process in their work environments, researchers could ask a question about that tool or process. | |
| • Inconsistent responses to open question items or inconsistent responses during correspondence via email, phone, and online meetings. | 3. Delay advertising incentive amounts until a participant is deemed eligible or consider offering and advertising other incentives that may be worthwhile to the POI, such as donating to a certain cause. 4. Encourage video conferencing or phone calls to meet with participants to review consent forms and determine eligibility if in-person encounters are impossible. | |
| Inability to see/track IP (Internet Protocol) address information. | ○ Collecting data from sampling locations inconsistent with the study protocol. | 1. Monitor IP addresses to determine where responses come from and compare this information to the desired sampling locations. The IP address should have a 1:1 ratio (one participant with a unique IP address). 2. Configure survey settings to identify and track IP addresses. 3. Verify time zones or geographical locations with the IP addresses associated with participants. |
Consulting With the IRB
As with all research involving human subjects, using social media methods for recruitment should also account for the protection of human subjects. As social media recruitment gains popularity among researchers, it would be beneficial for universities and research institutions to have IRB guidance documents and workshops. Creating social media recruitment teams to help researchers with this process is also possible (Doshi et al., 2021). In the absence of guiding documents, researchers can consult with their IRB representatives for further guidance on the ethical considerations of SMR. The IRB ensures researchers comply with ethical standards, outline clear procedures, and have adequate measures in the research plan to protect human subjects; creating a protocol that generates high-quality data is the researcher’s responsibility. For example, determining the validity of responses through an established criteria is both a data quality and ethical issue; researchers need to ensure they are obtaining information from knowledgeable POIs for the sake of data quality and IRBs need to assess whether a researcher’s criteria is placing undue burden on participants, such as asking for more information from participants than what is required to achieve study objectives. It is essential to balance the protection of human subjects while also upholding data quality standards.
Develop Effective Screening Questionnaires and Processes
As researchers prepare their study protocol, they can create effective screening questionnaires that differentiate between valid and invalid responses; it is important to define how a participant is deemed eligible for a study with objective and subjective criteria (Wisk et al., 2019). The criteria developed will look different depending on the type of study (qualitative, quantitative, etc.), but consider the time it takes participants to complete surveys, cross referencing participant-reported location information with the presumed location of the population of interest (POI) via internet protocol (IP) addresses, and even some qualitative methods, such as video calls or text-based survey questions, to assess whether the person providing information is an authentic, knowledgeable informant. For example, Miller-Perusse and colleagues (2019) reported several steps to verify participants recruited via social media before enrolling them in their study, such as creating a study-specific website for respondents to navigate, an eligibility screening questionnaire, and requiring all participants to create an account for their study. By having participants create accounts, Miller-Perusse and colleagues (2019) could screen for duplicate account creation, cross reference participants' information (names, emails, IP addresses, etc.), and contact participants if fraudulent activity was suspected. After determining the best questions or processes for screening procedures, it is important to identify survey platforms with security measures in place that can best support the study protocol.
Make Use of Online Survey Security Tools
Most online survey platforms designed for research have security capabilities and can be customized to fit the needs of different studies. Researchers can enable settings that screen for internet bot activity or possible bot-related responses, limit survey access to specific groups, and prevent multiple submissions. Qualtricsxm survey software, for example, includes an option that scans survey users’ IP addresses and operating systems to help detect responses that may be fraudulent (Qualtricsxm, n.d.). Before launching a study, we recommend researchers familiarize themselves with their survey software and consult with information technology experts as needed to understand the potential benefits and risks associated with the survey’s security settings. In addition to online survey security tools, other survey security processes can be incorporated into the study's design. For example, Pozzar et al. (2020) had respondents create a survey access code which they would later use to access the survey through a single-user link sent to their email. Creating unique URLs to questionnaires or surveys for each social media platform used can also help researchers identify suspected fraudulent activity specific to links listed on certain social media platforms (Reed et al., 2024). In the same vein, researchers should also consider the types of social media platforms that are used for recruitment and whether the platforms offer opportunities to target specific audiences.
Consider How Social Media Platforms Are Used for Recruitment
Deciding on which social media platforms to use can depend on the POI and how they interact with social media. For example, if the POI has a large presence on Facebook, using that platform could bolster recruitment. Hashtags relevant to the study can also help research advertisements appear on the news feeds of the intended POI (Arigo et al., 2018). Sharing study information via public posts on social media sites can be a quick way to spread information about studies, but using targeted recruitment on social media can help study information be seen by members of the POI (Sinclair et al., 2019). One example of targeted recruitment can involve purchasing advertisements for a study on social media, where pre-set demographic information, such as age or location, is used to help reach a POI (Regents of the University of Colorado, 2024). However, one should consider the cost-benefit ratio of using paid advertisements for recruitment as this can be an expensive endeavor.
Engaging with the online community of the POI and building online relationships with social media account holders of the POI are also effective strategies for SMR. Furthermore, considering the substantial time and resources required to recruit members of the POI, evaluating platform data metrics with test posts can reveal which platforms are most effective for engagement (Wright et al., 2024). Another example of targeted recruitment can involve posting study information on private social media groups, such as Facebook groups, Reddit threads, or LinkedIn groups. Be sure to contact the administrators or moderators of social media groups to inquire if posting is allowed; otherwise, research posts could be removed.
Using personal social media accounts for recruitment is often debated. While this action could make respondents more easily identifiable to the primary investigator (PI), it could also result in a nonrandom, homogenous sample if the PI’s personal social media accounts are homogenous (Anderson, 2021). If planning a large-scale or longitudinal study, researchers can consider creating social media accounts designed to specifically advertise the study, such as a TikTok or X (formerly Twitter) account. However, if using this method, consider collaboration with information technology and survey design specialists to ensure survey sites can handle high-volume traffic/activity. While use of certain social media platforms can help researchers reach their target audience, keep in mind that online users who are not part of a researcher’s intended POI will likely be able to see the online postings as well. Thus, the consideration of compensation for participants and how to advertise compensation for studies becomes an additional layer of complexity when recruiting online.
Carefully Consider the Use of Incentives and Compensation
The decision to offer incentives depends on several factors, including whether researchers have access to funding for incentives and whether participants should be compensated for their time and effort. While offering incentives can support effective recruitment (Abdelazeem et al., 2022), researchers should consider the amounts and types of incentives relative to the risk associated with the study, participants’ efforts, and whether incentives can place undue influence on participants’ decision-making (Resnik, 2015). Offering incentives in studies where online recruitment takes place adds a layer of complexity regarding data quality. If incentives are advertised with online recruitment methods, such as through social media posts or online flyers, it may increase the likelihood of individuals trying to participate in the study under false pretenses to receive incentives (Salem et al., 2023).
If incentives are to be given, consider offering incentives that are geared towards the POI, such as donating towards a charitable cause that interests the POI to honor their time and efforts (Galárraga et al., 2020). If financial incentives, such as electronic gift cards, are the best way to compensate participants and support recruitment, consider the criteria needed to determine whether respondents can contribute towards meaningful data collection and assess whether responses to surveys or interview questions indicate the participant is an authentic member of the POI. Additionally, the use of random drawing systems, where participation in a study gives individuals the opportunity, but not a guarantee, to receive an incentive, can also be an effective recruitment strategy to elicit responses from authentic participants (Gabrielli et al., 2020; Ichimiya et al., 2023). Once studies using SMR methods are designed, approved by the IRB, and completed, dissemination of results is a logical next step. It is important for researchers to describe how they used SMR methods in their publications and how they protected data quality so readers can replicate and evaluate the integrity of the collected data.
Reporting the Use of Social Media Recruitment Methods
When preparing manuscripts for publication, it is necessary to list recruitment and data collection methods. The methods section gives readers an understating of the study's validity and how it could be reproduced (Oermann & Hays, 2019). Often, researchers using SMR will report their use of social media in ways that would allow the replication of studies but fail to report on how they protected the quality and integrity of data. Similarly, scientific journals often list rubrics or data disclosure requirements for authors, but requirements for disclosing data quality measures upon manuscript submission are scarce. More often, the peer review process is where data quality reporting is noted as sufficient or insufficient. Researchers can include a statement in their manuscripts about how the protection of data quality was achieved and list their steps in the process. This can include how respondents were verified eligible through qualitative or quantitative measures (i.e., phone calls, follow-up emails, secure survey platforms, etc.). For example, Reed et al. (2024) developed and published a protocol complete with detailed steps to share how they protected their online survey from fraudulent users. Transparency with recruitment methods, particularly in online environments, is crucial for the replication and evaluation of research studies.
Conclusion
If bodies of science, particularly social and health-related sciences, are based on faulty data, the results of studies and the policy and practice reforms derived from studies would produce adverse impacts. In this paper, we presented an overview of SMR, scenarios using SMR, and recommendations for researchers as they aim to uphold data quality while using SMR methods. Social media can expand equity and representation in research while also improving the timeliness and cost-effectiveness of recruitment. Still, all researchers should use and report their use of social media in a way that ensures accurate, reliable data collection to create functional bodies of science.