Participation in Research Studies: Factors Associated with Failing to Meet Minority Recruitment Goals

Introduction

Racial and ethnic minority groups have been underrepresented in clinical research compared to their burden of disease and their representation in the U.S. population [1-3]. This pattern has spurred research on potential barriers to minority research participation [4-9], often focusing on the perceptions of potential study participants by surveying minority patients in a wide variety of clinical and community settings [5, 6, 9-12]. While potential study participants are valuable sources of information on barriers to minority recruitment, investigators can also provide valuable information as the coordinators of recruitment and enrollment. Previous assessments of researchers’ opinions about minority recruitment have identified barriers such as a perception of lower interest in clinical trials among minority patients and a lack of investigator confidence in explaining clinical trials in culturally appropriate terms [13].

Despite the perception of barriers to minority recruitment, we know very little about the recruitment goals that investigators set for racial and ethnic minority groups and whether they meet these goals. Investigators’ minority recruitment goals likely parallel their intentions to include diverse study samples in their research. The National Institutes of Health (NIH) requires investigators to report recruitment goals for minorities to demonstrate adequate minority representation in clinical studies [14]. Yet, there is a dearth of peer-reviewed data not only on investigators’ minority recruitment goals, but also on the actual rates of minority enrollment and the various factors that may influence those rates [15, 16]. Investigator surveys are essential tools for assessing recruitment goals and investigators’ failure rates in meeting their goals. We conducted a nationwide survey of federally funded Principal Investigators (PIs) to assess 1) the goals they set for minority recruitment, 2) the percentage of PIs that failed to reach their goals, and 3) whether PI and study characteristics and PIs’ perceptions of minority recruitment are associated with their failure to meet minority recruitment goals.

Methods

Results

Of the 683 surveys mailed, 497 were returned by respondents. Fifty-seven (11%) of the 497 respondents were ineligible to complete the survey, because they either 1) were not the PI on any NHLBI-funded studies that were active in 2001 or 2) did not have any studies in 2001 with human subjects. After excluding the 57 ineligible respondents, our response rate was 70%.

Of the 440 survey respondents (Table 1), the majority were male and white with a mean age of 50.8 years (SD 8.2). Most respondents held a medical degree and were full professors. Clinic/hospital-based recruitment, word-of-mouth, flyers, and physician referral were the most frequently reported recruitment methods used in recruiting all subjects (Table 2). On average, respondents reported experience with 6.7 (SD 3.5, range 0−17) different methods/sites for recruitment. Of the barriers identified by respondents, the average rating was between “minor” and “moderate” (average respondent barrier score 2.45, SD 0.62). Sixty percent of respondents reported that they had not completed enrollment for their most recent NHLBI-funded studies at the time of our survey.

Many PIs did not set recruitment goals for individual minority groups. Investigators set recruitment goals most frequently for African Americans (91%). Eighty-eight percent set goals for whites, 55% for Asians Americans, 35% for American Indians/Alaskan Natives, 22% for Native Hawaiians/Pacific Islanders, and 67% for Hispanics. Only 2.7% of respondents set goals for all of the racial/ethnic minorities. In unadjusted bivariable analyses, factors such as completed study enrollment (chi square p<0.05 for African Americans, American Indian/Native Alaskan, Native Hawaiians/Pacific Islanders and p<0.001 for Asian Americans and Hispanics) and the use of fewer recruitment sites or methods (chi square p<0.05 for Asian Americans, and American Indians/Native Alaskans, Native Hawaiians/Pacific Islanders) were associated with a lower likelihood of actually setting goals for minority recruitment across multiple racial/ethnic groups.

Based on the actual proportions (%) reported among investigators both setting recruitment goals and reporting completed enrollment, the mean minority recruitment goal was highest for African Americans at 33% and lowest for Asian Americans at 10% (Table 2). Mean recruitment goals were not significantly different when including those investigators who had not yet completed enrollment (data not shown). Respondents were more likely to fail to reach their recruitment goals for African Americans, Asian Americans, and Hispanics and when compared to whites (McNemar's p-value < 0.05 in pairwise comparisons). On average, investigators reached larger proportions of their recruitment targets for whites compared to the proportions of recruitment targets reached for the racial/ethnic minority groups (Table 2). The recruitment and enrollment figures for Pacific Islanders/Native Hawaiians and American Indians/Alaskan Natives were not included in these calculations because so few respondents set recruitment goals for these subgroups.

Additional analyses were based on the use of the calculated 95% upper bound for investigators’ reported enrollment proportions to define failure to meet minority recruitment goals among all PIs, regardless of completion of enrollment. Using this definition for our outcome, the mean minority recruitment goal was highest for African Americans at 31% and lowest for Pacific Islanders/Native Hawaiians at 3% (Figure). Respondents were least successful at reaching their minority recruitment goals for African Americans (26.7% failed to meet recruitment goals) and most successful with the recruitment of American Indians/Native Alaskans with a failure rate of 18.6% (Figure). Using McNemar's test in pairwise comparisons, we found that failure rates in meeting recruitment goals for African Americans were significantly higher when compared to the recruitment of both whites and Asian Americans (data not shown).

Unadjusted bivariable analyses identified those variables associated with the failure to meet recruitment goals for each group (Table 3). These factors varied across the different racial/ethnic groups. Likewise, multivariable regression analyses confirmed the independent association of these factors with failing to meet minority recruitment goals. These factors also varied across minority groups in multivariable analyses (Table 4). The investigators who had completed study enrollment, were conducting an observational study (vs. phase III trials), or identified a larger number of major barriers were more likely to fail to meet their minority recruitment goals for African Americans in the adjusted analysis. Conversely, having greater than 20 years of funding as a PI was associated with a lower likelihood of failure to meet minority recruitment goals for African Americans. In the recruitment of Asian Americans, PIs conducting observational studies and phase I or II trials were less likely to reach their recruitment goals when compared to those respondents conducting phase III trials. None of the factors that we assessed was independently associated with failure to meet recruitment goals for Hispanics. Multivariable analyses were not performed for the recruitment of American Indians/Native Alaskans or Native Hawaiians/Pacific Islanders due to the small number of respondents setting goals for both these groups.

Discussion

In our national survey of PIs, we found that many respondents did not set goals for one or more racial/ethnic minority groups. Those PIs who did set goals for the racial/ethnic minority groups reported both a wide range of goals and varying rates of failure to reach their goals for each minority group. We assessed multiple factors such as the number of recruitment methods or sites used, PI perceptions of recruitment barriers, and investigator and study characteristics, but none of these was consistently associated with the failure to meet minority recruitment goals across different racial and ethnic groups.

In fact, the majority of respondents did not set goals for the recruitment of smaller minority groups such as American Indians/Native Alaskans and Native Hawaiians/Pacific Islanders. However, many respondents also did not set goals for larger minority groups such as Hispanics and Asian Americans. Setting minority recruitment goals may be important to achieve a priori methodologic goals for the inclusion of minorities. For instance, the numbers of minority participants needed for hypothesis testing may differ greatly from the numbers needed for more exploratory analyses aimed at generating new hypotheses [19]. The large number of PIs not setting minority recruitment goals suggests that investigators may not be planning for the recruitment of some minority groups when designing their research studies.

The NIH Revitalization Act included a universal mandate for all NIH-funded researchers which broadly stipulated that trials be designed to allow for “valid analysis” of racial differences [14]. However, the NHLBI defines appropriate minority recruitment more specifically, mandating that investigators recruit underrepresented groups such that “minorities in the study population are in the same proportions as in the U.S. population having the disease entity being studied” [20]. The NHLBI guidelines even make allowances for studies focused on diseases with unknown prevalences, mandating that minorities in these studies be represented in proportions equal to their representation in the total U.S. population. If these criteria are not met, the NHLBI requires PIs seeking funding to justify why they have planned for study populations of limited diversity. Furthermore, while the adequate representation of every minority group is not mandated in every study, NHLBI guidelines state that “broad representation and diversity are strongly encouraged, even if multiple clinics and sites are needed to accomplish it”. Despite the broad NIH mandate and more specific NHLBI guidelines, many investigators in our sample did not set recruitment goals for both large and small racial/ethnic minority groups. The lack of data on investigators setting a priori minority recruitment goals has made past efforts to measure or define recruitment “success” very challenging [19]. Even PIs who set minority recruitment goals may have used a broad range of criteria, aside from the NIH and NHLBI guidelines, in establishing those goals. Rather than trying to apply a universal standard of “successful” or “adequate” minority recruitment, we assessed how PIs fared in reaching their own reported recruitment goals.

We reported failure to meet minority recruitment goals based on both actual recruitment proportions and a calculated 95% upper bound for enrollment proportions. The use of the 95% upper bound allowed us to control for variation in samples sizes and in the stages of subject accrual in investigators’ studies. The likelihood of PIs failing to meet minority recruitment goals was higher compared to the proportion failing to meet goals for whites based on the use of the enrollment proportions. These same differences were less pronounced when failure to meet goals was defined by the 95% upper bound for enrollment proportions. However, use of the 95% upper bound provides a very conservative definition of failure to meet goals, essentially requiring sufficient evidence to reject the null hypothesis that the recruitment process produces at least the target proportion for the particular racial or ethnic category. We feel this approach minimizes the bias in our assessment of the outcome of failure to meet recruitment goals.

Using the 95% upper bound to define our outcome, the rate of failure to meet minority recruitment goals was near 20% in the recruitment of all racial and ethnic groups. This finding suggests that, among those PIs setting recruitment goals, investigators have difficulty reaching goals for all racial and ethnic groups, including whites. However, investigators, on average, set much higher goals for whites than for any of the racial/ethnic minority groups. While failing to reach recruitment goals for whites may or may not jeopardize the overall statistical power of a study depending on the effect size, simultaneous failure to reach comparatively smaller recruitment goals for racial/ethnic minorities may actually preclude any “valid analysis” of differences between subgroups. Therefore, failing to meet recruitment goals for all racial/ethnic groups may pose a larger challenge in either the confirmation of previously observed racial differences or the generation of new hypotheses based on racial differences.

While some factors were associated with failure to meet recruitment goals for individual racial/ethnic groups, none of these associations remained consistent across the groups. Previous research has suggested that measures of investigator and study characteristics, such as perceived barriers and the number of recruitment sites or methods, could potentially be associated with successful minority recruitment [21, 22]. However, in our analysis, each of these two variables was an aggregate measure of several factors that, when considered individually, may be associated differently with success or failure in minority recruitment efforts. We did not analyze the association of individual barriers or the use of individual recruitment methods or sites with the likelihood of failure to meet minority recruitment goals. The collective effect of these factors may differ from their individual effects on PIs’ efforts to meet their minority recruitment goals. Moreover, whether analyzed individually or collectively, the impact of these factors also may not be consistent across different racial/ethnic groups because of the varying influences among different cultures. For example, among African Americans, recruitment methods such as establishing long-term community partnerships and enlisting the aid of community leaders have been successful in individual studies [23-27]. Likewise, some authors have attributed successful African American recruitment to their use of recruitment sites such as churches and barber and beauty shops [28]. However, less published data exists on the use of these recruitment methods among other minority groups such as Hispanics and Asian Americans. Because the other groups have unique cultural influences that are distinct from those of African Americans, the recruitment methods, as well as other factors that we assessed may not have the same impact in the recruitment of other racial and ethnic minorities. Future efforts may yield more specific information by focusing on minority groups individually when identifying barriers to research participation.

Our study has several limitations. First, we surveyed PIs receiving funding from a single institute of the NIH. While this group of PIs represents a broad spectrum of research interests and conducts a wide variety of research studies, their recruitment practices, perceptions of minority recruitment, and professional and study characteristics may not be representative of investigators receiving funding from other NIH institutes or other federal or non-federal agencies. However, because of the specific guidelines set forth by the NHLBI, these investigators may, in fact, be more likely to focus on increasing the diversity of their study samples. We only questioned each PI about the recruitment goals for his/her most recent NHLBI-funded study. Though this method allowed us to determine if PIs failed to reach their minority recruitment goals for the study of interest, it may not be indicative of their recruitment experiences in other studies. We were also limited by our inability to collect detailed information on PI access to minority populations. Finally, we do not know if PIs’ reported recruitment goals are congruent with the goals they were required to report in their original grant applications for the study of interest, and thus we do not have access to any justifications they may have provided in their grant applications for not planning for a diverse study sample.

Setting specific recruitment goals is important in efforts to determine minority recruitment “success” or “failure” [19], but our study shows that many PIs do not set recruitment goals for some minority groups. Of those PIs who set recruitment goals for one or more minority groups, some did not reach these goals. Because it appears that the factors associated with PIs’ failures to meet their own minority recruitment goals may differ across different racial or ethnic groups, the recruitment of different minority groups may have to be assessed individually, and ultimately unique interventions may have to be specifically designed for each group.