How Sensitive Are Multilevel Regression Findings to Defined Area of Context? A Case Study of Mammography Use in California

Modifiable Areal Unit Problem

Areal units are geographic entities such as counties or states. Zonal partitioning schemes are decisions about how to draw the boundaries for areal units. The MAUP emerges when data developed as contextual factors for various areal units have different statistical properties. When the underlying microdata values are not uniformly distributed across the landscape, merging smaller units into larger areas using different zonal partitioning schemes is the same as smoothing (averaging) across different combinations of spatial neighbors (Wong, 1996). When these zonal schemes overlap (are not nested), the different combinations of spatial neighbors can yield constructed variables with quite different statistical properties (Ali et al., 2005; Greenland, 2002; Tatalovich et al., 2006). Both signs and significance levels of regression coefficient estimates can be affected, and considerable variability has been found across small-area studies examining essentially the same phenomenon (Cockings & Martin, 2005; Fotheringham & Wong, 1991; Waller & Gotway, 2004).

Potential for MAUP in Our Data

The four areal units used in our California analysis are the county (n = 57), the medical service study area (MSSA; n = 519), the primary care service area (PCSA; n = 333), and the U.S. Census ZIP code tabulation area (ZCTA; n = 1,450).¹ Table 1 gives definitions of these areal units and information about nesting and non-nesting of these with other areal units. Areas that do not nest have overlapping boundaries, such as ZCTAs and counties. Areas that do nest are contained within or perfectly contain one another, such as MSSAs and counties. For example, Los Angeles County has 295 ZCTAs, 44 PCSAs, and 98 MSSAs; PCSA or ZCTA and MSSA boundaries cross, while ZCTAs are nested inside PCSAs.

Both MSSAs and PCSAs were defined (by the California Office of Statewide Health Planning and Development, 2005, and the Health Resources and Services Administration, n.d., respectively) on the basis of economic principles but for different populations. MSSAs partition counties into smaller areas on the basis of the health resource needs of the entire population and degree of rurality. MSSA markets are defined for the entire population and nest within the county, dividing them into subregions that are classified as urban, rural, or frontier (Christman, 2004). By contrast, PCSAs are physician market areas derived from observed flows of elderly patients to primary care providers (Goodman et al., 2003). Both the PCSA and MSSA areas are much smaller than counties, which in California are so large that both urban and rural areas are contained within single counties. Thus, PCSAs and MSSAs do a much better job than counties in separating rural from nonrural areas. The ZCTA is the smallest area we are able to study because of privacy limitations on the individual-level data.

The overlapping PCSA and MSSA boundaries and differences in partitioning of the underlying populations can yield MAUP. This is illustrated in Figures 1 and 2, which show the Los Angeles County, California, area, picked as an example to show how the MAUP problem can arise as a result of underlying heterogeneity in the microdata. Figure 1 shows the census of elderly women in Los Angeles County partitioned into 98 MSSAs, which nest completely inside and cover all spaces within the county. Figure 2 shows the census of elderly women in the Los Angeles County area partitioned into 44 PCSAs, several of which cross the county boundary. The PCSA map (Figure 2) is based on 574,592 elderly women residing in 44 PCSAs centered in (but not fully contained within) Los Angeles County. The MSSA map (Figure 1) is based on 541,915 women in 98 MSSAs completely inside Los Angeles County. To illustrate the potential for the MAUP, the counts of elderly women (variable) in these MSSA and PCSA areas are represented in the maps using standard deviations from their means. Contrasting patterns in the two maps show the different statistical properties of this same variable when partitioned into MSSAs versus PCSAs.

There are other differences in the MSSA and PCSA areal coverages. In Figure 2, there are some holes (missing areas) evident in the PCSAs; these are areas within Los Angeles County built up from alphanumeric ZCTAs, which cover water bodies or uninhabited lands and have no census data attached to them. More generally, the PCSA areal coverage has holes in places that are not inhabited, while the MSSA coverage includes all land areas and exhibits no holes. (Note that these holes coincide with MSSAs where the counts of elderly women are sparse, more than 2 standard deviations below the MSSA mean.)

The Hybrid Conceptual Model

The behavioral model of use developed by Aday and Andersen (A-A; 1974) is frequently used in describing factors associated with health care access and use, but these studies rarely include many contextual factors (Andersen, 1995; Phillips, Morrison, Andersen, & Aday, 1998). By contrast, models of health outcomes commonly use contextual factors at multiple levels within a socioecological framework (Booth et al., 2001; Northridge, Sclar, & Biswas, 2003; Schulz et al., 2005; Smedley & Syme, 2000).

Although the idea that the community influences human behavior has been around for some time (Bronfenbrenner, 1979), explicit consideration of the relevant zone of influence for ecological variables has only recently begun to appear in the socioecological literature (Booth et al., 2001; Northridge et al., 2003; Pickett & Pearl, 2000; Sampson, Morenoff, & Gannon-Rowley, 2002; Schulz et al., 2005; Smedley & Syme, 2000). Pickett and Pearl (2000) argued that findings may vary across studies using different zonal schemes precisely because the so-defined areal units presume very different zones of influence for the contextual variables, when these distinctions truly matter.

In this article, we blend the health behavioral and socioecological perspectives in our unique hybrid spatial interactions model. We begin with the model of health care access and use developed by Khan and Bhardwaj (1994), which defines the use outcome as a process involving the interplay between the characteristics of the health care service system, potential users, and the social and physical features of places. This model provides a distinctly spatial view of human interaction with the environment and other structural and social aspects of the community. Some limitations of the Khan and Bhardwaj (K-B) model are that it does not consider the levels of influence that are central to the mainstream socioecological perspective or differentiate personal characteristics into the enabling–predisposing need constructs in the A-A behavioral health model. These constructs are useful when considering multilevel interactions between people and environmental characteristics along the pathways to health care use (Litaker, Koroukian, & Love, 2005).

We adapt the K-B model to create a unique hybrid of the K-B and A-A models and the socioecological perspective, as shown in Figure 3. The white “health care system” and “individual characteristics” boxes reflect constructs in the K-B model, which are filtered through contextual factors in a separate barrier–facilitators box (not shown), determining observed use. This K-B barriers–facilitators box is represented in Figure 3 by a series of nested constructs: the large box surrounding the entire figure (fundamental–distal factors) and the two smaller nested ovals within it (intermediate–community and interpersonal–proximate factors). The two smaller ovals partition the K-B contextual variables into three different levels of influence, consistent with the traditional socioecological model. The hybrid model thus views the individual as making use choices (the final outcome) in a market context that has differentiated levels of influence for different classes of contextual variables.

The conceptual model guides the selection of variables to be included in the analysis and suggests the appropriate level of aggregation for contextual variables through the classification of zones of influence. The variables chosen for this work, where they fit in terms of the concept, and their sources are described in Table 2, which is divided into three sections: individual characteristics (categorized into enabling, predisposing, and need categories), interpersonal–proximate factors, and intermediate–community factors. Each variable is described as to where it fits in the conceptual model. The interpersonal–proximate factors are measured at three different levels of aggregation reflecting local communities (ZCTA, PCSA, and MSSA) so that we can assess whether these different definitions matter empirically. The intermediate–community factors are measured at the county level, which is larger than the local community areas. The fundamental–distal factors category is not represented in the analysis because we view these as state- or regional-level variables.

Empirical Model

A multilevel binary probit model was used to examine how the contextual factors affect screening uptake among elderly California women. The dependent variable in the probit model is a binary variable indicating whether a woman received at least one mammogram during the years 2002 and 2003. The predicted value from the regression is a Z score, which follows the standard normal density distribution. The estimated parameters were converted into marginal probabilities when presenting the findings, so coefficient estimates can be interpreted as changes in probability of use associated with a small (1-unit) change in the covariate. We used SAS general estimating equations to correct the standard errors for biases stemming from redundancies in the contextual variables within areal units.

The empirical model includes person- and area-level variables, and four versions of the model were estimated, each with a different areal unit for the interpersonal–proximate contextual variables (ZCTA, PCSA, MSSA, and county). The first three models (ZCTA, PCSA, and MSSA) are three-level models with county-level variables at the intermediate–community level. The fourth model is a two-level model that uses the county as the areal unit for all contextual variables, to enable answering our study questions about levels of influence. The variables and their sources are described in Table 2; Tables 3 and 4 contain sample statistics.

Because physician markets based on elderly person flows to doctors (the basis for PCSAs) and resource needs (the basis for MSSAs) are both based on economic factors, they are both relevant market definitions from an economic perspective and thus useful for small-area analyses. For the current analysis, however, they are used to approximate the zones of influence for various interpersonal–proximate contextual factors, which reflect small communities or neighborhoods that are not necessarily the same as health market areas. However, both areal units differentiate urban and rural areas as separate places, in contrast to counties, which include both in their borders. Because both of these area-level contextual variables account for urban intensity and are defined over areal units of similar size, we expect their levels of influence to be fairly similar but different from their county-level counterparts. However, the overlapping PCSA and MSSA boundaries and differences in partitioning of the underlying populations can yield MAUP (i.e., derived variables with different statistical properties); thus, dissimilar estimation results might emerge because of differences in the ways that the underlying heterogeneous data are partitioned.

ZCTAs are the smallest areal units that we can define, given privacy restrictions within the data. ZCTAs are zones used for mail delivery and may be more representative of small neighborhoods than the MSSA or PCSA measures, although they are most likely not ideal neighborhood definitions. ZCTAs nest inside PCSAs but cross MSSA borders, so we might expect more MAUP in the comparison of ZCTA and MSSA findings. Similarly, MSSAs nest inside counties, so we might expect less MAUP in comparisons of MSSAs and counties than in comparisons of MSSAs and PCSAs. In the empirical analysis, we answer the following research questions:

Question 1: Are the empirical findings similar across the three three-level models?

Question 2: If the three-level model findings are similar (thus providing no evidence of MAUP effects), are they also similar to the two-level (county) model findings?

If the answer to Question 1 is “yes,” then despite the crossing boundaries and different partitioning of underlying data when creating the areal-level data for the ZCTAs, PCSAs, and MSSAs, no MAUP is evident. If no MAUP is evident across these three areal units, then we would expect none with the fourth (county). Therefore, we would expect the answer to Question 2 to be “yes,” unless there are true zone-of-influence differences, since the county is much larger than the other three areal units. With this logic, the answers to these questions will help us determine whether there are apparent differences in findings deriving from the assumed zones of influence imposed by the areal units used in the models.

Individual Characteristics

The impacts of person-level variables are essentially the same across the four models, with the exception of the race or ethnicity variables. For Asians, African Americans, American Indian/Alaska Natives, and other, the findings are comparable across models; being of these races or ethnicities reduces the probability of mammography use, relative to Whites (the reference group). Hispanic ethnicity does not significantly affect use in any model except the county-level model, where there is a weakly significant negative effect. MAUP or zone-of-influence effects were not expected on the individual-level variables, and findings confirm this.

Interpersonal–Proximate Factors

The isolation index reflects residential segregation of minority groups and is (mostly) significant and negative for both African American and Hispanic variables. These are interpreted as lower probability of mammography use for women living in communities where there is higher segregation of African Americans or Hispanics. This finding is consistent across all four models, but significance levels are lower in the county-level model. The consistency in sign across models suggests that there is no MAUP effect, while the reduced significance at the county level suggests that there are perhaps important zone-of-influence effects. More specifically, these isolation variables seem to influence outcomes at areal units much smaller than the county.

Women living in communities with greater segregation of American Indian/Alaska Native populations seem more likely to use mammography, which is opposite the finding for women in communities with higher segregation of African Americans or Hispanics. This suggests that there is social support in segregated American Indian/Alaska Native communities that promotes use of mammography. For women living in these communities, the estimated association is large and positive in three models but not in the MSSA model, whereas both the ZCTA (smallest area) and county (largest area) models show significant positive coefficients. These findings suggest that there are MAUP effects causing inconsistent findings across the MSSA and other models for this variable, but this evidence suggests no distinctions between different zones of influence from small to large communities.

Regarding the MAUP effects, Native American populations in California tend to be most segregated in the isolated rural, frontier areas of MSSAs. These areas often coincide with the rural holes (noted earlier) in the PCSA coverage, which means that MSSAs and PCSAs will vary considerably in how they carve up the underlying populations used in calculating the American Indian/Alaska Native isolation index. No other racial or ethnic group in California has high segregation in rural areas, which may explain why we see MAUP for this variable and not others.

Women living in communities with greater proportions of those with long commutes (traveling 60 min or more each way to work) are significantly less likely to use mammography across all models. This is consistent with previous findings in small-area analysis of PCSA markets regarding commuter intensity and use of preventive services by elderly persons (Mobley, Root, Anselin, et al., 2006). Two other stressors expected to have negative associations are living in communities with greater proportions of elderly persons in poverty or communities with greater proportions of elderly women living alone. The elderly persons in poverty and women living alone coefficients are quite comparable across the models, and they are significant in all but the county model. This finding suggests that there is no MAUP, but there may be important zone-of-influence effects. These stressors seem more appropriately defined at the smaller areal units, which is consistent with our conceptual model. However, we did not expect to find a positive impact on a woman’s probability of use from living in communities where a greater proportion of elderly women live alone. This finding warrants further study.

Intermediate–Community Factors

The community-level factors were entered at the county level in all models, following the zones of influence identified in our conceptual model, so we did not expect to see any MAUP effects across models. Nurses per capita, violent crime rates, and land use mix are consistent, significant predictors across all four models. Physician variables do not have significant impacts, and managed care climate is only weakly significant. It is not surprising that elderly women living in areas with higher crime rates are less likely to use mammography. The land use mix variable is larger when communities are more diverse and urban, and living in a place with a more urban aspect seems to discourage mammography use. Managed care penetration seems to have a small positive effect, suggesting beneficial spillovers in preventive care behaviors from managed care presence in the community, but the evidence is statistically weak.

Limitations

Further elaboration of the model could include explicit interaction effects between the contextual factors at different levels and the enabling characteristics of individuals. We exclude these interactions for simplicity but include a multitude of factors and characteristics to make the model as general as possible. When applying the model to a particular problem, one caveat is that all items are not equally influential in modeling different aspects of access or use for different populations or services, and interaction effects may vary with the problem under study. It is important to consider carefully all of the relevant variables that should be included in the model because omitted variables bias is another source of inconsistency in parameter estimates across studies using different zonal schemes (Dietz, 2002). Thus, we may have erred, providing fruitful opportunities for future research.

In the single time interval examined here, this cross-sectional analysis is limited in its ability to infer causal effects; thus, the relationships discovered are merely statistical associations. The contextual variables all predate the outcome variable (Table 2) so they are predetermined; even so, there may be simultaneity within the probability-generating spatial interaction. Another limitation may be the ability to generalize the policy findings to other areas or states. Finally, we were not able to examine areas smaller than ZCTAs for our assessment of MAUP. However, it is unlikely that data would ever be available at finer areal units because of privacy concerns.