Effects of the Affordable Care Act on Health Care Access and Self-Assessed Health After 3 Years

Data

We use data from the BRFSS, an annual telephone survey organized by state health departments and the US Centers for Disease Control and Prevention. The survey collects information on various aspects of health care access and health for all 50 states and the District of Columbia. Having a large sample size is important for our study because the ACA affected health insurance coverage for only a fraction of the population, limiting plausible effect sizes. The BRFSS is the largest continuous health survey in the United States, collecting information on more than 300 000 adults per year.

We conduct our analysis using information from individuals 19 to 64 years old who were interviewed between 2011 and 2016. Individuals older than 64 years were excluded because the ACA was not intended to affect their health care coverage. Our sample starts in 2011 because this is the first year in which the BRFSS included cell phones in its sampling frame. A 2011-2016 sample period gives us 3 years of pretreatment data and 3 years of posttreatment data.

We utilize 9 outcome variables. The first 3 relate to access to care: indicators for any health insurance coverage, having a primary care doctor, and having any care needed but foregone because of cost in the past 12 months. The remaining outcomes relate to self-assessed health status. These include dummy variables for whether overall health is good or better (ie good, very good, or excellent), very good or excellent, and excellent, as well as days of the last 30 not in good mental health, not in good physical health, and with health-related functional limitations. Subjective self-assessed health variables such as these have been shown to be correlated with objective measures of health, including mortality.^31-33

The regressions include controls for demographic characteristics, household characteristics, economic characteristics, and measures that capture state differences in the implementation of the ACA. More specifically, we use BRFSS information to construct dummy variables for age groups (5-year increments from 25-29 to 60-64, with 19-24 as the reference group), gender (female), race/ethnicity (non-Hispanic black, Hispanic, and other, with non-Hispanic white as the reference group), marital status (married), education (high school degree, some college, and college graduate, with less than a high school degree as the reference group), household income ($10 000-$15 000, $15 000-$20 000, $20 000-$25 000, $25 000-$35 000, $35 000-$50 000, $50 000-$75 000, and >$75 000, with <$10 000 as the reference group), number of children in the household (separate indicators for 0 to 4, with 5 or more as the reference group), whether the respondent reports a primary occupation of student, and whether the respondent is unemployed. In addition, we use information from the Bureau of Labor Statistics to control for seasonally adjusted monthly state unemployment rate. Finally, we include dummy variables for whether states set up their own insurance exchanges and whether these exchanges experienced glitches.^34,35

One of our treatment variables, which measures the “dose” of the ACA’s impact, is the uninsured rate in the respondent’s “local area” in the pretreatment year of 2013. We compute each respondent’s “local area” pretreatment uninsured rate within our BRFSS sample. The BRFSS provides information regarding whether the respondent resides in the center city of a Metropolitan Statistical Area (MSA), outside the center city of a MSA but inside the county containing the center city, inside a suburban county of the MSA, or not in a MSA. The survey did not collect location information from cell phone respondents. We use this “local area” variable to construct 4 subgroups within each state: those living within a central city, suburbs, non-MSA, and location within the state unavailable (this is the case for respondents interviewed on cell phones). Based on these 4 within-state categories, we calculate the pretreatment average uninsured rates by location (considering “cell phone” to be a location for the sake of convenience) within a state.¹⁸ To ensure that each area contains enough respondents to reliably compute pretreatment uninsured rates, we combine the 7 areas with fewer than 200 respondents in 2013 with other larger areas. Specifically, we combine the central city and suburban parts of Wyoming into one area, and do the same for the states Vermont, South Dakota, and Montana. In addition, we combine the suburban and rural parts of Massachusetts, Arizona, and California. Ultimately, we have 194 areas with 2013 uninsured rates that are computed from 219 to 5804 respondents, with the average being 1475 respondents and the median being 1205.

Our Medicaid expansion variable is based on information collected by the Kaiser Family Foundation.³⁴ A total of 32 states expanded Medicaid by 2016. The majority of states expanded Medicaid in January 2014, with some exceptions. Michigan’s expansion took effect in April 2014 and New Hampshire’s in August 2014. Pennsylvania, Indiana, and Alaska expanded Medicaid in January, February, and September of 2015, respectively. Montana and Louisiana expanded Medicaid in January and July of 2016, respectively. States are classified as part of the Medicaid expansion treatment group beginning the month/year of their expansion.

Table 1 provides pretreatment means and standard deviations of the dependent variables, stratified into 4 groups based on whether the respondent’s state expanded Medicaid and whether her local area’s pretreatment uninsured rate was above or below the median within the sample. According to Table 1, 79% of the sample had insurance at baseline. Individuals in Medicaid expansion states were slightly more likely to have insurance prior to 2014 than those in nonexpansion states (regardless of baseline uninsurance rate levels). Residents who live in Medicaid expansion states with prereform uninsured rates below the median (column 3) had, on average, better health care access and self-assessed health than the rest of the sample even before the ACA was implemented. Our econometric design will account for these baseline differences. Our online appendix describes trends in our outcome variables over time as well as summary statistics for the control variables.

Data Analysis

Our goal is to estimate the 2014-2016 effects of both the fully implemented ACA (including the Medicaid expansion) and the ACA without the Medicaid expansion for each one of the outcomes of interest. The major challenge we face is in disentangling the impacts of the nationwide components of the ACA (eg, subsidized Marketplace coverage and the individual mandates) from underlying year-to-year fluctuations that would have occurred even in the ACA’s absence. To address this challenge, we adopt a DDD strategy used by 3 recent articles that estimate the ACA’s effects on health insurance coverage after 1 year and access to care, health, and ambulance response times after 2 years.^7,18,28 This approach differs from the DD linear regression strategy that has been used to compare Medicaid expansion to nonexpansion states before and after the ACA’s implementation by adding a third “difference.”

This third source of variation comes from differences in area pretreatment (2013) uninsured rates, which allows for the inclusion of time period fixed effects while still identifying the effect of the national components of the ACA.³⁰ Intuitively, we would expect to see larger responses to the national components of the ACA in areas with higher 2013 uninsured rates as a greater share of their residents could be affected. Combining this with a separate treatment from the Medicaid expansion implies that the intensity of treatment (size of the coverage expansion) was strongest in high pre-ACA uninsured rate areas in states that expanded Medicaid.

Formally, the DDD linear regression model, which combines 2014, 2015, and 2016 into a single postreform period, is given by Equation 1:

where

• $y_{iast}$ is the outcome for individual i in area type (central city, rest of MSA, non-MSA, cell phone) a in state s in month/year t,

• $POS{T}_t$ indicates whether period t is in the postreform period of January 2014 or later,

• $X_{iast}$ is a vector of control variables,

• $MEDICAI{D}_s$ indicates whether state s participated in the ACA’s Medicaid expansion,

• $UNINSURE{D}_{as}$ is the 2013 (pre-reform) uninsured rate in area type a within state s,

• ${\mathrm{\theta }}_{at}$ denotes fixed effects for each time-by-area-type combination (eg, non-MSA in March 2012); these control for time as flexibly as possible and also allow time trends to evolve differentially across individuals living in the 4 different area types,

• ${\mathrm{\alpha }}_{as}$ denotes fixed effects for each area (eg, central city in Georgia),

• and ${\mathrm{\epsilon }}_{iast}$ is the error term, which is heteroscedasticity-robust and clustered by state.

Note that $POS{T}_t$ is not separately included in Equation 1 since it is absorbed by the time fixed effects (${\mathrm{\theta }}_{at}$), while the terms $UNINSURE{D}_{as}$, $MEDICAI{D}_s$, and $UNINSURE{D}_{as}\times MEDICAI{D}_s$ are not separately included since they are absorbed by the area fixed effects $({\mathrm{\alpha }}_{as})$. Finally, BRFSS sampling weights are used to account for the complex survey design.

The effect of the ACA without the Medicaid expansion is given by ${\mathrm{\gamma }}_1$ × $UNINSURE{D}_{as}$, which means it is assumed to be 0 in a (hypothetical) area with a 0 percent uninsured rate at baseline and to increase linearly as the prereform uninsured rate rises. The effect of the Medicaid expansion alone is given by ${\mathrm{\gamma }}_3\times UNINSURE{D}_{as}\times MEDICAI{D}_s$, meaning it is 0 in nonexpansion states (where $MEDICAI{D}_s=0$) and ${\mathrm{\gamma }}_3\times UNINSURE{D}_{as}$ in expansion states (where $MEDICAI{D}_s=1$). As the Medicaid expansion should not causally affect coverage in an area with a 0 percent baseline uninsured rate, we consider ${\mathrm{\gamma }}_2$ to represent unobserved confounders rather than capturing part of the expansion’s causal effect.^7,30 The effect of the “full” ACA, ie, in Medicaid expansion states, combines the impacts of the Medicaid and non-Medicaid components: ${\mathrm{\gamma }}_1$× $UNINSURE{D}_{as}+{\mathrm{\gamma }}_3\times UNINSURE{D}_{as}$. In the tables, we report the predicted effect of the ACA at the sample mean pretreatment uninsured rate. This is given by ${\mathrm{\gamma }}_1$× $\overline{UNINSURE{D}_{as}}$ in nonexpansion states and ${\mathrm{\gamma }}_1$× $\overline{UNINSURE{D}_{as}}+{\mathrm{\gamma }}_3\times \overline{UNINSURE{D}_{as}}$ in expansion states.

In addition to average effects over the 2014-2016 time period, we are also interested in how the effects varied over time across these 3 years. To analyze changes over time, we estimate event-study models where we replace the before vs after 2014 time indicator in our DDD model with a set of individual year dummies. These models help to highlight the contribution associated with the inclusion of calendar year 2016 data. The event-study DDD model we estimate is formally given by Equation 2.

where Y_t is an indicator for whether year t is 2011, 2012, 2014, 2015, or 2016, respectively, for t = 1, 2, 3, 4, or 5, and the other terms are as described in Equation 1. Here the effects of the ACA without the Medicaid expansion during 2014, 2015, and 2016 are given by ${\mathrm{\theta }}_3\times UNINSURE{D}_{as},{\mathrm{\theta }}_4\times UNINSURE{D}_{as},$ and ${\mathrm{\theta }}_5\times UNINSURE{D}_{as}$, respectively, while the effects of the Medicaid expansion in 2014, 2015, and 2016 are similarly given by ${\mathrm{\beta }}_3\times UNINSURE{D}_{as},{\mathrm{\beta }}_4\times UNINSURE{D}_{as},$ and ${\mathrm{\beta }}_5\times UNINSURE{D}_{as}$.

Another advantage of the event-study model is that it allows us to test the identifying assumptions from our main DDD specification.^7,18 The first assumption is that, in the absence of the ACA, any changes in the outcomes that would have occurred in 2014-2016 would not have been systematically correlated with area uninsured rates, conditional on the controls. The second assumption is that, without the ACA, differential changes in the outcomes in 2014-2016 between Medicaid expansion and nonexpansion states would not have been correlated with prereform uninsured rates. While such counterfactuals cannot be directly observed, they can be predicted based on pretreatment trends. If the event-study model finds evidence that changes in the outcomes from 2011-2013 are correlated with area uninsured rate (ie, ${\mathrm{\theta }}_1$ or ${\mathrm{\theta }}_2$ are significant), this would suggest that the first assumption is likely violated. Similarly, evidence that changes in the outcomes from 2011-2013 are correlated with the interaction of area uninsured rate with Medicaid expansion status (ie, ${\mathrm{\beta }}_1$ or ${\mathrm{\beta }}_2$ are significant) would suggest a violation of the second assumption.

Effects on Access to Care

Table 2 focuses on the outcomes related to access. The top panel shows that the ACA led to statistically significant increases in access to care at the sample mean pretreatment uninsured rate in both Medicaid expansion and nonexpansion states in the postreform period. States that participated in the Medicaid expansion (and therefore received the national components of the ACA as well as the Medicaid expansion) saw a 9.5 percentage point increase in insurance coverage, a 3.4 percentage point increase in reporting having a primary care doctor, and a 5.6 percentage point reduction in the probability of reporting cost being a barrier to receiving care. Results were somewhat smaller in states that refused the Medicaid expansion (6.2 percentage point increase in insurance coverage, 3.1 percentage point increase in reporting having a primary care doctor, and 3.6 percentage point reduction in reporting cost being a barrier to receiving care). The difference between these 2 sets of estimates (a 3.3 percentage point increase in insurance coverage, no statistically significant change in reporting having a primary care doctor, and a 1.5 percentage point reduction in the probability of reporting cost being a barrier to receiving care) is also reported in Table 2 and is due to the Medicaid expansion alone. These results are generally larger than those found by a recent article using BRFSS data and the same identification strategy but only 2 posttreatment years, suggesting larger effects in 2016 than in 2014 and 2015.¹⁸

To investigate how the effects changed over time more directly, we next discuss the implied effects from the event-study specification. The results show that the gains in access to care, for the majority of measures, increased over time both in Medicaid expansion and nonexpansion states. The first 3 rows of the bottom panel of Table 2 show the results for nonexpansion states. The ACA led to an increase in the probability of having insurance coverage of 3.6 percentage points in 2014, 5.9 percentage points in 2015, and 8.3 percentage points in 2016 at the sample mean pretreatment uninsured rate. While we also see statistically significant increases in reports of having a primary care doctor in all 3 posttreatment years, the growth of the impact over time (2.0 percentage points in 2014 to 2.7 percentage points in 2016) is more modest than that for insurance coverage and is statistically insignificant. We observe a growing reduction over time in the probability of reporting cost being a barrier to receiving care, as the 2.7 percentage point reduction in 2014 rises to 4.1 percentage points in 2016.

The last 3 rows of Table 2 report the event-study results for states that participated in the ACA’s Medicaid expansion. While the increases in access are larger than the corresponding increases in nonexpansion states, we again observe the same general patterns of the gains strengthening over time. The effect on the probability of having coverage rises every year, from 6.5 percentage points in 2014 to 9.7 percentage points in 2015 and 11.8 percentage points in 2016 at the average pretreatment uninsured rate. Based on the sample means for the outcomes reported in Table 1, these results imply that the full ACA—including the Medicaid expansion—reduced the uninsured rate by 8.2% in 2014, by 12.3% in 2015, and by 14.9% in 2016. Next, while we see statistically significant increases in reports of having a primary care doctor in each year in expansion states, the change between 2014 and 2016, while sizable, is not statistically significant. The reduction in cost being a barrier to receiving care strengthens over time, from 3.0 percentage points in 2014 to 5.1 percentage points in 2015 and 6.0 percentage points in 2016. The difference between these 2 sets of results, representing the effect of the Medicaid expansion alone, is reported in the middle 3 rows of the bottom panel of Table 2.

Effects on Self-Assessed Health

Table 3 focuses on the outcomes related to self-assessed health. We find evidence of gains in some aspects of self-assessed health, which contrasts a recent article’s null results using BRFSS data and the same econometric strategy but only 2 years of posttreatment data.¹⁸ The top panel shows that, in the combined 3-year posttreatment period, residents of Medicaid expansion states (receiving both the national components of the ACA and the Medicaid expansion) saw a statistically significant 1.3 percentage point increase in the probability of reporting excellent health at the average pretreatment uninsured rate and a reduction in days not in good mental health of 0.2 per month. In nonexpansion states (receiving only the national components of the ACA), we find 2 statistically significant results—a 1.6 percentage point increase in reporting excellent health and a 1.2 percentage point increase in reporting very good or excellent health—as well as an impact on mental health that is almost as large as that for expansion states, though not significant. We also find that the Medicaid expansion alone led to an increase in days with health-related limitations. Together, these results suggest that gains in self-assessed health from the ACA are mostly or entirely attributable to the expansion of private insurance among low-to-middle-income individuals rather than the Medicaid expansion among those with low incomes.

Appendix Figure 2 provides suggestive evidence that the gains in self-assessed health in nonexpansion states are the result of both increases among those in states with prereform uninsured rates above the median and reductions in states with prereform uninsured rates below the median. Also discussed further in the appendix is an alternative specification to evaluate our finding that the Medicaid expansion is not a driver of the increase in self-assessed health: a DD analysis of the effect of the Medicaid expansion among a sample of low-income individuals (<$25 000 per year). Consistent with our DDD results, the DD results show that the Medicaid expansion led to improvements in access to care but not self-assessed health.

The bottom panel of Table 3 presents the event-study results for the self-assessed health outcomes, which clearly illustrate the importance of the third posttreatment year. In nonexpansion states, we see large and statistically significant gains in 2016 in reports of good or better health (1.5 percentage point increase), very good or excellent health (4.3 percentage point increase), and excellent health (3.5 percentage point increase). These results suggest gains across the entire distribution of self-assessed health. We also see a statistically significant reduction in days with health-related limitations in 2016. The only result in these outcomes that was significant in 2014 or 2015 was a 2 percentage point increase in very good/excellent health in 2014—and the size of this effect more than doubled in 2016. In states that fully implemented the ACA, we see statistically significant increases in reports of excellent health in 2015 (1.9 percentage point increase) and 2016 (2.7 percentage point increase). Based on the sample means for the outcomes reported in Table 1, these results imply that the full ACA increased the probability of reporting excellent health by 9.3% in 2015 and by 13.2% in 2016. As with the DDD results discussed previously, the fact that the event-study model reveals clearer gains in self-assessed health in non-Medicaid-expansion states than in expansion states suggests that health gains from the ACA are driven by the private expansion rather than the Medicaid expansion. This interpretation is also supported by our estimates of the impact of the Medicaid expansion alone, reported in the middle 3 lines of the bottom panel of Table 3, which suggest that the Medicaid expansion led to a 2.5 percentage point reduction in reporting good or better health and an increase in days with health-related limitations in 2016.

Given the large number of null hypotheses tested in Tables 2 and 3, we might expect a few significant results to emerge merely by chance. Specifically, with 9 outcomes and 12 reported results for each, there are a total of 108 hypothesis tests, meaning that 5 to 6 spurious results would be expected to emerge using a 5% significance level. However, we observe 45 significant results across the 2 tables, strongly suggesting that chance alone cannot explain our findings.

Testing Identifying Assumptions of the Model

As discussed previously, another benefit of our event-study analysis is that it enables tests of the model’s identifying assumptions by asking whether the coefficients on the interactions of $UNINSURE{D}_{as}$ and $UNINSURE{D}_{as}\times MEDICAI{D}_{st}$ with the year 2011 and 2012 dummies are statistically significant relative to the base year of 2013. We found only 1 placebo test failure out of 36 (4 for each of the 9 outcomes) using a 5% significance level. One out of 36 is 2.8%, which is slightly below the number of rejections that we would expect to obtain by chance. In addition, for each regression we conducted an F test of the null hypothesis that all of the interactions for the 2011 and 2012 year dummies are equal to 0. The results are reported at the bottom of Tables 2 and 3 and are statistically insignificant in all but 1 of the 9 regressions. The full event-study results are presented graphically in Figures 3 to 6 in the appendix. We also include in our appendix results from a large number of specification checks that strongly support our findings for the health care access outcomes in Table 2 and usually support those for the self-assessed health outcomes in Table 3. Individual exceptions are noted in our discussion of the appendix tables.