Do social interventions have different health effects for different people? Why heterogeneous treatment effects are important in population health research

This series of two Methods Notes focuses on the related phenomena referred to in different fields as “effect modification”, “interaction”, “effect moderation”, or “heterogeneous treatment effects”. Here we use the term “heterogeneous treatment effects”, or HTEs. We explain what HTEs are, why HTEs are so important for population health research, challenges in evaluating HTEs, and open questions for future research. HTEs are important regardless of the type of exposure being considered, including clinical treatments, programs, policies, or systems. For this note, we use the term “treatment” as shorthand for any type of exposure.

What are heterogeneous treatment effects?

Many social interventions may have larger benefits for some people but smaller benefits for other people. Such differences in the effect of a treatment on different types of individuals are referred to as heterogenous treatment effects. For example, evidence from the Moving To Opportunity intervention showed that providing subsidized housing vouchers to residents of public housing developments was beneficial for girls but harmful for boys, at least for some outcomes.1 As another example, the Korean War GI Bill provided socioeconomic benefits to veterans; the intervention was associated with fewer depressive symptoms for veterans with low childhood socioeconomic status but not for those from high childhood socioeconomic status backgrounds.2 Such HTEs are important for health equity research, especially if an intervention may harm one group while benefiting another.

Numerous research designs are used to evaluate HTEs. We distinguish between research evaluating whether a pre-specified characteristic (e.g., race) defines groups who on average respond differently to treatment and research evaluating whether there is any heterogeneity in treatment response, perhaps across as-yet-unknown characteristics. The vast majority of research on HTEs focuses on the former, but emerging methods address the latter type of question (see Part 2 of this Methods Notes series).

Why are heterogeneous treatment effects so important?

When describing the effect of a policy or program, we are often most interested in the total effect on the population—i.e., the difference in health that would be achieved if everyone in the population was treated compared to if nobody was treated. This is sometimes called the marginal effect. However, evaluating whether treatment effects differ for particular subgroups of the population is important to E4A for four reasons.

#1 Anticipating whether a proposed policy or program will exacerbate or reduce health disparities.

Conceptually, unequal health outcomes between two groups may be due to different exposure to factors that influence health, different effects of those exposures on different groups of people, or both differential exposure and differential effects. For example, compared to non-Hispanic Whites, Black or African American children may be more exposed to environmental pollutants that trigger asthma, and the same level of exposure to the same pollutant may be more harmful for Black or African American children than non-Hispanic White children. Both differential effects (i.e., HTEs) and differential exposure can contribute to inequalities.

#2 Predicting the effect of an intervention in a population that is different in composition from the one initially studied.

To do this, we need to understand how treatment effects differ for different population subgroups. For example, if a study of the causal effect of subsidized college education on health included effect estimates for low-income and high-income individuals, the results of this study can be used to anticipate the effect of subsidized college education for a different community–perhaps one that is predominantly high-income or predominantly low-income. Because E4A seeks to fund research on interventions that can be scaled to many diverse communities, anticipating how intervention effects may differ across populations is critical.

#3 Estimating the effect of a treatment for a specific subgroup of individuals instead of the effect for the entire population.

Certain study designs may naturally accomplish this. For example, estimates from most analyses leveraging instrumental variables or similar approaches, including randomized trials, differences-in-differences designs3-5, and regression discontinuity, are interpreted as the effect of treatment on a specific subset of the population. In the case of a randomized controlled trial (RCT), not everyone will adhere to their randomly assigned treatment, and the effect estimates from the RCT tell us about the effects for the compliers. Some people in an RCT may refuse to follow their random assignment. Treatment effects may be different for this group of non-compliers compared to the compliers, but we generally cannot learn about the effect of the assigned treatment for non-compliers from the trial itself. Because these study designs are often used to draw causal inferences, and E4A is particularly focused on funding studies that assess the population health impacts of interventions, HTEs are especially relevant for being clear about to whom study results refer.

#4 Understanding who would be most likely to have the largest benefit.

Resource constraints may preclude treating everyone in many situations. For example, many more people are financially eligible for public housing subsidies than actually receive such subsidies, due to limited funds. Similarly, many low-income households are eligible for home weatherization interventions but limited funds result in long waitlists. Knowing who benefits most could help guide policies to deliver resources in a manner that will yield the largest total population health impact. For some interventions, HTEs may be trivial in magnitude; but for other interventions, positive overall effects may mask substantial differences in impacts by subgroup that, if identified, would alter recommendations for policy or practice.

Learn more about what we know about HTEs, challenges in evaluating HTEs, and open questions for future research in Part 2 of this Methods Notes series.

References

1. Nguyen QC, Rehkopf DH, Schmidt NM, Osypuk TL. Heterogeneous effects of housing vouchers on the mental health of US adolescents. Am J Public Health. 2016;106(4):755-762. doi:10.2105/ AJPH.2015.303006.
2. Vable AM, Canning D, Glymour MM, Kawachi I, Jimenez MP, Subramanian SV. Can social policy influence socioeconomic disparities? Korean War GI Bill eligibility and markers of depression. Ann Epidemiol. 2016;26(2):129-135.e3. doi:10.1016/j.annepidem.2015.12.003.
3. See https://www.evidenceforaction.org/sites/default/ files/Expanding_Prenatal_Care_One-Pager.pdf.
4. See https://www.evidenceforaction.org/physical-activity-and-redesigned-community-spaces-parcs-youth-cohort.
5. See https://www.evidenceforaction.org/community-colleges-and-adult-health.

The E4A Methods Lab was developed to address common methods questions or challenges in Culture of Health research. Our goals are to strengthen the research of E4A grantees and the larger community of population health researchers, to help prospective grantees recognize compelling research opportunities, and to stimulate cross-disciplinary conversation and appreciation across the community of population health researchers.