Who Benefits Most? The Importance of Identifying Whether Social Interventions Have Different Effects for Different People

Introduction

Many social programs and policies may have larger benefits for some people but smaller benefits for other people or may even benefit one group, while harming another. Evidence from the Moving To Opportunity intervention showed that providing subsidized housing vouchers to residents of public housing developments reduced harmful substance use for girls but increased substance use for boys, for example.¹ Differences in the effect of an intervention on different types of individuals are referred to as heterogeneous treatment effects (HTEs). Identifying HTEs is a major focus of health equity research and of particular interest to E4A as we seek to determine what works best, for whom, and under what circumstances.

Example Research Questions

When describing the effect of a policy or program, we are often most interested in the total effect on the population. However, evaluations of whether effects of an intervention differ for particular subsets of the population are important to E4A because they can address four questions:

1. Will the proposed policy or program remediate or exacerbate health disparities? If worse-off groups benefit more from an intervention than better-off groups, then the intervention will remediate health disparities. Conversely, if better-off groups benefit more, then the intervention will exacerbate health disparities.
2. How will the effect of an intervention differ if it is implemented in a new population with a different composition from the population initially studied?
   If the new population is comprised of more people who benefit more, then the effect of the intervention may be larger than in the population initially studied. If the new population includes fewer people that are high need, and are therefore likely to benefit less, then the intervention may be less effective in the new population.
3. What is the effect of an intervention for a specific subgroup of individuals?
   Sometimes, we are specifically interested in the effect of an intervention in a subset of the population, rather than the entire population. For interventions that target particular subgroups, we are most often interested in the effect among those receiving the intervention. For example, the Stockton Basic Income experiment only included households within census tracts at or below the area median income. Stakeholders are most interested in the effect of the program on the subgroup receiving the intervention.
4. Who is most likely to have the largest benefit from an intervention?
   Funding and other resource constraints may limit the ability to reach everyone who could benefit from an intervention. For example, finite funding means that not everyone who is financially eligible for public housing subsidies actually receives them. Similarly, many low-income households are eligible for home weatherization interventions but limited funds result in long waitlists. Knowing who benefits most could help guide policies to deliver resources in a manner that will yield the largest total population health impact.

Possible Approaches

It can be challenging to evaluate HTEs because the research involves examining subsets of the population, resulting in smaller sample sizes and less precision. There are two major types of HTE research: (1) research evaluating whether groups defined by a pre-specified characteristic (e.g., race/ethnicity, income, educational attainment, age) respond differently to an intervention and (2) research evaluating whether there is any heterogeneity in response to an intervention, perhaps across as-yet-unknown characteristics. The vast majority of HTE research falls into the first type, but methods are emerging for the latter type of research.

Putting Evidence/Methods into Practice

High-quality evidence on HTEs can inform decision-making. For example, E4A funded a research project examining the causal effect of a school-based mental health services model on mental health and human capital outcomes for children and adolescents. The researchers are investigating both the overall effect of the program and whether program effectiveness varies for subgroups of children and adolescents at higher risk of mental health problems. Research findings from this project will inform policy and programmatic decisions around expansion of school-based mental health services. To maximize benefits from investments, expansions could prioritize schools with a greater proportion of students who are likely to benefit most.

Tools & Resources

For more on HTEs, read our two-part Methods Note series:
Do social interventions have different effects for different people? Why heterogeneous treatment effects are important in population health research

Why is there so much uncertainty about heterogeneous treatment effects?

We welcome your comments and feedback on these ideas!

References

1. Nguyen QC, Rehkopf DH, Schmidt NM, Osypuk TL. Heterogeneous Effects of Housing Vouchers on the Mental Health of US Adolescents. Am J Public Health. 2016;106(4):755-762. doi:10.2105/AJPH.2015.303006