IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v34y2025i1p85-104.html
   My bibliography  Save this article

Going beyond randomised controlled trials to assess treatment effect heterogeneity across target populations

Author

Listed:
  • David G. Lugo‐Palacios
  • Patrick Bidulka
  • Stephen O’Neill
  • Orlagh Carroll
  • Anirban Basu
  • Amanda Adler
  • Karla DíazOrdaz
  • Andrew Briggs
  • Richard Grieve

Abstract

Methods have been developed for transporting evidence from randomised controlled trials (RCTs) to target populations. However, these approaches allow only for differences in characteristics observed in the RCT and real‐world data (overt heterogeneity). These approaches do not recognise heterogeneity of treatment effects (HTE) according to unmeasured characteristics (essential heterogeneity). We use a target trial design and apply a local instrumental variable (LIV) approach to electronic health records from the Clinical Practice Research Datalink, and examine both forms of heterogeneity in assessing the comparative effectiveness of two second‐line treatments for type 2 diabetes mellitus. We first estimate individualised estimates of HTE across the entire target population defined by applying eligibility criteria from national guidelines (n = 13,240) within an overall target trial framework. We define a subpopulation who meet a published RCT's eligibility criteria (‘RCT‐eligible’, n = 6497), and a subpopulation who do not (‘RCT‐ineligible’, n = 6743). We compare average treatment effects for pre‐specified subgroups within the RCT‐eligible subpopulation, the RCT‐ineligible subpopulation, and within the overall target population. We find differences across these subpopulations in the magnitude of subgroup‐level treatment effects, but that the direction of estimated effects is stable. Our results highlight that LIV methods can provide useful evidence about treatment effect heterogeneity including for those subpopulations excluded from RCTs.

Suggested Citation

  • David G. Lugo‐Palacios & Patrick Bidulka & Stephen O’Neill & Orlagh Carroll & Anirban Basu & Amanda Adler & Karla DíazOrdaz & Andrew Briggs & Richard Grieve, 2025. "Going beyond randomised controlled trials to assess treatment effect heterogeneity across target populations," Health Economics, John Wiley & Sons, Ltd., vol. 34(1), pages 85-104, January.
  • Handle: RePEc:wly:hlthec:v:34:y:2025:i:1:p:85-104
    DOI: 10.1002/hec.4903
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.4903
    Download Restriction: no

    File URL: https://libkey.io/10.1002/hec.4903?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Achim Ahrens & Christian B. Hansen & Mark E. Schaffer, 2020. "lassopack: Model selection and prediction with regularized regression in Stata," Stata Journal, StataCorp LP, vol. 20(1), pages 176-235, March.
    2. Thomas Cornelissen & Christian Dustmann & Anna Raute & Uta Schönberg, 2018. "Who Benefits from Universal Child Care? Estimating Marginal Returns to Early Child Care Attendance," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2356-2409.
    3. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    4. Silvia Moler‐Zapata & Richard Grieve & Anirban Basu & Stephen O’Neill, 2023. "How does a local instrumental variable method perform across settings with instruments of differing strengths? A simulation study and an evaluation of emergency surgery," Health Economics, John Wiley & Sons, Ltd., vol. 32(9), pages 2113-2126, September.
    5. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    6. Douglas Staiger & James H. Stock, 1997. "Instrumental Variables Regression with Weak Instruments," Econometrica, Econometric Society, vol. 65(3), pages 557-586, May.
    7. Edward Vytlacil & James J. Heckman, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May.
    8. Adrian Gheorghe & Tracy E Roberts & Jonathan C Ives & Benjamin R Fletcher & Melanie Calvert, 2013. "Centre Selection for Clinical Trials and the Generalisability of Results: A Mixed Methods Study," PLOS ONE, Public Library of Science, vol. 8(2), pages 1-9, February.
    9. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    10. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    11. Anirban Basu, 2015. "Person-centered treatment (PeT) effects: Individualized treatment effects using instrumental variables," Stata Journal, StataCorp LP, vol. 15(2), pages 397-410, June.
    12. Anirban Basu & Norma B. Coe & Cole G. Chapman, 2018. "2SLS versus 2SRI: Appropriate methods for rare outcomes and/or rare exposures," Health Economics, John Wiley & Sons, Ltd., vol. 27(6), pages 937-955, June.
    13. Anirban Basu, 2014. "ESTIMATING PERSON‐CENTERED TREATMENT (PeT) EFFECTS USING INSTRUMENTAL VARIABLES: AN APPLICATION TO EVALUATING PROSTATE CANCER TREATMENTS," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 671-691, June.
    14. Erin Hartman & Richard Grieve & Roland Ramsahai & Jasjeet S. Sekhon, 2015. "From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 757-778, June.
    15. Tan, Zhiqiang, 2006. "Regression and Weighting Methods for Causal Inference Using Instrumental Variables," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1607-1618, December.
    16. Wooldridge, Jeffrey M., 2007. "Inverse probability weighted estimation for general missing data problems," Journal of Econometrics, Elsevier, vol. 141(2), pages 1281-1301, December.
    17. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    18. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: An application in breast cancer patients," Health, Econometrics and Data Group (HEDG) Working Papers 07/07, HEDG, c/o Department of Economics, University of York.
    19. Kosuke Imai & Gary King & Elizabeth A. Stuart, 2008. "Misunderstandings between experimentalists and observationalists about causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(2), pages 481-502, April.
    20. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    21. Daryl Pregibon, 1980. "Goodness of Link Tests for Generalized Linear Models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 29(1), pages 15-24, March.
    22. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    23. Manuel Gomes & Nick Latimer & Marta Soares & Sofia Dias & Gianluca Baio & Nick Freemantle & Dalia Dawoud & Allan Wailoo & Richard Grieve, 2022. "Target Trial Emulation for Transparent and Robust Estimation of Treatment Effects for Health Technology Assessment Using Real-World Data: Opportunities and Challenges," PharmacoEconomics, Springer, vol. 40(6), pages 577-586, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Silvia Moler‐Zapata & Richard Grieve & Anirban Basu & Stephen O’Neill, 2023. "How does a local instrumental variable method perform across settings with instruments of differing strengths? A simulation study and an evaluation of emergency surgery," Health Economics, John Wiley & Sons, Ltd., vol. 32(9), pages 2113-2126, September.
    2. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    3. Tafti, Elena Ashtari, 2023. "Technology, Skills, and Performance: The Case of Robots in Surgery," CINCH Working Paper Series (since 2020) 78746, Duisburg-Essen University Library, DuEPublico.
    4. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    5. Akabayashi, Hideo & Ruberg, Tim & Shikishima, Chizuru & Yamashita, Jun, 2023. "Education-oriented and care-oriented preschools: Implications on child development," Labour Economics, Elsevier, vol. 84(C).
    6. Péron, M.; & Dormont, B.;, 2018. "Heterogeneous moral hazard in Supplementary Health Insurance," Health, Econometrics and Data Group (HEDG) Working Papers 18/27, HEDG, c/o Department of Economics, University of York.
    7. Elena Ashtari Tafti, 2022. "Technology, skills, and performance: the case of robots in surgery," IFS Working Papers W22/46, Institute for Fiscal Studies.
    8. Robert A. Moffitt & Matthew V. Zahn, 2019. "The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation," NBER Working Papers 26028, National Bureau of Economic Research, Inc.
    9. Daniel A Kamhöfer & Hendrik Schmitz & Matthias Westphal, 2019. "Heterogeneity in Marginal Non-Monetary Returns to Higher Education," Journal of the European Economic Association, European Economic Association, vol. 17(1), pages 205-244.
    10. Gerten, Elisa & Beckmann, Michael & Kräkel, Matthias, 2022. "Information and Communication Technology, Hierarchy, and Job Design," IZA Discussion Papers 15491, Institute of Labor Economics (IZA).
    11. Bartalotti, Otávio & Kédagni, Désiré & Possebom, Vitor, 2023. "Identifying marginal treatment effects in the presence of sample selection," Journal of Econometrics, Elsevier, vol. 234(2), pages 565-584.
    12. Olivier De Groote & Koen Declercq, 2021. "Tracking and specialization of high schools: Heterogeneous effects of school choice," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(7), pages 898-916, November.
    13. Mogstad, Magne & Torgovitsky, Alexander & Walters, Christopher R., 2024. "Policy evaluation with multiple instrumental variables," Journal of Econometrics, Elsevier, vol. 243(1).
    14. Pedro Carneiro & Michael Lokshin & Nithin Umapathi, 2017. "Average and Marginal Returns to Upper Secondary Schooling in Indonesia," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 16-36, January.
    15. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.
    16. Tobias Klein, 2013. "College education and wages in the U.K.: estimating conditional average structural functions in nonadditive models with binary endogenous variables," Empirical Economics, Springer, vol. 44(1), pages 135-161, February.
    17. Matthias Westphal & Daniel A Kamhöfer & Hendrik Schmitz, 2022. "Marginal College Wage Premiums Under Selection Into Employment," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2231-2272.
    18. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    19. Lina Zhang & David T. Frazier & D. S. Poskitt & Xueyan Zhao, 2020. "Decomposing Identification Gains and Evaluating Instrument Identification Power for Partially Identified Average Treatment Effects," Papers 2009.02642, arXiv.org, revised Sep 2022.
    20. Sasaki, Yuya & Ura, Takuya, 2023. "Estimation and inference for policy relevant treatment effects," Journal of Econometrics, Elsevier, vol. 234(2), pages 394-450.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:34:y:2025:i:1:p:85-104. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.