IDEAS home Printed from https://ideas.repec.org/a/spr/pharme/v38y2020i11d10.1007_s40273-020-00946-y.html
   My bibliography  Save this article

Taking the Analysis of Trial-Based Economic Evaluations to the Next Level: The Importance of Accounting for Clustering

Author

Listed:
  • Mohamed El Alili

    (Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute)

  • Johanna M. Dongen

    (Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute
    Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute)

  • Keith S. Goldfeld

    (NYU School of Medicine)

  • Martijn W. Heymans

    (Amsterdam UMC, Location VU, Amsterdam Public Health Research Institute)

  • Maurits W. Tulder

    (Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute
    Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute
    Aarhus University Hospital)

  • Judith E. Bosmans

    (Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute)

Abstract

Objectives The aim of this study was to assess the performance and impact of multilevel modelling (MLM) compared with ordinary least squares (OLS) regression in trial-based economic evaluations with clustered data. Methods Three thousand datasets with balanced and unbalanced clusters were simulated with correlation coefficients between costs and effects of − 0.5, 0, and 0.5, and intraclass correlation coefficients (ICCs) varying between 0.05 and 0.30. Each scenario was analyzed using both MLM and OLS. Statistical uncertainty around MLM and OLS estimates was estimated using bootstrapping. Performance measures were estimated and compared between approaches, including bias, root mean squared error (RMSE) and coverage probability. Cost and effect differences, and their corresponding confidence intervals and standard errors, incremental cost-effectiveness ratios, incremental net-monetary benefits and cost-effectiveness acceptability curves were compared. Results Cost-effectiveness outcomes were similar between OLS and MLM. MLM produced larger statistical uncertainty and coverage probabilities closer to nominal levels than OLS. The higher the ICC, the larger the effect on statistical uncertainty between MLM and OLS. Significant cost-effectiveness outcomes as estimated by OLS became non-significant when estimated by MLM. At all ICCs, MLM resulted in lower probabilities of cost effectiveness than OLS, and this difference became larger with increasing ICCs. Performance measures and cost-effectiveness outcomes were similar across scenarios with varying correlation coefficients between costs and effects. Conclusions Although OLS produced similar cost-effectiveness outcomes, it substantially underestimated the amount of variation in the data compared with MLM. To prevent suboptimal conclusions and a possible waste of scarce resources, it is important to use MLM in trial-based economic evaluations when data are clustered.

Suggested Citation

  • Mohamed El Alili & Johanna M. Dongen & Keith S. Goldfeld & Martijn W. Heymans & Maurits W. Tulder & Judith E. Bosmans, 2020. "Taking the Analysis of Trial-Based Economic Evaluations to the Next Level: The Importance of Accounting for Clustering," PharmacoEconomics, Springer, vol. 38(11), pages 1247-1261, November.
    • Handle: RePEc:spr:pharme:v:38:y:2020:i:11:d:10.1007_s40273-020-00946-y
    DOI: 10.1007/s40273-020-00946-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40273-020-00946-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40273-020-00946-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Andrew Briggs & Richard Nixon & Simon Dixon & Simon Thompson, 2005. "Parametric modelling of cost data: some simulation evidence," Health Economics, John Wiley & Sons, Ltd., vol. 14(4), pages 421-428, April.
    2. Anthony O'Hagan & John W. Stevens, 2003. "Assessing and comparing costs: how robust are the bootstrap and methods based on asymptotic normality?," Health Economics, John Wiley & Sons, Ltd., vol. 12(1), pages 33-49, January.
    3. Elisabeth Fenwick & Bernie J. O'Brien & Andrew Briggs, 2004. "Cost‐effectiveness acceptability curves – facts, fallacies and frequently asked questions," Health Economics, John Wiley & Sons, Ltd., vol. 13(5), pages 405-415, May.
    4. Borislava Mihaylova & Andrew Briggs & Anthony O'Hagan & Simon G. Thompson, 2011. "Review of statistical methods for analysing healthcare resources and costs," Health Economics, John Wiley & Sons, Ltd., vol. 20(8), pages 897-916, August.
    5. Drummond, Michael F. & Sculpher, Mark J. & Claxton, Karl & Stoddart, Greg L. & Torrance, George W., 2015. "Methods for the Economic Evaluation of Health Care Programmes," OUP Catalogue, Oxford University Press, edition 4, number 9780199665884.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ângela Jornada Ben & Johanna M. Dongen & Mohamed El Alili & Martijn W. Heymans & Jos W. R. Twisk & Janet L. MacNeil-Vroomen & Maartje Wit & Susan E. M. Dijk & Teddy Oosterhuis & Judith E. Bosmans, 2023. "The handling of missing data in trial-based economic evaluations: should data be multiply imputed prior to longitudinal linear mixed-model analyses?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(6), pages 951-965, August.
    2. Thomas Lung & Lei Si & Richard Hooper & Gian Luca Di Tanna, 2021. "Health Economic Evaluation Alongside Stepped Wedge Trials: A Methodological Systematic Review," PharmacoEconomics, Springer, vol. 39(1), pages 63-80, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Richard M. Nixon & David Wonderling & Richard D. Grieve, 2010. "Non‐parametric methods for cost‐effectiveness analysis: the central limit theorem and the bootstrap compared," Health Economics, John Wiley & Sons, Ltd., vol. 19(3), pages 316-333, March.
    2. Salah Ghabri & Françoise F. Hamers & Jean Michel Josselin, 2016. "Exploring Uncertainty in Economic Evaluations of Drugs and Medical Devices: Lessons from the First Review of Manufacturers’ Submissions to the French National Authority for Health," PharmacoEconomics, Springer, vol. 34(6), pages 617-624, June.
    3. Manuel Gomes & Richard Grieve & Richard Nixon & W. J. Edmunds, 2012. "Statistical Methods for Cost-Effectiveness Analyses That Use Data from Cluster Randomized Trials," Medical Decision Making, , vol. 32(1), pages 209-220, January.
    4. Baptiste Leurent & Manuel Gomes & Rita Faria & Stephen Morris & Richard Grieve & James R. Carpenter, 2018. "Sensitivity Analysis for Not-at-Random Missing Data in Trial-Based Cost-Effectiveness Analysis: A Tutorial," PharmacoEconomics, Springer, vol. 36(8), pages 889-901, August.
    5. Mohamed El Alili & Johanna M. van Dongen & Jonas L. Esser & Martijn W. Heymans & Maurits W. van Tulder & Judith E. Bosmans, 2022. "A scoping review of statistical methods for trial‐based economic evaluations: The current state of play," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2680-2699, December.
    6. Andrew R. Willan & Matthew E. Kowgier, 2008. "Cost‐effectiveness analysis of a multinational RCT with a binary measure of effectiveness and an interacting covariate," Health Economics, John Wiley & Sons, Ltd., vol. 17(7), pages 777-791, July.
    7. Christoph F. Kurz & Michael Laxy, 2020. "Application of Mendelian Randomization to Investigate the Association of Body Mass Index with Health Care Costs," Medical Decision Making, , vol. 40(2), pages 156-169, February.
    8. Borislava Mihaylova & Andrew Briggs & Anthony O'Hagan & Simon G. Thompson, 2011. "Review of statistical methods for analysing healthcare resources and costs," Health Economics, John Wiley & Sons, Ltd., vol. 20(8), pages 897-916, August.
    9. Bing Wang & Renee Santoreneos & Hossein Afzali & Lynne Giles & Helen Marshall, 2018. "Costs of Invasive Meningococcal Disease: A Global Systematic Review," PharmacoEconomics, Springer, vol. 36(10), pages 1201-1222, October.
    10. Claudia Geue & James Lewsey & Paula Lorgelly & Lindsay Govan & Carole Hart & Andrew Briggs, 2012. "Spoilt For Choice: Implications Of Using Alternative Methods Of Costing Hospital Episode Statistics," Health Economics, John Wiley & Sons, Ltd., vol. 21(10), pages 1201-1216, October.
    11. Matthew Franklin & James Lomas & Simon Walker & Tracey Young, 2019. "An Educational Review About Using Cost Data for the Purpose of Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 37(5), pages 631-643, May.
    12. Anne Prenzler & Bernd Bokemeyer & J.-Matthias Schulenburg & Thomas Mittendorf, 2011. "Health care costs and their predictors of inflammatory bowel diseases in Germany," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 12(3), pages 273-283, June.
    13. Mohamed El Alili & Johanna M. Dongen & Judith A. F. Huirne & Maurits W. Tulder & Judith E. Bosmans, 2017. "Reporting and Analysis of Trial-Based Cost-Effectiveness Evaluations in Obstetrics and Gynaecology," PharmacoEconomics, Springer, vol. 35(10), pages 1007-1033, October.
    14. Manuel Gomes & Edmond S.-W. Ng & Richard Grieve & Richard Nixon & James Carpenter & Simon G. Thompson, 2012. "Developing Appropriate Methods for Cost-Effectiveness Analysis of Cluster Randomized Trials," Medical Decision Making, , vol. 32(2), pages 350-361, March.
    15. Bebu, Ionut & Luta, George & Mathew, Thomas & Kennedy, Paul A. & Agan, Brian K., 2016. "Parametric cost-effectiveness inference with skewed data," Computational Statistics & Data Analysis, Elsevier, vol. 94(C), pages 210-220.
    16. Caterina Conigliani & Andrea Tancredi, 2009. "A Bayesian model averaging approach for cost‐effectiveness analyses," Health Economics, John Wiley & Sons, Ltd., vol. 18(7), pages 807-821, July.
    17. Noemi Kreif & Richard Grieve & Rosalba Radice & Zia Sadique & Roland Ramsahai & Jasjeet S. Sekhon, 2012. "Methods for Estimating Subgroup Effects in Cost-Effectiveness Analyses That Use Observational Data," Medical Decision Making, , vol. 32(6), pages 750-763, November.
    18. Irina Pokhilenko & Luca M. M. Janssen & Aggie T. G. Paulus & Ruben M. W. A. Drost & William Hollingworth & Joanna C. Thorn & Sian Noble & Judit Simon & Claudia Fischer & Susanne Mayer & Luis Salvador-, 2023. "Development of an Instrument for the Assessment of Health-Related Multi-sectoral Resource Use in Europe: The PECUNIA RUM," Applied Health Economics and Health Policy, Springer, vol. 21(2), pages 155-166, March.
    19. Chiranjeev Sanyal & Don Husereau, 2020. "Systematic Review of Economic Evaluations of Services Provided by Community Pharmacists," Applied Health Economics and Health Policy, Springer, vol. 18(3), pages 375-392, June.
    20. Majmudar, Ishani Kartik & Mihalopoulos, Cathy & Abimanyi-Ochom, Julie & Mohebbi, Mohammadreza & Engel, Lidia, 2024. "The association between loneliness with health service use and quality of life among informal carers in Australia," Social Science & Medicine, Elsevier, vol. 348(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:pharme:v:38:y:2020:i:11:d:10.1007_s40273-020-00946-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.