IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v19y2010i3p350-364.html
   My bibliography  Save this article

Systematic reviews of economic evaluations: utility or futility?

Author

Listed:
  • Rob Anderson

Abstract

Systematic reviews of studies of effectiveness are the centrepiece of evidence‐based medicine and policy making. Increasingly, systematic reviews of economic evaluations are also an expected input into much evidence‐based policy making, with some health economists even calling for ‘an economics approach to systematic review’. This paper questions the value of conducting systematic reviews of economic evaluations to inform decision making in health care. It argues that the value of systematic reviews of economic evaluations is usually undermined by three things. Firstly, compared with effectiveness studies, there is a much wider range of factors that limit the generalisability of cost–effectiveness results, over time and between health systems and service settings, including the context‐dependency of resource use and opportunity costs, and different decision contexts and budget constraints. Secondly, because economic evaluations are more explicitly intended to be decision‐informing, the requirements for generalisability take primacy, and considerations of internal validity become more secondary. Thirdly, since one of the two main forms of economic evaluation – decision analytic modelling – is itself a well‐developed method of evidence synthesis, in most cases the need for a comprehensive systematic review of previous economic evaluations of a particular health technology or policy choice is unwarranted. I conclude that apparent ‘meta‐analytic expectations’ for clear and widely applicable cost–effectiveness conclusions from systematic reviews of economic evaluations are optimistic and generally futile. For more useful insights and knowledge from previous economic studies in evidence‐based policy making, a more limited range of reasons for conducting systematic reviews of health economic studies is proposed. Copyright © 2009 John Wiley & Sons, Ltd.

Suggested Citation

  • Rob Anderson, 2010. "Systematic reviews of economic evaluations: utility or futility?," Health Economics, John Wiley & Sons, Ltd., vol. 19(3), pages 350-364, March.
    • Handle: RePEc:wly:hlthec:v:19:y:2010:i:3:p:350-364
    DOI: 10.1002/hec.1486
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.1486
    Download Restriction: no
    File URL: https://libkey.io/10.1002/hec.1486?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sarah Byford & Tom Sefton, 2003. "Economic Evaluation of Complex Health and Social Care Interventions," National Institute Economic Review, National Institute of Economic and Social Research, vol. 186(1), pages 98-108, October.
    2. Birch, Stephen & Gafni, Amiram, 2003. "Economics and the evaluation of health care programmes: generalisability of methods and implications for generalisability of results," Health Policy, Elsevier, vol. 64(2), pages 207-219, May.
    3. Mark J. Sculpher & Karl Claxton & Mike Drummond & Chris McCabe, 2006. "Whither trial‐based economic evaluation for health care decision making?," Health Economics, John Wiley & Sons, Ltd., vol. 15(7), pages 677-687, July.
    4. N. J. Cooper & A. J. Sutton & A. E. Ades & S. Paisley & D. R. Jones & on behalf of the working group on the ‘use of evidence in economic decision models’, 2007. "Use of evidence in economic decision models: practical issues and methodological challenges," Health Economics, John Wiley & Sons, Ltd., vol. 16(12), pages 1277-1286, December.
    5. N. J. Cooper & A. J. Sutton & A. E. Ades & S. Paisley & D. R. Jones, 2007. "Use of evidence in economic decision models: practical issues and methodological challenges," Health Economics, John Wiley & Sons, Ltd., vol. 16(12), pages 1277-1286.
    6. M. F. Drummond & M. Barbieri & J. B. Wong, 2005. "Analytic Choices in Economic Models of Treatments for Rheumatoid Arthritis: What Makes a Difference?," Medical Decision Making, , vol. 25(5), pages 520-533, September.
    7. Louise B. Russell, 2007. "Is All Cost-Effectiveness Analysis Local?," Medical Decision Making, , vol. 27(3), pages 231-232, May.
    8. Drummond, Michael F. & Sculpher, Mark J. & Torrance, George W. & O'Brien, Bernie J. & Stoddart, Greg L., 2005. "Methods for the Economic Evaluation of Health Care Programmes," OUP Catalogue, Oxford University Press, edition 3, number 9780198529453.
    9. Edward Godber & Ray Robinson & Andrea Steiner, 1997. "Economic Evaluation and the Shifting Balance Towards Primary Care: Definitions, Evidence and Methodological Issues," Health Economics, John Wiley & Sons, Ltd., vol. 6(3), pages 275-294, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. P. Watson & L. Preston & H. Squires & J. Chilcott & A. Brennan, 2014. "Modelling the Economics of Type 2 Diabetes Mellitus Prevention: A Literature Review of Methods," Applied Health Economics and Health Policy, Springer, vol. 12(3), pages 239-253, June.
    2. Nadia Pillai & Mark Dusheiko & Bernard Burnand & Valérie Pittet, 2017. "A systematic review of cost-effectiveness studies comparing conventional, biological and surgical interventions for inflammatory bowel disease," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-22, October.
    3. Bonny Parkinson & Sallie-Anne Pearson & Rosalie Viney, 2014. "Economic evaluations of trastuzumab in HER2-positive metastatic breast cancer: a systematic review and critique," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 15(1), pages 93-112, January.
    4. Lopert, Ruth & Ruiz, Francis & Chalkidou, Kalipso, 2013. "Applying rapid ‘de-facto’ HTA in resource-limited settings: Experience from Romania," Health Policy, Elsevier, vol. 112(3), pages 202-208.
    5. Elisabet Jacobsen & Dwayne Boyers & Alison Avenell, 2020. "Challenges of Systematic Reviews of Economic Evaluations: A Review of Recent Reviews and an Obesity Case Study," PharmacoEconomics, Springer, vol. 38(3), pages 259-267, March.
    6. Chen Min & Mi Xue & Fei Haotian & Li Jialian & Zhang Lingli, 2021. "An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-13, February.
    7. Edward Burn & Alexander D. Liddle & Thomas W. Hamilton & Sunil Pai & Hemant G. Pandit & David W. Murray & Rafael Pinedo-Villanueva, 2017. "Choosing Between Unicompartmental and Total Knee Replacement: What Can Economic Evaluations Tell Us? A Systematic Review," PharmacoEconomics - Open, Springer, vol. 1(4), pages 241-253, December.
    8. Sanjib Saha & Ulf-G Gerdtham & Pia Johansson, 2010. "Economic Evaluation of Lifestyle Interventions for Preventing Diabetes and Cardiovascular Diseases," IJERPH, MDPI, vol. 7(8), pages 1-46, August.
    9. Matt Wilkinson-Stokes & Michelle Tew & Celene Y. L. Yap & Di Crellin & Marie Gerdtz, 2024. "The Economic Impact of Community Paramedics Within Emergency Medical Services: A Systematic Review," Applied Health Economics and Health Policy, Springer, vol. 22(5), pages 665-684, September.
    10. Kalle Hirvonen, 2020. "This is US: Geography of evidence in top health economics journals," Health Economics, John Wiley & Sons, Ltd., vol. 29(10), pages 1316-1323, October.
    11. Tim Mathes & Maren Walgenbach & Sunya-Lee Antoine & Dawid Pieper & Michaela Eikermann, 2014. "Methods for Systematic Reviews of Health Economic Evaluations," Medical Decision Making, , vol. 34(7), pages 826-840, October.
    12. Nora Döring & Susanne Mayer & Finn Rasmussen & Diana Sonntag, 2016. "Economic Evaluation of Obesity Prevention in Early Childhood: Methods, Limitations and Recommendations," IJERPH, MDPI, vol. 13(9), pages 1-11, September.
    13. Frank Pega & Nick Wilson, 2016. "A Systematic Review of Health Economic Analyses of Housing Improvement Interventions and Insecticide-Treated Bednets in the Home," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-29, June.
    14. Garattini, Livio & van de Vooren, Katelijne & Curto, Alessandro, 2015. "Cost-effectiveness of trastuzumab in metastatic breast cancer: Mainly a matter of price in the EU?," Health Policy, Elsevier, vol. 119(2), pages 212-216.
    15. Mandana Zanganeh & Peymane Adab & Bai Li & Emma Frew, 2019. "A Systematic Review of Methods, Study Quality, and Results of Economic Evaluation for Childhood and Adolescent Obesity Intervention," IJERPH, MDPI, vol. 16(3), pages 1-14, February.
    16. Vanessa Scarf & Christine Catling & Rosalie Viney & Caroline Homer, 2016. "Costing Alternative Birth Settings for Women at Low Risk of Complications: A Systematic Review," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-17, February.
    17. Claudine Bommer & Judith Lupatsch & Nicole Bürki & Matthias Schwenkglenks, 2022. "Cost–utility analysis of risk-reducing strategies to prevent breast and ovarian cancer in BRCA-mutation carriers in Switzerland," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 23(5), pages 807-821, July.
    18. Katelijne Vooren & Alessandro Curto & Livio Garattini, 2015. "Pricing of forthcoming therapies for hepatitis C in Europe: beyond cost-effectiveness?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(4), pages 341-345, May.
    19. Rajan Sharma & Yuanyuan Gu & Teresa Y. C. Ching & Vivienne Marnane & Bonny Parkinson, 2019. "Economic Evaluations of Childhood Hearing Loss Screening Programmes: A Systematic Review and Critique," Applied Health Economics and Health Policy, Springer, vol. 17(3), pages 331-357, June.
    20. Mathilda Bongers & Veerle Coupé & Elise Jansma & Egbert Smit & Carin Groot, 2012. "Cost Effectiveness of Treatment with New Agents in Advanced Non-Small-Cell Lung Cancer," PharmacoEconomics, Springer, vol. 30(1), pages 17-34, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brousselle, Astrid & Lessard, Chantale, 2011. "Economic evaluation to inform health care decision-making: Promise, pitfalls and a proposal for an alternative path," Social Science & Medicine, Elsevier, vol. 72(6), pages 832-839, March.
    2. Saha, Sanjib & Gerdtham, Ulf-G. & Toresson, Håkan & Minthon, Lennart & Jarl, Johan, 2018. "Economic Evaluation of Nonpharmacological Interventions for Dementia Patients and their Caregivers - A Systematic Literature Review," Working Papers 2018:10, Lund University, Department of Economics.
    3. Sanjib Saha & Katarina Steen Carlsson & Ulf-G Gerdtham & Margareta K Eriksson & Lars Hagberg & Mats Eliasson & Pia Johansson, 2013. "Are Lifestyle Interventions in Primary Care Cost-Effective? – An Analysis Based on a Markov Model, Differences-In-Differences Approach and the Swedish Björknäs Study," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    4. Zafar Zafari & Kristian Thorlund & J. FitzGerald & Carlo Marra & Mohsen Sadatsafavi, 2014. "Network vs. Pairwise Meta-Analyses: A Case Study of the Impact of an Evidence-Synthesis Paradigm on Value of Information Outcomes," PharmacoEconomics, Springer, vol. 32(10), pages 995-1004, October.
    5. Dyfrig Hughes & Joanna Charles & Dalia Dawoud & Rhiannon Tudor Edwards & Emily Holmes & Carys Jones & Paul Parham & Catrin Plumpton & Colin Ridyard & Huw Lloyd-Williams & Eifiona Wood & Seow Tien Yeo, 2016. "Conducting Economic Evaluations Alongside Randomised Trials: Current Methodological Issues and Novel Approaches," PharmacoEconomics, Springer, vol. 34(5), pages 447-461, May.
    6. Ulla Griffiths & Benedict Anigbogu & Kiran Nanchahal, 2012. "Economic evaluations of adult weight management interventions," Applied Health Economics and Health Policy, Springer, vol. 10(3), pages 145-162, May.
    7. Edward Wilson, 2015. "A Practical Guide to Value of Information Analysis," PharmacoEconomics, Springer, vol. 33(2), pages 105-121, February.
    8. Laura Bojke & Andrea Manca & Miqdad Asaria & Ronan Mahon & Shijie Ren & Stephen Palmer, 2017. "How to Appropriately Extrapolate Costs and Utilities in Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 35(8), pages 767-776, August.
    9. J.- Graf v. d. Schulenburg & Christoph Vauth & Thomas Mittendorf & Wolfgang Greiner, 2007. "Methods for determining cost-benefit ratios for pharmaceuticals in Germany," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 8(1), pages 5-31, September.
    10. Daisuke Goto & Ya-Chen Tina Shih & Pascal Lecomte & Melvin Olson & Chukwukadibia Udeze & Yujin Park & C. Daniel Mullins, 2017. "Regression-Based Approaches to Patient-Centered Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 35(7), pages 685-695, July.
    11. Ka Keat Lim & Rositsa Koleva-Kolarova & Julia Fox-Rushby, 2022. "A Comparison of the Content and Consistency of Methodological Quality and Transferability Checklists for Reviewing Model-Based Economic Evaluations," PharmacoEconomics, Springer, vol. 40(10), pages 989-1003, October.
    12. Simon French, 2012. "Expert Judgment, Meta-analysis, and Participatory Risk Analysis," Decision Analysis, INFORMS, vol. 9(2), pages 119-127, June.
    13. Zhi Qu & Shanshan Zhang & Christian Krauth & Xuenan Liu, 2019. "A systematic review of decision analytic modeling techniques for the economic evaluation of dental caries interventions," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-17, May.
    14. Saha, Sanjib & Gerdtham, Ulf-G. & Toresson, Håkan & Minthon, Lennart & Jarl, Johan, 2018. "Economic Evaluation of Management of Dementia Patients - A Systematic Literature Review," Working Papers 2018:41, Lund University, Department of Economics.
    15. Ulla Kou Griffiths & Rosa Legood & Catherine Pitt, 2016. "Comparison of Economic Evaluation Methods Across Low‐income, Middle‐income and High‐income Countries: What are the Differences and Why?," Health Economics, John Wiley & Sons, Ltd., vol. 25(S1), pages 29-41, February.
    16. Saha, Sanjib & Gerdtham, Ulf-G. & Toresson, Håkan & Minthon, Lennart & Jarl, Johan, 2018. "Economic Evaluation of Interventions for Screening of Dementia," Working Papers 2018:20, Lund University, Department of Economics.
    17. Mark Oppe & Daniela Ortín-Sulbarán & Carlos Vila Silván & Anabel Estévez-Carrillo & Juan M. Ramos-Goñi, 2021. "Cost-effectiveness of adding Sativex® spray to spasticity care in Belgium: using bootstrapping instead of Monte Carlo simulation for probabilistic sensitivity analyses," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 22(5), pages 711-721, July.
    18. Ties Hoomans & Johan Severens & Nicole Roer & Gepke Delwel, 2012. "Methodological Quality of Economic Evaluations of New Pharmaceuticals in the Netherlands," PharmacoEconomics, Springer, vol. 30(3), pages 219-227, March.
    19. Khan, Md. Tajuddin & Kishore, Avinash & Joshi, Pramod Kumar, 2016. "Gender dimensions on farmers’ preferences for direct-seeded rice with drum seeder in India:," IFPRI discussion papers 1550, International Food Policy Research Institute (IFPRI).
    20. Jose L Burgos & Thomas L Patterson & Joshua S Graff-Zivin & James G Kahn & M Gudelia Rangel & M Remedios Lozada & Hugo Staines & Steffanie A Strathdee, 2016. "Cost-Effectiveness of Combined Sexual and Injection Risk Reduction Interventions among Female Sex Workers Who Inject Drugs in Two Very Distinct Mexican Border Cities," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-15, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:19:y:2010:i:3:p:350-364. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.