IDEAS home Printed from https://ideas.repec.org/a/spr/pharme/v37y2019i11d10.1007_s40273-019-00836-y.html
   My bibliography  Save this article

Can You Repeat That? Exploring the Definition of a Successful Model Replication in Health Economics

Author

Listed:
  • Emma McManus

    (University of East Anglia)

  • David Turner

    (University of East Anglia)

  • Tracey Sach

    (University of East Anglia)

Abstract

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) modelling taskforce suggests decision models should be thoroughly reported and transparent. However, the level of transparency and indeed how transparency should be assessed are yet to be defined. One way may be to attempt to replicate the model and its outputs. The ability to replicate a decision model could demonstrate adequate reporting transparency. This review aims to explore published definitions of replication success across all scientific disciplines and to consider how such a definition should be tailored for use in health economic models. A literature review was conducted to identify published definitions of a ‘successful replication’. Using these as a foundation, several definitions of replication success were constructed, to be applicable to replications of economic decision models, with the associated strengths and weaknesses of such definitions discussed. A substantial body of literature discussing replicability was found; however, relatively few studies, ten, explicitly defined a successful replication. These definitions varied from subjective assessments to expecting exactly the same results to be reproduced. Whilst the definitions that have been found may help to construct a definition specific to health economics, no definition was found that completely encompassed the unique requirements for decision models. Replication is widely discussed in other scientific disciplines; however, as of yet, there is no consensus on how replicable models should be within health economics or what constitutes a successful replication. Replication studies can demonstrate how transparently a model is reported, identify potential calculation errors and inform future reporting practices. It may therefore be a useful adjunct to other transparency or quality measures.

Suggested Citation

  • Emma McManus & David Turner & Tracey Sach, 2019. "Can You Repeat That? Exploring the Definition of a Successful Model Replication in Health Economics," PharmacoEconomics, Springer, vol. 37(11), pages 1371-1381, November.
    • Handle: RePEc:spr:pharme:v:37:y:2019:i:11:d:10.1007_s40273-019-00836-y
    DOI: 10.1007/s40273-019-00836-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40273-019-00836-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40273-019-00836-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jan H. Höffler, 2017. "Replication and Economics Journal Policies," American Economic Review, American Economic Association, vol. 107(5), pages 52-55, May.
    2. Christopher James Sampson & Tim Wrightson, 2017. "Model Registration: A Call to Action," PharmacoEconomics - Open, Springer, vol. 1(2), pages 73-77, June.
    3. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    4. Don Husereau & Michael Drummond & Stavros Petrou & Chris Carswell & David Moher & Dan Greenberg & Federico Augustovski & Andrew Briggs & Josephine Mauskopf & Elizabeth Loder, 2013. "Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement," PharmacoEconomics, Springer, vol. 31(5), pages 361-367, May.
    5. Maren Duvendack & Richard W. Palmer-Jones & W. Robert Reed, 2015. "Replications in Economics: A Progress Report," Econ Journal Watch, Econ Journal Watch, vol. 12(2), pages 164–191-1, May.
    6. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    7. Chang, Andrew C., 2017. "A replication recipe: List your ingredients before you start cooking," Economics Discussion Papers 2017-74, Kiel Institute for the World Economy (IfW Kiel).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paul Tappenden & J. Jaime Caro, 2019. "Improving Transparency in Decision Models: Current Issues and Potential Solutions," PharmacoEconomics, Springer, vol. 37(11), pages 1303-1304, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucas C. Coffman & Muriel Niederle & Alistair J. Wilson, 2017. "A Proposal to Organize and Promote Replications," American Economic Review, American Economic Association, vol. 107(5), pages 41-45, May.
    2. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    3. Jan H. Höffler, 2020. "Making replicability the norm starting with oneself and depersonalizing research debates," Replication Working Papers 2/2020, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
    4. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    5. Benjamin D K Wood & Rui Müller & Annette N Brown, 2018. "Push button replication: Is impact evaluation evidence for international development verifiable?," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-15, December.
    6. Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    7. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    8. Daniels, Gerald Eric & Kakar, Venoo, 2018. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (2007)," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-11.
    9. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    10. Tom Coupé & W. Robert Reed & Christian Zimmerman, 2021. "Paving the Road for Replications: Experimental Results from an Online Research Repository," Working Papers in Economics 21/09, University of Canterbury, Department of Economics and Finance.
    11. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    12. Frank Mueller-Langer & Benedikt Fecher & Dietmar Harhoff & Gert G. Wagner, 2017. "The Economics of Replication," Discussion Papers of DIW Berlin 1640, DIW Berlin, German Institute for Economic Research.
    13. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
    14. Hirschauer Norbert & Grüner Sven & Mußhoff Oliver & Becker Claudia, 2019. "Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in Econometrics," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 239(4), pages 703-721, August.
    15. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    16. Tom Coupé & W. Robert Reed & Christian Zimmermann, 2021. "Paving the Road for Replications: Experimental Results from an Online Research Bibliography," Working Papers 2021-013, Federal Reserve Bank of St. Louis, revised 24 Feb 2022.
    17. Kenneth Button, 2020. "Studying the empirical implications of the liberalization of airport markets," Competition and Regulation in Network Industries, , vol. 21(3), pages 223-243, September.
    18. Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.
    19. Giovanna Elisa Calabrò & Sara Boccalini & Donatella Panatto & Caterina Rizzo & Maria Luisa Di Pietro & Fasika Molla Abreha & Marco Ajelli & Daniela Amicizia & Angela Bechini & Irene Giacchetta & Piero, 2022. "The New Quadrivalent Adjuvanted Influenza Vaccine for the Italian Elderly: A Health Technology Assessment," IJERPH, MDPI, vol. 19(7), pages 1-14, March.
    20. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:pharme:v:37:y:2019:i:11:d:10.1007_s40273-019-00836-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.