IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0121357.html
   My bibliography  Save this article

External Validation and Calibration of IVFpredict: A National Prospective Cohort Study of 130,960 In Vitro Fertilisation Cycles

Author

Listed:
  • Andrew D A C Smith
  • Kate Tilling
  • Debbie A Lawlor
  • Scott M Nelson

Abstract

Background: Accurately predicting the probability of a live birth after in vitro fertilisation (IVF) is important for patients, healthcare providers and policy makers. Two prediction models (Templeton and IVFpredict) have been previously developed from UK data and are widely used internationally. The more recent of these, IVFpredict, was shown to have greater predictive power in the development dataset. The aim of this study was external validation of the two models and comparison of their predictive ability. Methods and Findings: 130,960 IVF cycles undertaken in the UK in 2008–2010 were used to validate and compare the Templeton and IVFpredict models. Discriminatory power was calculated using the area under the receiver-operator curve and calibration assessed using a calibration plot and Hosmer-Lemeshow statistic. The scaled modified Brier score, with measures of reliability and resolution, were calculated to assess overall accuracy. Both models were compared after updating for current live birth rates to ensure that the average observed and predicted live birth rates were equal. The discriminative power of both methods was comparable: the area under the receiver-operator curve was 0.628 (95% confidence interval (CI): 0.625–0.631) for IVFpredict and 0.616 (95% CI: 0.613–0.620) for the Templeton model. IVFpredict had markedly better calibration and higher diagnostic accuracy, with calibration plot intercept of 0.040 (95% CI: 0.017–0.063) and slope of 0.932 (95% CI: 0.839–1.025) compared with 0.080 (95% CI: 0.044–0.117) and 1.419 (95% CI: 1.149–1.690) for the Templeton model. Both models underestimated the live birth rate, but this was particularly marked in the Templeton model. Updating the models to reflect improvements in live birth rates since the models were developed enhanced their performance, but IVFpredict remained superior. Conclusion: External validation in a large population cohort confirms IVFpredict has superior discrimination and calibration for informing patients, clinicians and healthcare policy makers of the probability of live birth following IVF.

Suggested Citation

  • Andrew D A C Smith & Kate Tilling & Debbie A Lawlor & Scott M Nelson, 2015. "External Validation and Calibration of IVFpredict: A National Prospective Cohort Study of 130,960 In Vitro Fertilisation Cycles," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-15, April.
    • Handle: RePEc:plo:pone00:0121357
    DOI: 10.1371/journal.pone.0121357
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0121357
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0121357&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0121357?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ewout W Steyerberg & Karel G M Moons & Danielle A van der Windt & Jill A Hayden & Pablo Perel & Sara Schroter & Richard D Riley & Harry Hemingway & Douglas G Altman & for the PROGRESS Group, 2013. "Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research," PLOS Medicine, Public Library of Science, vol. 10(2), pages 1-9, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alwin Schierenberg & Margaretha C Minnaard & Rogier M Hopstaken & Alma C van de Pol & Berna D L Broekhuizen & Niek J de Wit & Johannes B Reitsma & Saskia F van Vugt & Aleida W Graffelman & Hasse Melby, 2016. "External Validation of Prediction Models for Pneumonia in Primary Care Patients with Lower Respiratory Tract Infection: An Individual Patient Data Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-16, February.
    2. Nath Adulkasem & Phichayut Phinyo & Jiraporn Khorana & Dumnoensun Pruksakorn & Theerachai Apivatthakakul, 2021. "Development of Clinical Prediction Rules for One-Year Postoperative Functional Outcome in Patients with Intertrochanteric Fractures: The Intertrochanteric Fracture Ambulatory Prediction (IT-AP) Tool," IJERPH, MDPI, vol. 19(1), pages 1-16, December.
    3. Julius Sim & Lucy Teece & Martin S Dennis & Christine Roffe & SOࠢS Study Team, 2016. "Validation and Recalibration of Two Multivariable Prognostic Models for Survival and Independence in Acute Stroke," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-17, May.
    4. Phung Khanh Lam & Dong Thi Hoai Tam & Nguyen Minh Dung & Nguyen Thi Hanh Tien & Nguyen Tan Thanh Kieu & Cameron Simmons & Jeremy Farrar & Bridget Wills & Marcel Wolbers, 2015. "A Prognostic Model for Development of Profound Shock among Children Presenting with Dengue Shock Syndrome," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-13, May.
    5. Mario Dioguardi & Francesca Spirito & Diego Sovereto & Mario Alovisi & Giuseppe Troiano & Riccardo Aiuto & Daniele Garcovich & Vito Crincoli & Luigi Laino & Angela Pia Cazzolla & Giorgia Apollonia Cal, 2022. "MicroRNA-21 Expression as a Prognostic Biomarker in Oral Cancer: Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 19(6), pages 1-12, March.
    6. Catherine Beauregard-Paultre & Claire Nour Abou Chakra & Allison McGeer & Annie-Claude Labbé & Andrew E Simor & Wayne Gold & Matthew P Muller & Jeff Powis & Kevin Katz & Suzanne M Cadarette & Jacques , 2019. "External validation of clinical prediction rules for complications and mortality following Clostridioides difficile infection," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-15, December.
    7. Stephana J Cherak & Andrea Soo & Kyla N Brown & E Wesley Ely & Henry T Stelfox & Kirsten M Fiest, 2020. "Development and validation of delirium prediction model for critically ill adults parameterized to ICU admission acuity," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-18, August.
    8. Igor O Korolev & Laura L Symonds & Andrea C Bozoki & Alzheimer's Disease Neuroimaging Initiative, 2016. "Predicting Progression from Mild Cognitive Impairment to Alzheimer's Dementia Using Clinical, MRI, and Plasma Biomarkers via Probabilistic Pattern Classification," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-25, February.
    9. Jia You & Yu Guo & Yi Zhang & Ju-Jiao Kang & Lin-Bo Wang & Jian-Feng Feng & Wei Cheng & Jin-Tai Yu, 2023. "Plasma proteomic profiles predict individual future health risk," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    10. Jakob Steinfeldt & Benjamin Wild & Thore Buergel & Maik Pietzner & Julius Upmeier zu Belzen & Andre Vauvelle & Stefan Hegselmann & Spiros Denaxas & Harry Hemingway & Claudia Langenberg & Ulf Landmesse, 2024. "RETRACTED ARTICLE: Medical history predicts phenome-wide disease onset and enables the rapid response to emerging health threats," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    11. Todd J. Levy & Kevin Coppa & Jinxuan Cang & Douglas P. Barnaby & Marc D. Paradis & Stuart L. Cohen & Alex Makhnevich & David Klaveren & David M. Kent & Karina W. Davidson & Jamie S. Hirsch & Theodoros, 2022. "Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    12. Jiakun Jiang & Wei Yang & Erin M. Schnellinger & Stephen E. Kimmel & Wensheng Guo, 2023. "Dynamic logistic state space prediction model for clinical decision making," Biometrics, The International Biometric Society, vol. 79(1), pages 73-85, March.
    13. François Luthi & Olivier Deriaz & Philippe Vuistiner & Cyrille Burrus & Roger Hilfiker, 2014. "Predicting Non Return to Work after Orthopaedic Trauma: The Wallis Occupational Rehabilitation RisK (WORRK) Model," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-11, April.
    14. Daan Nieboer & Tjeerd van der Ploeg & Ewout W Steyerberg, 2016. "Assessing Discriminative Performance at External Validation of Clinical Prediction Models," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-10, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0121357. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.