IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-50601-9.html
   My bibliography  Save this article

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Author

Listed:
  • Sarabeth M. Mathis

    (Centers for Disease Control and Prevention)

  • Alexander E. Webber

    (Centers for Disease Control and Prevention)

  • Tomás M. León

    (California Department of Public Health)

  • Erin L. Murray

    (California Department of Public Health)

  • Monica Sun

    (California Department of Public Health)

  • Lauren A. White

    (California Department of Public Health)

  • Logan C. Brooks

    (Carnegie Mellon University
    Berkeley)

  • Alden Green

    (Carnegie Mellon University)

  • Addison J. Hu

    (Carnegie Mellon University)

  • Roni Rosenfeld

    (Carnegie Mellon University)

  • Dmitry Shemetov

    (Carnegie Mellon University)

  • Ryan J. Tibshirani

    (Carnegie Mellon University
    Berkeley)

  • Daniel J. McDonald

    (University of British Columbia)

  • Sasikiran Kandula

    (Norwegian Institute of Public Health)

  • Sen Pei

    (Columbia University)

  • Rami Yaari

    (Columbia University)

  • Teresa K. Yamana

    (Columbia University)

  • Jeffrey Shaman

    (Columbia University
    Columbia University School of Climate)

  • Pulak Agarwal

    (Georgia Institute of Technology)

  • Srikar Balusu

    (Georgia Institute of Technology)

  • Gautham Gururajan

    (Georgia Institute of Technology)

  • Harshavardhan Kamarthi

    (Georgia Institute of Technology)

  • B. Aditya Prakash

    (Georgia Institute of Technology)

  • Rishi Raman

    (Georgia Institute of Technology)

  • Zhiyuan Zhao

    (Georgia Institute of Technology)

  • Alexander Rodríguez

    (University of Michigan)

  • Akilan Meiyappan

    (Guidehouse Advisory and Consulting Services)

  • Shalina Omar

    (Guidehouse Advisory and Consulting Services)

  • Prasith Baccam

    (IEM)

  • Heidi L. Gurung

    (IEM)

  • Brad T. Suchoski

    (IEM)

  • Steve A. Stage

    (IEM)

  • Marco Ajelli

    (Indiana University School of Public Health)

  • Allisandra G. Kummer

    (Indiana University School of Public Health)

  • Maria Litvinova

    (Indiana University School of Public Health)

  • Paulo C. Ventura

    (Indiana University School of Public Health)

  • Spencer Wadsworth

    (Iowa State University)

  • Jarad Niemi

    (Iowa State University)

  • Erica Carcelen

    (Johns Hopkins University)

  • Alison L. Hill

    (Johns Hopkins University)

  • Sara L. Loo

    (Johns Hopkins University)

  • Clifton D. McKee

    (Johns Hopkins University)

  • Koji Sato

    (Johns Hopkins University)

  • Claire Smith

    (Johns Hopkins University)

  • Shaun Truelove

    (Johns Hopkins University)

  • Sung-mok Jung

    (University of North Carolina at Chapel Hill)

  • Joseph C. Lemaitre

    (University of North Carolina at Chapel Hill)

  • Justin Lessler

    (University of North Carolina at Chapel Hill)

  • Thomas McAndrew

    (Lehigh University)

  • Wenxuan Ye

    (Lehigh University)

  • Nikos Bosse

    (London School of Health and Tropical Medicine)

  • William S. Hlavacek

    (Los Alamos National Laboratory)

  • Yen Ting Lin

    (Los Alamos National Laboratory)

  • Abhishek Mallela

    (Los Alamos National Laboratory)

  • Graham C. Gibson

    (Los Alamos National Laboratory)

  • Ye Chen

    (Northern Arizona University)

  • Shelby M. Lamm

    (Northern Arizona University)

  • Jaechoul Lee

    (Northern Arizona University)

  • Richard G. Posner

    (Northern Arizona University)

  • Amanda C. Perofsky

    (National Institutes of Health)

  • Cécile Viboud

    (National Institutes of Health)

  • Leonardo Clemente

    (Northeastern University)

  • Fred Lu

    (Northeastern University)

  • Austin G. Meyer

    (Northeastern University)

  • Mauricio Santillana

    (Northeastern University)

  • Matteo Chinazzi

    (Northeastern University)

  • Jessica T. Davis

    (Northeastern University)

  • Kunpeng Mu

    (Northeastern University)

  • Ana Pastore y Piontti

    (Northeastern University)

  • Alessandro Vespignani

    (Northeastern University)

  • Xinyue Xiong

    (Northeastern University)

  • Michal Ben-Nun

    (Predictive Science Inc)

  • Pete Riley

    (Predictive Science Inc)

  • James Turtle

    (Predictive Science Inc)

  • Chis Hulme-Lowe

    (LLC)

  • Shakeel Jessa

    (LLC)

  • V. P. Nagraj

    (LLC)

  • Stephen D. Turner

    (LLC)

  • Desiree Williams

    (LLC)

  • Avranil Basu

    (University of Georgia)

  • John M. Drake

    (University of Georgia)

  • Spencer J. Fox

    (University of Georgia)

  • Ehsan Suez

    (University of Georgia)

  • Monica G. Cojocaru

    (University of Guelph)

  • Edward W. Thommes

    (University of Guelph
    Sanofi)

  • Estee Y. Cramer

    (University of Massachusetts Amherst)

  • Aaron Gerding

    (University of Massachusetts Amherst)

  • Ariane Stark

    (University of Massachusetts Amherst)

  • Evan L. Ray

    (University of Massachusetts Amherst)

  • Nicholas G. Reich

    (University of Massachusetts Amherst)

  • Li Shandross

    (University of Massachusetts Amherst)

  • Nutcha Wattanachit

    (University of Massachusetts Amherst)

  • Yijin Wang

    (University of Massachusetts Amherst)

  • Martha W. Zorn

    (University of Massachusetts Amherst)

  • Majd Al Aawar

    (University of Southern California)

  • Ajitesh Srivastava

    (University of Southern California)

  • Lauren A. Meyers

    (University of Texas Austin)

  • Aniruddha Adiga

    (University of Virginia)

  • Benjamin Hurt

    (University of Virginia)

  • Gursharn Kaur

    (University of Virginia)

  • Bryan L. Lewis

    (University of Virginia)

  • Madhav Marathe

    (University of Virginia)

  • Srinivasan Venkatramanan

    (University of Virginia)

  • Patrick Butler

    (Virginia Tech)

  • Andrew Farabow

    (Virginia Tech)

  • Naren Ramakrishnan

    (Virginia Tech)

  • Nikhil Muralidhar

    (Stevens Institute of Technology)

  • Carrie Reed

    (Centers for Disease Control and Prevention)

  • Matthew Biggerstaff

    (Centers for Disease Control and Prevention)

  • Rebecca K. Borchering

    (Centers for Disease Control and Prevention)

Abstract

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021–22 and 2022–23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021–22 and 12 out of 18 models in 2022–23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021–22 and the 5th most accurate in the 2022–23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

Suggested Citation

  • Sarabeth M. Mathis & Alexander E. Webber & Tomás M. León & Erin L. Murray & Monica Sun & Lauren A. White & Logan C. Brooks & Alden Green & Addison J. Hu & Roni Rosenfeld & Dmitry Shemetov & Ryan J. Ti, 2024. "Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50601-9
    DOI: 10.1038/s41467-024-50601-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-50601-9
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-50601-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Emily Howerton & Lucie Contamin & Luke C. Mullany & Michelle Qin & Nicholas G. Reich & Samantha Bents & Rebecca K. Borchering & Sung-mok Jung & Sara L. Loo & Claire P. Smith & John Levander & Jessica , 2023. "Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    2. Johannes Bracher & Evan L Ray & Tilmann Gneiting & Nicholas G Reich, 2021. "Evaluating epidemic forecasts in an interval format," PLOS Computational Biology, Public Library of Science, vol. 17(2), pages 1-15, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chiang, Wen-Hao & Liu, Xueying & Mohler, George, 2022. "Hawkes process modeling of COVID-19 with mobility leading indicators and spatial covariates," International Journal of Forecasting, Elsevier, vol. 38(2), pages 505-520.
    2. Zhichao Li, 2022. "Forecasting Weekly Dengue Cases by Integrating Google Earth Engine-Based Risk Predictor Generation and Google Colab-Based Deep Learning Modeling in Fortaleza and the Federal District, Brazil," IJERPH, MDPI, vol. 19(20), pages 1-16, October.
    3. Tim K. Tsang & Qiurui Du & Benjamin J. Cowling & Cécile Viboud, 2024. "An adaptive weight ensemble approach to forecast influenza activity in an irregular seasonality context," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    4. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    5. Fabian Kruger & Hendrik Plett, 2022. "Prediction intervals for economic fixed-event forecasts," Papers 2210.13562, arXiv.org, revised Mar 2024.
    6. Luis A. Barboza & Shu Wei Chou Chen & Marcela Alfaro Córdoba & Eric J. Alfaro & Hugo G. Hidalgo, 2023. "Spatio‐temporal downscaling emulator for regional climate models," Environmetrics, John Wiley & Sons, Ltd., vol. 34(7), November.
    7. Kathryn S Taylor & James W Taylor, 2022. "Interval forecasts of weekly incident and cumulative COVID-19 mortality in the United States: A comparison of combining methods," PLOS ONE, Public Library of Science, vol. 17(3), pages 1-25, March.
    8. Jurgen A. Doornik & Jennifer L. Castle & David F. Hendry, 2021. "Modeling and forecasting the COVID‐19 pandemic time‐series data," Social Science Quarterly, Southwestern Social Science Association, vol. 102(5), pages 2070-2087, September.
    9. Coroneo, Laura & Iacone, Fabrizio & Paccagnini, Alessia & Santos Monteiro, Paulo, 2023. "Testing the predictive accuracy of COVID-19 forecasts," International Journal of Forecasting, Elsevier, vol. 39(2), pages 606-622.
    10. Ray, Evan L. & Brooks, Logan C. & Bien, Jacob & Biggerstaff, Matthew & Bosse, Nikos I. & Bracher, Johannes & Cramer, Estee Y. & Funk, Sebastian & Gerding, Aaron & Johansson, Michael A. & Rumack, Aaron, 2023. "Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1366-1383.
    11. Taylor, James W. & Taylor, Kathryn S., 2023. "Combining probabilistic forecasts of COVID-19 mortality in the United States," European Journal of Operational Research, Elsevier, vol. 304(1), pages 25-41.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50601-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.