IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v64y2018i5p2407-2426.html
   My bibliography  Save this article

Probability Forecasts Made at Multiple Lead Times

Author

Listed:
  • Eva Regnier

    (Naval Postgraduate School, Monterey, California 93943)

Abstract

Many probability forecasts are revised as new information becomes available, generating a time series of forecasts for a single event. Although methods for evaluating probability forecasts have been extensively studied, they apply to a single forecast per event. This paper is the first to evaluate probability forecasts that are made—and therefore revised—at many lead times for a single event. I postulate a norm for multi-period probability-forecasting systems and derive properties that should hold regardless of the forecasting process. I use these properties to develop methods for evaluating a forecasting system based on a sample. I apply these methods to the National Hurricane Center’s wind-speed probability forecasts and to statistical election forecasts, finding evidence that both can be improved using the current set of predictors.

Suggested Citation

  • Eva Regnier, 2018. "Probability Forecasts Made at Multiple Lead Times," Management Science, INFORMS, vol. 64(5), pages 2407-2426, May.
  • Handle: RePEc:inm:ormnsc:v:64:y:2018:i:5:p:2407-2426
    DOI: mnsc.2016.2720
    as

    Download full text from publisher

    File URL: https://doi.org/mnsc.2016.2720
    Download Restriction: no

    File URL: https://libkey.io/mnsc.2016.2720?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
    2. J. Carlos Escanciano & Ignacio N. Lobato, 2009. "Testing the Martingale Hypothesis," Palgrave Macmillan Books, in: Terence C. Mills & Kerry Patterson (ed.), Palgrave Handbook of Econometrics, chapter 20, pages 972-1003, Palgrave Macmillan.
    3. Lo, Andrew W. & MacKinlay, A. Craig, 1989. "The size and power of the variance ratio test in finite samples : A Monte Carlo investigation," Journal of Econometrics, Elsevier, vol. 40(2), pages 203-238, February.
    4. Justin Wolfers & Eric Zitzewitz, 2004. "Prediction Markets," Journal of Economic Perspectives, American Economic Association, vol. 18(2), pages 107-126, Spring.
    5. Charles, Amélie & Darné, Olivier & Kim, Jae H., 2011. "Small sample properties of alternative tests for martingale difference hypothesis," Economics Letters, Elsevier, vol. 110(2), pages 151-154, February.
    6. Drew A. Linzer, 2013. "Dynamic Bayesian Forecasting of Presidential Elections in the States," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(501), pages 124-134, March.
    7. Jae H. Kim & Abul Shamsuddin, 2015. "A closer look at return predictability of the US stock market: evidence from new panel variance ratio tests," Quantitative Finance, Taylor & Francis Journals, vol. 15(9), pages 1501-1514, September.
    8. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    9. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    10. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    11. Harvey, Nigel, 1995. "Why Are Judgments Less Consistent in Less Predictable Task Situations?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 63(3), pages 247-263, September.
    12. Nordhaus, William D, 1987. "Forecasting Efficiency: Concepts and Applications," The Review of Economics and Statistics, MIT Press, vol. 69(4), pages 667-674, November.
    13. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2009. "Sensitivity to Distance and Baseline Distributions in Forecast Evaluation," Management Science, INFORMS, vol. 55(4), pages 582-590, April.
    14. Dean Croushore, 1993. "Introducing: the survey of professional forecasters," Business Review, Federal Reserve Bank of Philadelphia, issue Nov, pages 3-15.
    15. Kenny, Geoff & Genre, Véronique & Bowles, Carlos & Friz, Roberta & Meyler, Aidan & Rautanen, Tuomas, 2007. "The ECB survey of professional forecasters (SPF) - A review after eight years' experience," Occasional Paper Series 59, European Central Bank.
    16. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    17. Robert L. Winkler, 1994. "Evaluating Probabilities: Asymmetric Scoring Rules," Management Science, INFORMS, vol. 40(11), pages 1395-1405, November.
    18. Wang, Samuel S.-H., 2015. "Origins of Presidential poll aggregation: A perspective from 2004 to 2012," International Journal of Forecasting, Elsevier, vol. 31(3), pages 898-909.
    19. Reinhard Selten, 1998. "Axiomatic Characterization of the Quadratic Scoring Rule," Experimental Economics, Springer;Economic Science Association, vol. 1(1), pages 43-61, June.
    20. Kian‐Ping Lim & Robert Brooks, 2011. "The Evolution Of Stock Market Efficiency Over Time: A Survey Of The Empirical Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 25(1), pages 69-108, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eva D. Regnier, 2020. "What Is Six Hours Worth? The Impact of Lead Time on Tropical-Storm Preparation Decisions," Decision Analysis, INFORMS, vol. 17(1), pages 9-23, March.
    2. James S. Dyer & James E. Smith, 2021. "Innovations in the Science and Practice of Decision Analysis: The Role of Management Science," Management Science, INFORMS, vol. 67(9), pages 5364-5378, September.
    3. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    4. Chidambaram Subbiah & Andrea C. Hupman & Haitao Li & Joseph Simonis, 2023. "Improving Software Development Effort Estimation with a Novel Design Pattern Model," Interfaces, INFORMS, vol. 53(3), pages 192-206, May.
    5. Ho Cheung Brian Lee & Jan Stallaert & Ming Fan, 2020. "Anomalies in Probability Estimates for Event Forecasting on Prediction Markets," Production and Operations Management, Production and Operations Management Society, vol. 29(9), pages 2077-2095, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    2. Charles, Amélie & Darné, Olivier & Kim, Jae H., 2012. "Exchange-rate return predictability and the adaptive markets hypothesis: Evidence from major foreign exchange rates," Journal of International Money and Finance, Elsevier, vol. 31(6), pages 1607-1626.
    3. Lazăr, Dorina & Todea, Alexandru & Filip, Diana, 2012. "Martingale difference hypothesis and financial crisis: Empirical evidence from European emerging foreign exchange markets," Economic Systems, Elsevier, vol. 36(3), pages 338-350.
    4. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    5. Edgar C. Merkle & Mark Steyvers, 2013. "Choosing a Strictly Proper Scoring Rule," Decision Analysis, INFORMS, vol. 10(4), pages 292-304, December.
    6. Kim, Jae & Doucouliagos, Hristos & Stanley, T. D., 2014. "Market efficiency in Asian and Australasian stock markets: a fresh look at the evidence," Working Papers eco_2014_9, Deakin University, Department of Economics.
    7. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    8. Raphael Flepp & Stephan Nüesch & Egon Franck, 2013. "Liquidity, Market Efficiency and the Influence of Noise Traders: Quasi-Experimental Evidence from the Betting Industry," Working Papers 341, University of Zurich, Department of Business Administration (IBW).
    9. David J. Johnstone & Victor Richmond R. Jose & Robert L. Winkler, 2011. "Tailored Scoring Rules for Probabilities," Decision Analysis, INFORMS, vol. 8(4), pages 256-268, December.
    10. Victor Richmond R. Jose & Robert L. Winkler, 2009. "Evaluating Quantile Assessments," Operations Research, INFORMS, vol. 57(5), pages 1287-1297, October.
    11. Zhong, Meirui & Zhang, Rui & Ren, Xiaohang, 2023. "The time-varying effects of liquidity and market efficiency of the European Union carbon market: Evidence from the TVP-SVAR-SV approach," Energy Economics, Elsevier, vol. 123(C).
    12. Carmen López-Martín & Sonia Benito Muela & Raquel Arguedas, 2021. "Efficiency in cryptocurrency markets: new evidence," Eurasian Economic Review, Springer;Eurasia Business and Economics Society, vol. 11(3), pages 403-431, September.
    13. Kian-Ping Lim & Weiwei Luo & Jae H. Kim, 2013. "Are US stock index returns predictable? Evidence from automatic autocorrelation-based tests," Applied Economics, Taylor & Francis Journals, vol. 45(8), pages 953-962, March.
    14. Charles, Amélie & Darné, Olivier & Fouilloux, Jessica, 2011. "Testing the martingale difference hypothesis in CO2 emission allowances," Economic Modelling, Elsevier, vol. 28(1-2), pages 27-35, January.
    15. Righi, Marcelo Brutti & Ceretta, Paulo Sergio, 2013. "Risk prediction management and weak form market efficiency in Eurozone financial crisis," International Review of Financial Analysis, Elsevier, vol. 30(C), pages 384-393.
    16. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207, April.
    17. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    18. Stéphane Goutte & David Guerreiro & Bilel Sanhaji & Sophie Saglio & Julien Chevallier, 2019. "International Financial Markets," Post-Print halshs-02183053, HAL.
    19. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207.
    20. Chambers, Christopher P. & Healy, Paul J. & Lambert, Nicolas S., 2019. "Proper scoring rules with general preferences: A dual characterization of optimal reports," Games and Economic Behavior, Elsevier, vol. 117(C), pages 322-341.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:64:y:2018:i:5:p:2407-2426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.