IDEAS home Printed from https://ideas.repec.org/p/zbw/sfb475/200324.html
   My bibliography  Save this paper

Evaluating probability forecasts in terms of refinement and strictly proper scoring rules

Author

Listed:
  • Krämer, Walter

Abstract

This note gives an easily verified necessary and sufficient condition for one probability forecaster to empirically outperform another one in terms of all strictly proper scoring rules.

Suggested Citation

  • Krämer, Walter, 2003. "Evaluating probability forecasts in terms of refinement and strictly proper scoring rules," Technical Reports 2003,24, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    • Handle: RePEc:zbw:sfb475:200324
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/49329/1/373258887.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Walter Krämer & Michael Bücker, 2011. "Probleme des Qualitätsvergleichs von Kreditausfallprognosen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 5(1), pages 39-58, March.
    2. Walter Krämer & André Güttler, 2008. "On comparing the accuracy of default predictions in the rating industry," Empirical Economics, Springer, vol. 34(2), pages 343-356, March.
    3. Krämer, Walter & Neumärker, Simon, 2016. "Comparing the accuracy of default predictions in the rating industry for different sets of obligors," Economics Letters, Elsevier, vol. 145(C), pages 48-51.
    4. Krämer, Walter & Neumärker, Simon, 2019. "Skill Scores and modified Lorenz domination in default forecasts," Economics Letters, Elsevier, vol. 181(C), pages 61-64.
    5. Krämer, Walter, 2004. "Qualitätsvergleiche bei Kreditausfallprognosen," Technical Reports 2004,07, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Victor Jose, 2009. "A Characterization for the Spherical Scoring Rule," Theory and Decision, Springer, vol. 66(3), pages 263-281, March.
    2. Sohail Abbas & Zulfiqar Ali Mayo, 2021. "Impact of temperature and rainfall on rice production in Punjab, Pakistan," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(2), pages 1706-1728, February.
    3. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    4. Clements, Michael P. & Harvey, David I., 2011. "Combining probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 208-223.
    5. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    6. Daniel Benjamin & David R Mandel & Jonathan Kimmelman, 2017. "Can cancer researchers accurately judge whether preclinical reports will reproduce?," PLOS Biology, Public Library of Science, vol. 15(6), pages 1-17, June.
    7. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 211-235, August.
    8. David Kaplan & Chansoon Lee, 2018. "Optimizing Prediction Using Bayesian Model Averaging: Examples Using Large-Scale Educational Assessments," Evaluation Review, , vol. 42(4), pages 423-457, August.
    9. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    10. Michaël Zamo & Liliane Bel & Olivier Mestre, 2021. "Sequential aggregation of probabilistic forecasts—Application to wind speed ensemble forecasts," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 202-225, January.
    11. Laurent L. Pauwels & Andrey L. Vasnev, 2017. "Forecast combination for discrete choice models: predicting FOMC monetary policy decisions," Empirical Economics, Springer, vol. 52(1), pages 229-254, February.
    12. Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose & Robert L. Winkler, 2017. "Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs," Operations Research, INFORMS, vol. 65(3), pages 712-728, June.
    13. David J. Johnstone, 2007. "The Parimutuel Kelly Probability Scoring Rule," Decision Analysis, INFORMS, vol. 4(2), pages 66-75, June.
    14. Cameron J. Williams & Kevin J. Wilson & Nina Wilson, 2021. "A comparison of prior elicitation aggregation using the classical method and SHELF," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 920-940, July.
    15. Braun, Julia & Sabanés Bové, Daniel & Held, Leonhard, 2014. "Choice of generalized linear mixed models using predictive crossvalidation," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 190-202.
    16. Taillardat, Maxime & Fougères, Anne-Laure & Naveau, Philippe & de Fondeville, Raphaël, 2023. "Evaluating probabilistic forecasts of extremes using continuous ranked probability score distributions," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1448-1459.
    17. Sushil Bikhchandani & John W. Mamer, 2013. "Decreasing Marginal Value of Information Under Symmetric Loss," Decision Analysis, INFORMS, vol. 10(3), pages 245-256, September.
    18. Walter Krämer & Michael Bücker, 2011. "Probleme des Qualitätsvergleichs von Kreditausfallprognosen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 5(1), pages 39-58, March.
    19. Michael P. Clements & David I. Harvey, 2010. "Forecast encompassing tests and probability forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(6), pages 1028-1062.
    20. Raja Chakir & Thibault Laurent & Anne Ruiz-Gazen & Christine Thomas-Agnan & Céline Vignes, 2017. "Prédiction de l’usage des sols sur un zonage régulier à différentes résolutions et à partir de covariables facilement accessibles," Revue économique, Presses de Sciences-Po, vol. 68(3), pages 435-469.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:sfb475:200324. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/isdorde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.