IDEAS home Printed from https://ideas.repec.org/a/jof/jforec/v25y2006i3p223-226.html
   My bibliography  Save this article

Evaluating probability forecasts in terms of refinement and strictly proper scoring rules

Author

Listed:
  • Walter Krämer

    (Fachbereich Statistik, Universität Dortmund, Germany)

Abstract

This note gives an easily verified necessary and sufficient condition for one probability forecaster to empirically outperform another one in terms of all strictly proper scoring rules. Copyright © 2006 John Wiley & Sons, Ltd.

Suggested Citation

  • Walter Krämer, 2006. "Evaluating probability forecasts in terms of refinement and strictly proper scoring rules," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(3), pages 223-226.
    • Handle: RePEc:jof:jforec:v:25:y:2006:i:3:p:223-226
    DOI: 10.1002/for.976
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/for.976
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.976?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Krämer, Walter, 2004. "Qualitätsvergleiche bei Kreditausfallprognosen," Technical Reports 2004,07, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    2. Walter Krämer & Michael Bücker, 2011. "Probleme des Qualitätsvergleichs von Kreditausfallprognosen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 5(1), pages 39-58, March.
    3. Walter Krämer & André Güttler, 2008. "On comparing the accuracy of default predictions in the rating industry," Empirical Economics, Springer, vol. 34(2), pages 343-356, March.
    4. Krämer, Walter & Neumärker, Simon, 2016. "Comparing the accuracy of default predictions in the rating industry for different sets of obligors," Economics Letters, Elsevier, vol. 145(C), pages 48-51.
    5. Krämer, Walter & Neumärker, Simon, 2019. "Skill Scores and modified Lorenz domination in default forecasts," Economics Letters, Elsevier, vol. 181(C), pages 61-64.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Victor Jose, 2009. "A Characterization for the Spherical Scoring Rule," Theory and Decision, Springer, vol. 66(3), pages 263-281, March.
    2. Sohail Abbas & Zulfiqar Ali Mayo, 2021. "Impact of temperature and rainfall on rice production in Punjab, Pakistan," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(2), pages 1706-1728, February.
    3. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    4. Clements, Michael P. & Harvey, David I., 2011. "Combining probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 208-223.
    5. Daniel Benjamin & David R Mandel & Jonathan Kimmelman, 2017. "Can cancer researchers accurately judge whether preclinical reports will reproduce?," PLOS Biology, Public Library of Science, vol. 15(6), pages 1-17, June.
    6. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 211-235, August.
    7. David Kaplan & Chansoon Lee, 2018. "Optimizing Prediction Using Bayesian Model Averaging: Examples Using Large-Scale Educational Assessments," Evaluation Review, , vol. 42(4), pages 423-457, August.
    8. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    9. Michaël Zamo & Liliane Bel & Olivier Mestre, 2021. "Sequential aggregation of probabilistic forecasts—Application to wind speed ensemble forecasts," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 202-225, January.
    10. Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose & Robert L. Winkler, 2017. "Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs," Operations Research, INFORMS, vol. 65(3), pages 712-728, June.
    11. David J. Johnstone, 2007. "The Parimutuel Kelly Probability Scoring Rule," Decision Analysis, INFORMS, vol. 4(2), pages 66-75, June.
    12. Braun, Julia & Sabanés Bové, Daniel & Held, Leonhard, 2014. "Choice of generalized linear mixed models using predictive crossvalidation," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 190-202.
    13. Sushil Bikhchandani & John W. Mamer, 2013. "Decreasing Marginal Value of Information Under Symmetric Loss," Decision Analysis, INFORMS, vol. 10(3), pages 245-256, September.
    14. Walter Krämer & Michael Bücker, 2011. "Probleme des Qualitätsvergleichs von Kreditausfallprognosen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 5(1), pages 39-58, March.
    15. Michael P. Clements & David I. Harvey, 2010. "Forecast encompassing tests and probability forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(6), pages 1028-1062.
    16. Raja Chakir & Thibault Laurent & Anne Ruiz-Gazen & Christine Thomas-Agnan & Céline Vignes, 2017. "Prédiction de l’usage des sols sur un zonage régulier à différentes résolutions et à partir de covariables facilement accessibles," Revue économique, Presses de Sciences-Po, vol. 68(3), pages 435-469.
    17. Wei Wei & Leonhard Held, 2014. "Calibration tests for count data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(4), pages 787-805, December.
    18. David Johnstone, 2007. "Discussion of Altman and Sabato," Abacus, Accounting Foundation, University of Sydney, vol. 43(3), pages 358-362, September.
    19. David Johnstone, 2007. "Economic Darwinism: Who has the Best Probabilities?," Theory and Decision, Springer, vol. 62(1), pages 47-96, February.
    20. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jof:jforec:v:25:y:2006:i:3:p:223-226. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.