IDEAS home Printed from https://ideas.repec.org/a/spr/aistmt/v72y2020i3d10.1007_s10463-019-00705-7.html
   My bibliography  Save this article

Properization: constructing proper scoring rules via Bayes acts

Author

Listed:
  • Jonas R. Brehmer

    (University of Mannheim)

  • Tilmann Gneiting

    (Karlsruhe Institute of Technology (KIT)
    Heidelberg Institute for Theoretical Studies)

Abstract

Scoring rules serve to quantify predictive performance. A scoring rule is proper if truth telling is an optimal strategy in expectation. Subject to customary regularity conditions, every scoring rule can be made proper, by applying a special case of the Bayes act construction studied by Grünwald and Dawid (Ann Stat 32:1367–1433, 2004) and Dawid (Ann Inst Stat Math 59:77–93, 2007), to which we refer as properization. We discuss examples from the recent literature and apply the construction to create new types, and reinterpret existing forms, of proper scoring rules and consistent scoring functions. In an abstract setting, we formulate sufficient conditions under which Bayes acts exist and scoring rules can be made proper.

Suggested Citation

  • Jonas R. Brehmer & Tilmann Gneiting, 2020. "Properization: constructing proper scoring rules via Bayes acts," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(3), pages 659-673, June.
    • Handle: RePEc:spr:aistmt:v:72:y:2020:i:3:d:10.1007_s10463-019-00705-7
    DOI: 10.1007/s10463-019-00705-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10463-019-00705-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10463-019-00705-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    2. Petra Friederichs & Thordis L. Thorarinsdottir, 2012. "Forecast verification for extreme value distributions with an application to probabilistic peak wind prediction," Environmetrics, John Wiley & Sons, Ltd., vol. 23(7), pages 579-594, November.
    3. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    4. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2018. "The M4 Competition: Results, findings, conclusion and way forward," International Journal of Forecasting, Elsevier, vol. 34(4), pages 802-808.
    5. A. Dawid, 2007. "The geometry of proper scoring rules," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 59(1), pages 77-93, March.
    6. repec:hal:journl:peer-00834423 is not listed on IDEAS
    7. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    8. Alexander Dawid & Monica Musio, 2014. "Theory and applications of proper scoring rules," METRON, Springer;Sapienza Università di Roma, vol. 72(2), pages 169-183, August.
    9. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    10. Granger, Clive W.J. & Machina, Mark J., 2006. "Forecasting and Decision Theory," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 2, pages 81-98, Elsevier.
    11. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    12. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    13. Charalambos D. Aliprantis & Kim C. Border, 2006. "Infinite Dimensional Analysis," Springer Books, Springer, edition 0, number 978-3-540-29587-7, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yannick Hoga & Timo Dimitriadis, 2021. "On Testing Equal Conditional Predictive Ability Under Measurement Error," Papers 2106.11104, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ruben Loaiza‐Maya & Gael M. Martin & David T. Frazier, 2021. "Focused Bayesian prediction," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(5), pages 517-543, August.
    2. Olafsdottir, Helga Kristin & Rootzén, Holger & Bolin, David, 2024. "Locally tail-scale invariant scoring rules for evaluation of extreme value forecasts," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1701-1720.
    3. Taillardat, Maxime & Fougères, Anne-Laure & Naveau, Philippe & de Fondeville, Raphaël, 2023. "Evaluating probabilistic forecasts of extremes using continuous ranked probability score distributions," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1448-1459.
    4. Xiaochun Meng & James W. Taylor & Souhaib Ben Taieb & Siran Li, 2020. "Scores for Multivariate Distributions and Level Sets," Papers 2002.09578, arXiv.org, revised Jun 2023.
    5. Alexander Henzi & Johanna F. Ziegel & Tilmann Gneiting, 2021. "Isotonic distributional regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 963-993, November.
    6. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    7. Emilio Zanetti Chini, 2018. "Forecaster’s utility and forecasts coherence," DEM Working Papers Series 145, University of Pavia, Department of Economics and Management.
    8. Sándor Baran & Patrícia Szokol & Marianna Szabó, 2021. "Truncated generalized extreme value distribution‐based ensemble model output statistics model for calibration of wind speed ensemble forecasts," Environmetrics, John Wiley & Sons, Ltd., vol. 32(6), September.
    9. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    10. Diks, Cees & Fang, Hao, 2020. "Comparing density forecasts in a risk management context," International Journal of Forecasting, Elsevier, vol. 36(2), pages 531-551.
    11. Tobias Fissler & Hajo Holzmann, 2022. "Measurability of functionals and of ideal point forecasts," Papers 2203.08635, arXiv.org.
    12. Kapetanios, G. & Mitchell, J. & Price, S. & Fawcett, N., 2015. "Generalised density forecast combinations," Journal of Econometrics, Elsevier, vol. 188(1), pages 150-165.
    13. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    14. Martin, Gael M. & Loaiza-Maya, Rubén & Maneesoonthorn, Worapree & Frazier, David T. & Ramírez-Hassan, Andrés, 2022. "Optimal probabilistic forecasts: When do they work?," International Journal of Forecasting, Elsevier, vol. 38(1), pages 384-406.
    15. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    16. Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose & Robert L. Winkler, 2017. "Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs," Operations Research, INFORMS, vol. 65(3), pages 712-728, June.
    17. David T. Frazier & Ruben Loaiza-Maya & Gael M. Martin, 2021. "Variational Bayes in State Space Models: Inferential and Predictive Accuracy," Papers 2106.12262, arXiv.org, revised Feb 2022.
    18. Ramon de Punder & Timo Dimitriadis & Rutger-Jan Lange, 2024. "Kullback-Leibler-based characterizations of score-driven updates," Papers 2408.02391, arXiv.org, revised Sep 2024.
    19. Souhaib Ben Taieb & James W. Taylor & Rob J. Hyndman, 2017. "Coherent Probabilistic Forecasts for Hierarchical Time Series," Monash Econometrics and Business Statistics Working Papers 3/17, Monash University, Department of Econometrics and Business Statistics.
    20. Silius M. Vandeskog & Sara Martino & Daniela Castro-Camilo & Håvard Rue, 2022. "Modelling Sub-daily Precipitation Extremes with the Blended Generalised Extreme Value Distribution," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(4), pages 598-621, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aistmt:v:72:y:2020:i:3:d:10.1007_s10463-019-00705-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.