IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v16y2019i4p301-313.html
   My bibliography  Save this article

A Probability Scoring Rule for Simultaneous Events

Author

Listed:
  • Andrew Grant

    (Discipline of Finance, University of Sydney Business School, Sydney, New South Wales 2006, Australia)

  • David Johnstone

    (Discipline of Finance, University of Sydney Business School, Sydney, New South Wales 2006, Australia; School of Accounting, Economics and Finance, University of Wollongong, Wollongong, New South Wales 2500, Australia)

  • Oh Kang Kwon

    (Discipline of Finance, University of Sydney Business School, Sydney, New South Wales 2006, Australia)

Abstract

We develop a scoring rule tailored to a decision maker who makes simultaneous bets on events that occur at times that require bets to be placed together. The rule proposed captures the economic benefit to a well-defined bettor who acts on one set of probabilities p against a baseline or rival set q . To allow for simultaneous bets, we assume a myopic power utility function with a risk aversion parameter tailored to suit the given user or application. Our score function is “proper” in the usual sense of motivating honesty. Apart from a special case of power utility, namely, log utility, the score is not “local,” which we excuse because a local scoring rule cannot capture the economic result that our score reflects. An interesting property of our rule is that the individual scores from individual events are multiplicative, rather than additive. Probability scores are often added to give a measure of aggregate performance over a set of trials. Our rule is unique in that scores must be multiplied to reach a meaningful aggregate.

Suggested Citation

  • Andrew Grant & David Johnstone & Oh Kang Kwon, 2019. "A Probability Scoring Rule for Simultaneous Events," Decision Analysis, INFORMS, vol. 16(4), pages 301-313, December.
    • Handle: RePEc:inm:ordeca:v:16:y:2019:i:4:p:301-313
    DOI: 10.1287/deca.2019.0391
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/deca.2019.0393
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.2019.0391?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. D. J. Johnstone, 2011. "Economic Interpretation of Probabilities Estimated by Maximum Likelihood or Score," Management Science, INFORMS, vol. 57(2), pages 308-314, February.
    2. Arthur Carvalho, 2016. "An Overview of Applications of Proper Scoring Rules," Decision Analysis, INFORMS, vol. 13(4), pages 223-242, December.
    3. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    4. MacLean, Leonard C. & Sanegre, Rafael & Zhao, Yonggan & Ziemba, William T., 2004. "Capital growth with security," Journal of Economic Dynamics and Control, Elsevier, vol. 28(5), pages 937-954, February.
    5. Andrew Grant & David Johnstone & Oh Kang Kwon, 2008. "Optimal Betting Strategies for Simultaneous Games," Decision Analysis, INFORMS, vol. 5(1), pages 10-18, March.
    6. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    7. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    8. J. Eric Bickel, 2007. "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules," Decision Analysis, INFORMS, vol. 4(2), pages 49-65, June.
    9. Joseph B. Kadane, 2011. "Partial-Kelly Strategies and Expected Utility: Small-Edge Asymptotics," Decision Analysis, INFORMS, vol. 8(1), pages 4-9, March.
    10. Rose D. Baker & Ian G. McHale, 2013. "Optimal Betting Under Parameter Uncertainty: Improving the Kelly Criterion," Decision Analysis, INFORMS, vol. 10(3), pages 189-199, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zachary J. Smith & J. Eric Bickel, 2020. "Additive Scoring Rules for Discrete Sample Spaces," Decision Analysis, INFORMS, vol. 17(2), pages 115-133, June.
    2. David Johnstone & Stewart Jones & Oliver Jones & Steve Tulig, 2021. "Scoring Probability Forecasts by a User’s Bets Against a Market Consensus," Decision Analysis, INFORMS, vol. 18(3), pages 169-184, September.
    3. Vicki M. Bier & Simon French, 2020. "From the Editors: Decision Analysis Focus and Trends," Decision Analysis, INFORMS, vol. 17(1), pages 1-8, March.
    4. David J Johnstone, 2023. "Capital budgeting and Kelly betting," Australian Journal of Management, Australian School of Business, vol. 48(3), pages 625-651, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Edgar C. Merkle & Mark Steyvers, 2013. "Choosing a Strictly Proper Scoring Rule," Decision Analysis, INFORMS, vol. 10(4), pages 292-304, December.
    2. David J. Johnstone & Victor Richmond R. Jose & Robert L. Winkler, 2011. "Tailored Scoring Rules for Probabilities," Decision Analysis, INFORMS, vol. 8(4), pages 256-268, December.
    3. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    4. Zachary J. Smith & J. Eric Bickel, 2020. "Additive Scoring Rules for Discrete Sample Spaces," Decision Analysis, INFORMS, vol. 17(2), pages 115-133, June.
    5. David Johnstone & Stewart Jones & Oliver Jones & Steve Tulig, 2021. "Scoring Probability Forecasts by a User’s Bets Against a Market Consensus," Decision Analysis, INFORMS, vol. 18(3), pages 169-184, September.
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. D. J. Johnstone & S. Jones & V. R. R. Jose & M. Peat, 2013. "Measures of the economic value of probabilities of bankruptcy," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 635-653, June.
    8. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2008. "Scoring Rules, Generalized Entropy, and Utility Maximization," Operations Research, INFORMS, vol. 56(5), pages 1146-1157, October.
    9. David Kaplan & Chansoon Lee, 2018. "Optimizing Prediction Using Bayesian Model Averaging: Examples Using Large-Scale Educational Assessments," Evaluation Review, , vol. 42(4), pages 423-457, August.
    10. D. J. Johnstone, 2021. "Accounting information, disclosure, and expected utility: Do investors really abhor uncertainty?," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 48(1-2), pages 3-35, January.
    11. L. Robin Keller & Ali Abbas & J. Eric Bickel & Vicki M. Bier & David V. Budescu & John C. Butler & Philippe Delquié & Kenneth C. Lichtendahl & Jason R. W. Merrick & Ahti Salo & George Wu, 2011. "From the Editors ---Probability Scoring Rules, Ambiguity, Multiattribute Terrorist Utility, and Sensitivity Analysis," Decision Analysis, INFORMS, vol. 8(4), pages 251-255, December.
    12. Rakesh K. Sarin, 2013. "From the Editor —Optimal Betting, Reducing Unnecessary Mammography in Breast Cancer Diagnosis, Product Line Design, and Value of Information," Decision Analysis, INFORMS, vol. 10(3), pages 187-188, September.
    13. C. Alexander & M. Coulon & Y. Han & X. Meng, 2024. "Evaluating the discrimination ability of proper multi-variate scoring rules," Annals of Operations Research, Springer, vol. 334(1), pages 857-883, March.
    14. Tino Werner, 2022. "Elicitability of Instance and Object Ranking," Decision Analysis, INFORMS, vol. 19(2), pages 123-140, June.
    15. Chambers, Christopher P. & Healy, Paul J. & Lambert, Nicolas S., 2019. "Proper scoring rules with general preferences: A dual characterization of optimal reports," Games and Economic Behavior, Elsevier, vol. 117(C), pages 322-341.
    16. Victor Richmond R. Jose & Robert F. Nau & Robert L. Winkler, 2009. "Sensitivity to Distance and Baseline Distributions in Forecast Evaluation," Management Science, INFORMS, vol. 55(4), pages 582-590, April.
    17. D. J. Johnstone, 2011. "Economic Interpretation of Probabilities Estimated by Maximum Likelihood or Score," Management Science, INFORMS, vol. 57(2), pages 308-314, February.
    18. Victor Richmond R. Jose & Robert L. Winkler, 2009. "Evaluating Quantile Assessments," Operations Research, INFORMS, vol. 57(5), pages 1287-1297, October.
    19. David Kaplan, 2021. "On the Quantification of Model Uncertainty: A Bayesian Perspective," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 215-238, March.
    20. Peysakhovich, Alexander & Plagborg-Møller, Mikkel, 2012. "A note on proper scoring rules and risk aversion," Economics Letters, Elsevier, vol. 117(1), pages 357-361.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:16:y:2019:i:4:p:301-313. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.