IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v15y2018i2p72-89.html
   My bibliography  Save this article

On the Road to Making Science of “Art”: Risk Bias in Market Scoring Rules

Author

Listed:
  • Majid Karimi

    (College of Business Administration, California State University San Marcos, San Marcos, California 92096)

  • Stanko Dimitrov

    (Department of Management Sciences, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada)

Abstract

We study market scoring rule (MSR) prediction markets in the presence of risk-averse or risk-seeking agents that have unknown yet bounded risk preferences. It is well known that if agents can be prescreened, then MSRs can be corrected to elicit agents’ beliefs. However, agents cannot always be screened, and instead, an online MSR mechanism is needed. We show that agents’ submitted reports always deviate from their beliefs, unless their beliefs are identical to the current market estimate. This means, in most cases it is impossible for a MSR prediction market to elicit an individual agent’s exact belief. To analyze this issue, we introduce a measure to calculate the deviation between an agent’s reported belief and personal belief. We further derive the necessary and sufficient conditions for a MSR to yield a lower deviation relative to another MSR. We find that the deviation of a MSR prediction market is related to the liquidity provided in the MSR’s corresponding cost-function prediction market. We use the relation between deviation and liquidity to present a systematic approach to help determine the amount of liquidity required for cost-function prediction markets, an activity that up to this point has been described as “art” in the literature.

Suggested Citation

  • Majid Karimi & Stanko Dimitrov, 2018. "On the Road to Making Science of “Art”: Risk Bias in Market Scoring Rules," Decision Analysis, INFORMS, vol. 15(2), pages 72-89, June.
    • Handle: RePEc:inm:ordeca:v:15:y:2018:i:2:p:72-89
    DOI: 10.1287/deca.2017.0362
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/deca.2017.0362
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.2017.0362?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Manski, Charles F., 2006. "Interpreting the predictions of prediction markets," Economics Letters, Elsevier, vol. 91(3), pages 425-429, June.
    2. Babcock, Bruce A. & Choi, E. Kwan & Feinerman, Eli, 1993. "Risk And Probability Premiums For Cara Utility Functions," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 18(01), pages 1-8, July.
    3. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    4. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, March.
    5. Robin Hanson, 2003. "Combinatorial Information Market Design," Information Systems Frontiers, Springer, vol. 5(1), pages 107-119, January.
    6. Franklin Allen, 1987. "Notes--Discovering Personal Probabilities When Utility Functions are Unknown," Management Science, INFORMS, vol. 33(4), pages 542-544, April.
    7. Joyce E. Berg & George R. Neumann & Thomas A. Rietz, 2009. "Searching for Google's Value: Using Prediction Markets to Forecast Market Capitalization Prior to an Initial Public Offering," Management Science, INFORMS, vol. 55(3), pages 348-361, March.
    8. Rajiv Sethi & Jennifer Wortman Vaughan, 2016. "Belief Aggregation with Automated Market Makers," Computational Economics, Springer;Society for Computational Economics, vol. 48(1), pages 155-178, June.
    9. Wolfers, Justin & Zitzewitz, Eric, 2006. "Interpreting Prediction Market Prices as Probabilities," IZA Discussion Papers 2092, Institute of Labor Economics (IZA).
    10. Paul J. Healy & Sera Linardi & J. Richard Lowery & John O. Ledyard, 2010. "Prediction Markets: Alternative Mechanisms for Complex Environments with Few Traders," Management Science, INFORMS, vol. 56(11), pages 1977-1996, November.
    11. Thomas R. Palfrey & Stephanie W. Wang, 2012. "Speculative Overpricing in Asset Markets With Information Flows," Econometrica, Econometric Society, vol. 80(5), pages 1937-1976, September.
    12. Felipe A. Csaszar & J. P. Eggers, 2013. "Organizational Decision Making: An Information Aggregation View," Management Science, INFORMS, vol. 59(10), pages 2257-2277, October.
    13. Bo Cowgill & Eric Zitzewitz, 2015. "Corporate Prediction Markets: Evidence from Google, Ford, and Firm X," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 82(4), pages 1309-1341.
    14. Michael Ostrovsky, 2012. "Information Aggregation in Dynamic Markets With Strategic Traders," Econometrica, Econometric Society, vol. 80(6), pages 2595-2647, November.
    15. repec:reg:rpubli:460 is not listed on IDEAS
    16. Bossaerts, Peter & Fine, Leslie & Ledyard, John, 2002. "Inducing liquidity in thin financial markets through combined-value trading mechanisms," European Economic Review, Elsevier, vol. 46(9), pages 1671-1695, October.
    17. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    18. Stephanie Wang, 2012. "Speculative Overpricing in Asset Markets with Information Flows," Working Paper 489, Department of Economics, University of Pittsburgh, revised Jan 2012.
    19. Kay-Yut Chen & Leslie R. Fine & Bernardo A. Huberman, 2004. "Eliminating Public Knowledge Biases in Information-Aggregation Mechanisms," Management Science, INFORMS, vol. 50(7), pages 983-994, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Di, Chen & Dimitrov, Stanko & He, Qi-Ming, 2019. "Incentive compatibility in prediction markets: Costly actions and external incentives," International Journal of Forecasting, Elsevier, vol. 35(1), pages 351-370.
    2. Karimi, Majid & Zaerpour, Nima, 2022. "Put your money where your forecast is: Supply chain collaborative forecasting with cost-function-based prediction markets," European Journal of Operational Research, Elsevier, vol. 300(3), pages 1035-1049.
    3. Vicki M. Bier & Simon French, 2020. "From the Editors: Decision Analysis Focus and Trends," Decision Analysis, INFORMS, vol. 17(1), pages 1-8, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karimi, Majid & Zaerpour, Nima, 2022. "Put your money where your forecast is: Supply chain collaborative forecasting with cost-function-based prediction markets," European Journal of Operational Research, Elsevier, vol. 300(3), pages 1035-1049.
    2. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    3. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    4. Pavel Atanasov & Phillip Rescober & Eric Stone & Samuel A. Swift & Emile Servan-Schreiber & Philip Tetlock & Lyle Ungar & Barbara Mellers, 2017. "Distilling the Wisdom of Crowds: Prediction Markets vs. Prediction Polls," Management Science, INFORMS, vol. 63(3), pages 691-706, March.
    5. Dian Yu & Jianjun Gao & Weiping Wu & Zizhuo Wang, 2022. "Price Interpretability of Prediction Markets: A Convergence Analysis," Papers 2205.08913, arXiv.org, revised Nov 2023.
    6. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    7. Razvan Tarnaud, 2019. "Convergence within binary market scoring rules," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 68(4), pages 1017-1050, November.
    8. Arthur Carvalho & Stanko Dimitrov & Kate Larson, 2018. "On proper scoring rules and cumulative prospect theory," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 6(3), pages 343-376, November.
    9. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    10. Snowberg, Erik & Wolfers, Justin & Zitzewitz, Eric, 2013. "Prediction Markets for Economic Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 657-687, Elsevier.
    11. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    12. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    13. Ho Cheung Brian Lee & Jan Stallaert & Ming Fan, 2020. "Anomalies in Probability Estimates for Event Forecasting on Prediction Markets," Production and Operations Management, Production and Operations Management Society, vol. 29(9), pages 2077-2095, September.
    14. Florian Teschner & David Rothschild & Henner Gimpel, 2017. "Manipulation in Conditional Decision Markets," Group Decision and Negotiation, Springer, vol. 26(5), pages 953-971, September.
    15. Lawrence Choo & Todd R. Kaplan & Ro’i Zultan, 2022. "Manipulation and (Mis)trust in Prediction Markets," Management Science, INFORMS, vol. 68(9), pages 6716-6732, September.
    16. Tsakas, Elias, 2018. "Robust scoring rules," Research Memorandum 023, Maastricht University, Graduate School of Business and Economics (GSBE).
    17. Yiling Chen & Mike Ruberry & Jennifer Wortman Vaughan, 2012. "Designing Informative Securities," Papers 1210.4837, arXiv.org.
    18. Rajiv Sethi & Jennifer Wortman Vaughan, 2016. "Belief Aggregation with Automated Market Makers," Computational Economics, Springer;Society for Computational Economics, vol. 48(1), pages 155-178, June.
    19. Boris Maciejovsky & David V. Budescu, 2020. "Too Much Trust in Group Decisions: Uncovering Hidden Profiles by Groups and Markets," Organization Science, INFORMS, vol. 31(6), pages 1497-1514, November.
    20. Wolfers, Justin & Zitzewitz, Eric, 2006. "Prediction Markets in Theory and Practice," CEPR Discussion Papers 5578, C.E.P.R. Discussion Papers.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:15:y:2018:i:2:p:72-89. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.