IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/67179.html
   My bibliography  Save this paper

Testing best practices to reduce the overconfidence bias in multi-criteria decision analysis

Author

Listed:
  • Ferretti, Valentina
  • Guney, Sule
  • Montibeller, Gilberto
  • Winterfeldt, Detlof von

Abstract

This paper explores the effectiveness of several methods to reduce the overconfidence bias when eliciting continuous probability distributions in the context of multicriteria decision analysis. We examine the effectiveness of using a fixed value method (as opposed to the standard fixed probability method) and the use of counterfactuals and hypothetical bets to increase the range of the distributions and to correct possible median displacements. The results show that the betting procedure to correct the median is quite effective, but the methods to increase the range of estimates have only a have small, but positive effect.

Suggested Citation

  • Ferretti, Valentina & Guney, Sule & Montibeller, Gilberto & Winterfeldt, Detlof von, 2016. "Testing best practices to reduce the overconfidence bias in multi-criteria decision analysis," LSE Research Online Documents on Economics 67179, London School of Economics and Political Science, LSE Library.
    • Handle: RePEc:ehl:lserod:67179
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/67179/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Klayman, Joshua & Soll, Jack B. & Gonzalez-Vallejo, Claudia & Barlas, Sema, 1999. "Overconfidence: It Depends on How, What, and Whom You Ask, , , , , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 79(3), pages 216-247, September.
    2. James E. Matheson & Robert L. Winkler, 1976. "Scoring Rules for Continuous Probability Distributions," Management Science, INFORMS, vol. 22(10), pages 1087-1096, June.
    3. Itzhak Ben-David & John R. Graham, 2013. "Managerial Miscalibration," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(4), pages 1547-1584.
    4. repec:cup:judgdm:v:5:y:2010:i:7:p:467-476 is not listed on IDEAS
    5. repec:bla:jfinan:v:53:y:1998:i:6:p:1839-1885 is not listed on IDEAS
    6. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    7. Larrick, Richard P. & Burson, Katherine A. & Soll, Jack B., 2007. "Social comparison and confidence: When thinking you're better than average predicts overconfidence (and when it does not)," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 76-94, January.
    8. Subbotin, Vadim, 1996. "Outcome Feedback Effects on Under- and Overconfident Judgments (General Knowledge Tasks)," Organizational Behavior and Human Decision Processes, Elsevier, vol. 66(3), pages 268-276, June.
    9. Robert T. Clemen & Canan Ulu, 2008. "Interior Additivity and Subjective Probability Assessment of Continuous Variables," Management Science, INFORMS, vol. 54(4), pages 835-851, April.
    10. Terrance Odean, 1999. "Do Investors Trade Too Much?," American Economic Review, American Economic Association, vol. 89(5), pages 1279-1298, December.
    11. McKenzie, Craig R.M. & Liersch, Michael J. & Yaniv, Ilan, 2008. "Overconfidence in interval estimates: What does expertise buy you?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(2), pages 179-191, November.
    12. Du, Ning & Budescu, David V. & Shelly, Marjorie K. & Omer, Thomas C., 2011. "The appeal of vague financial forecasts," Organizational Behavior and Human Decision Processes, Elsevier, vol. 114(2), pages 179-189, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gerda Ana Melnik-Leroy & Gintautas Dzemyda, 2021. "How to Influence the Results of MCDM?—Evidence of the Impact of Cognitive Biases," Mathematics, MDPI, vol. 9(2), pages 1-25, January.
    2. Jared A. Beekman & Ronald F. A. Woodaman & Dennis M. Buede, 2020. "A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat Detection," Decision Analysis, INFORMS, vol. 17(1), pages 39-55, March.
    3. Alessia Buda & Ernst Jan de Place Hansen & Alexander Rieser & Emanuela Giancola & Valeria Natalina Pracchi & Sara Mauri & Valentina Marincioni & Virginia Gori & Kalliopi Fouseki & Cristina S. Polo Lóp, 2021. "Conservation-Compatible Retrofit Solutions in Historic Buildings: An Integrated Approach," Sustainability, MDPI, vol. 13(5), pages 1-19, March.
    4. Ferretti, Valentina & Montibeller, Gilberto & von Winterfeldt, Detlof, 2023. "Testing the effectiveness of debiasing techniques to reduce overprecision in the elicitation of subjective continuous probability distributions," European Journal of Operational Research, Elsevier, vol. 304(2), pages 661-675.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markus Glaser & Martin Weber, 2007. "Overconfidence and trading volume," The Geneva Papers on Risk and Insurance Theory, Springer;International Association for the Study of Insurance Economics (The Geneva Association), vol. 32(1), pages 1-36, June.
    2. repec:cup:judgdm:v:12:y:2017:i:1:p:29-41 is not listed on IDEAS
    3. Julia P. Prims & Don A. Moore, 2017. "Overconfidence over the lifespan," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 12(1), pages 29-41, January.
    4. Michał Krawczyk, 2011. "Overconfident for real? Proper scoring for confidence intervals," Working Papers 2011-15, Faculty of Economic Sciences, University of Warsaw.
    5. Don A. Moore & Samuel A. Swift & Angela Minster & Barbara Mellers & Lyle Ungar & Philip Tetlock & Heather H. J. Yang & Elizabeth R. Tenney, 2017. "Confidence Calibration in a Multiyear Geopolitical Forecasting Competition," Management Science, INFORMS, vol. 63(11), pages 3552-3565, November.
    6. Katharina Dowling & Daniel Guhl & Daniel Klapper & Martin Spann & Lucas Stich & Narine Yegoryan, 2020. "Behavioral biases in marketing," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 449-477, May.
    7. Johannes Brunzel, 2021. "Overconfidence and narcissism among the upper echelons: a systematic literature review," Management Review Quarterly, Springer, vol. 71(3), pages 585-623, July.
    8. Merkle, Christoph, 2018. "The curious case of negative volatility," Journal of Financial Markets, Elsevier, vol. 40(C), pages 92-108.
    9. Merkle, Christoph, 2017. "Financial overconfidence over time: Foresight, hindsight, and insight of investors," Journal of Banking & Finance, Elsevier, vol. 84(C), pages 68-87.
    10. Glaser, Markus & Nöth, Markus & Weber, Martin, 2003. "Behavioral finance," Papers 03-14, Sonderforschungsbreich 504.
    11. Vetter, J. & Benlian, Alexander & Hess, T., 2011. "Overconfidence in IT Investment Decisions: Why Knowledge can be Boon and Bane at the same Time," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 58030, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    12. Bruhin, Adrian & Santos-Pinto, Luís & Staubli, David, 2018. "How do beliefs about skill affect risky decisions?," Journal of Economic Behavior & Organization, Elsevier, vol. 150(C), pages 350-371.
    13. Christian Schumacher & Steffen Keck & Wenjie Tang, 2020. "Biased interpretation of performance feedback: The role of CEO overconfidence," Strategic Management Journal, Wiley Blackwell, vol. 41(6), pages 1139-1165, June.
    14. Daniel Fonseca Costa & Francisval Carvalho & Bruno César Moreira & José Willer Prado, 2017. "Bibliometric analysis on the association between behavioral finance and decision making with cognitive biases such as overconfidence, anchoring effect and confirmation bias," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1775-1799, June.
    15. Niculaescu, Corina E. & Sangiorgi, Ivan & Bell, Adrian R., 2023. "Does personal experience with COVID-19 impact investment decisions? Evidence from a survey of US retail investors," International Review of Financial Analysis, Elsevier, vol. 88(C).
    16. Youki Kohsaka & Grzegorz Mardyla & Shinji Takenaka & Yoshiro Tsutsui, 2017. "Disposition Effect and Diminishing Sensitivity: An Analysis Based on a Simulated Experimental Stock Market," Journal of Behavioral Finance, Taylor & Francis Journals, vol. 18(2), pages 189-201, April.
    17. Barberis, Nicholas & Xiong, Wei, 2012. "Realization utility," Journal of Financial Economics, Elsevier, vol. 104(2), pages 251-271.
    18. Fildes, Robert & Goodwin, Paul & Onkal, Dilek, 2015. "Information use in supply chain forecasting," MPRA Paper 66034, University Library of Munich, Germany.
    19. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    20. Camille Magron & Maxime Merli, 2012. "Stocks repurchase and sophistication of individual investors," Working Papers of LaRGE Research Center 2012-02, Laboratoire de Recherche en Gestion et Economie (LaRGE), Université de Strasbourg.
    21. Ricardo Crisóstomo, 2021. "Estimating real‐world probabilities: A forward‐looking behavioral framework," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 41(11), pages 1797-1823, November.

    More about this item

    JEL classification:

    • J50 - Labor and Demographic Economics - - Labor-Management Relations, Trade Unions, and Collective Bargaining - - - General
    • G32 - Financial Economics - - Corporate Finance and Governance - - - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:67179. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.