IDEAS home Printed from https://ideas.repec.org/a/spr/decfin/v37y2014i2p423-452.html
   My bibliography  Save this article

Measuring and adjusting for overconfidence

Author

Listed:
  • P. Schanbacher

Abstract

To evaluate density forecasts, the applied scoring rule is often arbitrarily chosen. The selection of the scoring rule strongly influences the ranking of forecasts. This paper identifies overconfidence as the main driver for scoring differences. A novel approach to measure overconfidence is proposed. Based on a non-proper scoring rule, the forecasts can be individually adjusted toward a calibrated forecast. Applying the adjustment procedure to the survey of professional forecasters, it can be shown that out-of-sample forecasts can be significantly improved. Also the ranking of the adjusted forecasts becomes less sensitive to the selection of scoring rules. Copyright Springer-Verlag Italia 2014

Suggested Citation

  • P. Schanbacher, 2014. "Measuring and adjusting for overconfidence," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 37(2), pages 423-452, October.
    • Handle: RePEc:spr:decfin:v:37:y:2014:i:2:p:423-452
    DOI: 10.1007/s10203-013-0153-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10203-013-0153-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10203-013-0153-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    2. Gilles Hilary & Lior Menzly, 2006. "Does Past Success Lead Analysts to Become Overconfident?," Management Science, INFORMS, vol. 52(4), pages 489-500, April.
    3. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    4. William F. Wright, 1988. "Empirical comparison of subjective probability elicitation methods," Contemporary Accounting Research, John Wiley & Sons, vol. 5(1), pages 47-57, September.
    5. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    6. Shiller, Robert J & Kon-Ya, Fumiko & Tsutsui, Yoshiro, 1996. "Why Did the Nikkei Crash? Expanding the Scope of Expectations Data Collection," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 156-164, February.
    7. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    8. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    9. Soll, Jack B., 1996. "Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure," Organizational Behavior and Human Decision Processes, Elsevier, vol. 65(2), pages 117-137, February.
    10. Craig Burnside & Bing Han & David Hirshleifer & Tracy Yue Wang, 2011. "Investor Overconfidence and the Forward Premium Puzzle," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(2), pages 523-558.
    11. Ghysels, Eric & Wright, Jonathan H., 2009. "Forecasting Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 504-516.
    12. Roopesh Ranjan & Tilmann Gneiting, 2010. "Combining probability forecasts," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 71-91, January.
    13. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    14. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    15. Fischer, Stanley, 1995. "Central-Bank Independence Revisited," American Economic Review, American Economic Association, vol. 85(2), pages 201-206, May.
    16. J. Eric Bickel, 2007. "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules," Decision Analysis, INFORMS, vol. 4(2), pages 49-65, June.
    17. Giordani, Paolo & Soderlind, Paul, 2006. "Is there evidence of pessimism and doubt in subjective distributions? Implications for the equity premium puzzle," Journal of Economic Dynamics and Control, Elsevier, vol. 30(6), pages 1027-1043, June.
    18. Kenneth F. Wallis, 2005. "Combining Density and Interval Forecasts: A Modest Proposal," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 983-994, December.
    19. Reinhard Selten, 1998. "Axiomatic Characterization of the Quadratic Scoring Rule," Experimental Economics, Springer;Economic Science Association, vol. 1(1), pages 43-61, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chambers, Christopher P. & Healy, Paul J. & Lambert, Nicolas S., 2019. "Proper scoring rules with general preferences: A dual characterization of optimal reports," Games and Economic Behavior, Elsevier, vol. 117(C), pages 322-341.
    2. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 16 Nov 2020.
    5. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    6. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    7. de Haan, Thomas, 2020. "Eliciting belief distributions using a random two-level partitioning of the state space," Working Papers in Economics 1/20, University of Bergen, Department of Economics.
    8. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    9. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2016. "In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments," Theory and Decision, Springer, vol. 80(3), pages 363-387, March.
    10. Tommaso Proietti & Martyna Marczak & Gianluigi Mazzi, 2017. "Euromind‐ D : A Density Estimate of Monthly Gross Domestic Product for the Euro Area," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(3), pages 683-703, April.
    11. Tony Chernis & Taylor Webley, 2022. "Nowcasting Canadian GDP with Density Combinations," Discussion Papers 2022-12, Bank of Canada.
    12. Yanwei Jia & Jussi Keppo & Ville Satopää, 2023. "Herding in Probabilistic Forecasts," Management Science, INFORMS, vol. 69(5), pages 2713-2732, May.
    13. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207, April.
    14. Tsakas, Elias, 2018. "Robust scoring rules," Research Memorandum 023, Maastricht University, Graduate School of Business and Economics (GSBE).
    15. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    16. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207.
    17. Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Robert L. Winkler, 2013. "Is It Better to Average Probabilities or Quantiles?," Management Science, INFORMS, vol. 59(7), pages 1594-1611, July.
    18. Becchetti, Leonardo & Degli Antoni, Giacomo & Ottone, Stefania & Solferino, Nazaria, 2013. "Allocation criteria under task performance: The gendered preference for protection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 45(C), pages 96-111.
    19. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    20. Peysakhovich, Alexander & Plagborg-Møller, Mikkel, 2012. "A note on proper scoring rules and risk aversion," Economics Letters, Elsevier, vol. 117(1), pages 357-361.

    More about this item

    Keywords

    Belief measurement; Proper scoring rules; Overconfidence; Probability adjustment; C53; E37; D81;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:decfin:v:37:y:2014:i:2:p:423-452. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.