IDEAS home Printed from https://ideas.repec.org/p/mib/wpaper/427.html
   My bibliography  Save this paper

Uncertainty measures from partially rounded probabilistic forecast surveys

Author

Listed:
  • Alexander Glas
  • Matthias Hartmann

Abstract

Although survey-based point predictions have been found to outperform successful forecasting models, corresponding variance forecasts are frequently diagnosed as heavily distorted. Forecasters who report inconspicuously low ex-ante variances often produce squared forecast errors that are much larger on average. In this paper, we document the novel stylized fact that this variance misalignment is related to the rounding behavior of survey participants. Rounding may reflect the fact that some survey participants employ a rather judgmental approach to forecasting as opposed to using a formal model. We use the distinct numerical accuracies of panelists' reported probabilities as a means to propose several alternative and easily implementable corrections that i) can be carried out in real time, i.e., before outcomes are observed, and ii) deliver a significantly improved match between ex-ante and ex-post forecast uncertainty. According to our estimates, uncertainty about inflation, output growth and unemployment in the U.S. and the Euro area is higher after correcting for the rounding effect. The increase in the share of non-rounded responses in recent years also helps to understand the trajectory of survey-based average uncertainty during the years since the financial and sovereign debt crisis.

Suggested Citation

  • Alexander Glas & Matthias Hartmann, 2020. "Uncertainty measures from partially rounded probabilistic forecast surveys," Working Papers 427, University of Milano-Bicocca, Department of Economics, revised Jan 2020.
  • Handle: RePEc:mib:wpaper:427
    as

    Download full text from publisher

    File URL: http://repec.dems.unimib.it/repec/pdf/mibwpaper427.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kajal Lahiri & Huaming Peng & Xuguang Simon Sheng, 2022. "Measuring Uncertainty of a Combined Forecast and Some Tests for Forecaster Heterogeneity," Advances in Econometrics, in: Essays in Honor of M. Hashem Pesaran: Prediction and Macro Modeling, volume 43, pages 29-50, Emerald Group Publishing Limited.
    2. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    3. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    4. Soojin Jo & Rodrigo Sekkel, 2019. "Macroeconomic Uncertainty Through the Lens of Professional Forecasters," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(3), pages 436-446, July.
    5. Glas, Alexander & Hartmann, Matthias, 2016. "Inflation uncertainty, disagreement and monetary policy: Evidence from the ECB Survey of Professional Forecasters," Journal of Empirical Finance, Elsevier, vol. 39(PB), pages 215-228.
    6. Glas, Alexander, 2020. "Five dimensions of the uncertainty–disagreement linkage," International Journal of Forecasting, Elsevier, vol. 36(2), pages 607-627.
    7. Geoff Kenny & Thomas Kostka & Federico Masera, 2014. "How Informative are the Subjective Density Forecasts of Macroeconomists?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(3), pages 163-185, April.
    8. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1163-1212, May.
    9. Faust, Jon & Wright, Jonathan H., 2009. "Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 468-479.
    10. Joshua Abel & Robert Rich & Joseph Song & Joseph Tracy, 2016. "The Measurement and Behavior of Uncertainty: Evidence from the ECB Survey of Professional Forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 533-550, April.
    11. Paul Ruud & Daniel Schunk & Joachim Winter, 2014. "Uncertainty causes rounding: an experimental study," Experimental Economics, Springer;Economic Science Association, vol. 17(3), pages 391-413, September.
    12. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    13. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    14. Michael P. Clements, 2011. "An Empirical Investigation of the Effects of Rounding on the SPF Probabilities of Decline and Output Growth Histograms," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43(1), pages 207-220, February.
    15. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    16. Kajal Lahiri & Xuguang Sheng, 2010. "Measuring forecast uncertainty by disagreement: The missing link," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 514-538.
    17. Binder, Carola C., 2017. "Measuring uncertainty based on rounding: New method and application to inflation expectations," Journal of Monetary Economics, Elsevier, vol. 90(C), pages 1-12.
    18. Michael Clements, 2006. "Evaluating the survey of professional forecasters probability distributions of expected inflation based on derived event probability forecasts," Empirical Economics, Springer, vol. 31(1), pages 49-64, March.
    19. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    20. Boero, Gianna & Smith, Jeremy & Wallis, Kenneth F., 2011. "Scoring rules and survey density forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 379-393.
    21. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    22. Michael P. Clements, 2014. "Forecast Uncertainty- Ex Ante and Ex Post : U.S. Inflation and Output Growth," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(2), pages 206-216, April.
    23. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    24. Casey, Eddie, 2021. "Are professional forecasters overconfident?," International Journal of Forecasting, Elsevier, vol. 37(2), pages 716-732.
    25. Lahiri, Kajal & Sheng, Xuguang, 2008. "Evolution of forecast disagreement in a Bayesian learning model," Journal of Econometrics, Elsevier, vol. 144(2), pages 325-340, June.
    26. Meyler, Aidan, 2020. "Forecast performance in the ECB SPF: ability or chance?," Working Paper Series 2371, European Central Bank.
    27. Gianna Boero & Jeremy Smith & Kenneth F. Wallis, 2015. "The Measurement and Characteristics of Professional Forecasters' Uncertainty," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1029-1046, November.
    28. Giordani, Paolo & Soderlind, Paul, 2006. "Is there evidence of pessimism and doubt in subjective distributions? Implications for the equity premium puzzle," Journal of Economic Dynamics and Control, Elsevier, vol. 30(6), pages 1027-1043, June.
    29. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    30. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Glas, Alexander & Heinisch, Katja, 2023. "Conditional macroeconomic survey forecasts: Revisions and errors," Journal of International Money and Finance, Elsevier, vol. 138(C).
    2. Becker, Christoph & Dürsch, Peter & Eife, Thomas A. & Glas, Alexander, 2023. "Households' probabilistic inflation expectations in high-inflation regimes," ZEW Discussion Papers 23-072, ZEW - Leibniz Centre for European Economic Research.
    3. Conrad, Christian & Lahiri, Kajal, 2023. "Heterogeneous expectations among professional forecasters," ZEW Discussion Papers 23-062, ZEW - Leibniz Centre for European Economic Research.
    4. Todd E. Clark & Gergely Ganics & Elmar Mertens, 2022. "What is the Predictive Value of SPF Point and Density Forecasts?," Working Papers 22-37, Federal Reserve Bank of Cleveland.
    5. Czudaj, Robert L., 2023. "Anchoring of Inflation Expectations and the Role of Monetary Policy and Cost-Push Factors," MPRA Paper 119029, University Library of Munich, Germany.
    6. Czudaj, Robert L., 2023. "Expectation Formation and the Phillips Curve Revisited," MPRA Paper 119478, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    2. Glas, Alexander, 2020. "Five dimensions of the uncertainty–disagreement linkage," International Journal of Forecasting, Elsevier, vol. 36(2), pages 607-627.
    3. repec:zbw:bofrdp:037 is not listed on IDEAS
    4. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    5. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    6. Kajal Lahiri & Huaming Peng & Xuguang Simon Sheng, 2022. "Measuring Uncertainty of a Combined Forecast and Some Tests for Forecaster Heterogeneity," Advances in Econometrics, in: Essays in Honor of M. Hashem Pesaran: Prediction and Macro Modeling, volume 43, pages 29-50, Emerald Group Publishing Limited.
    7. repec:zbw:bofrdp:2017_037 is not listed on IDEAS
    8. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    9. Casey, Eddie, 2021. "Are professional forecasters overconfident?," International Journal of Forecasting, Elsevier, vol. 37(2), pages 716-732.
    10. Fabian Krüger, 2017. "Survey-based forecast distributions for Euro Area growth and inflation: ensembles versus histograms," Empirical Economics, Springer, vol. 53(1), pages 235-246, August.
    11. Geoff Kenny & Thomas Kostka & Federico Masera, 2015. "Density characteristics and density forecast performance: a panel analysis," Empirical Economics, Springer, vol. 48(3), pages 1203-1231, May.
    12. Robert Rich & Joseph Tracy, 2021. "A Closer Look at the Behavior of Uncertainty and Disagreement: Micro Evidence from the Euro Area," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 53(1), pages 233-253, February.
    13. Krüger, Fabian & Nolte, Ingmar, 2016. "Disagreement versus uncertainty: Evidence from distribution forecasts," Journal of Banking & Finance, Elsevier, vol. 72(S), pages 172-186.
    14. Clements, Michael P, 2012. "Subjective and Ex Post Forecast Uncertainty : US Inflation and Output Growth," The Warwick Economics Research Paper Series (TWERPS) 995, University of Warwick, Department of Economics.
    15. Rossi, Barbara & Ganics, Gergely & Sekhposyan, Tatevik, 2020. "From Fixed-event to Fixed-horizon Density Forecasts: Obtaining Measures of Multi-horizon Uncertainty from Survey Density Foreca," CEPR Discussion Papers 14267, C.E.P.R. Discussion Papers.
    16. Gergely Ganics & Barbara Rossi & Tatevik Sekhposyan, 2019. "From fixed-event to fixed-horizon density forecasts: Obtaining measures of multi-horizon uncertainty from survey density forecasts," Economics Working Papers 1689, Department of Economics and Business, Universitat Pompeu Fabra.
    17. Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 16 Nov 2020.
    18. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    19. Malte Knüppel & Fabian Krüger, 2022. "Forecast uncertainty, disagreement, and the linear pool," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 23-41, January.
    20. Joshua Abel & Robert Rich & Joseph Song & Joseph Tracy, 2016. "The Measurement and Behavior of Uncertainty: Evidence from the ECB Survey of Professional Forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 533-550, April.
    21. Huang, Rong & Pilbeam, Keith & Pouliot, William, 2022. "Are macroeconomic forecasters optimists or pessimists? A reassessment of survey based forecasts," Journal of Economic Behavior & Organization, Elsevier, vol. 197(C), pages 706-724.
    22. Todd E. Clark & Michael W. McCracken & Elmar Mertens, 2020. "Modeling Time-Varying Uncertainty of Multiple-Horizon Forecast Errors," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 17-33, March.

    More about this item

    Keywords

    Survey data; probabilistic forecasting; rounding; macroeconomic uncertainty.;
    All these keywords.

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mib:wpaper:427. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Matteo Pelagatti (email available below). General contact details of provider: https://edirc.repec.org/data/dpmibit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.