IDEAS home Printed from https://ideas.repec.org/p/zbw/pp1859/300847.html
   My bibliography  Save this paper

Inefficient forecast narratives: A BERT-based approach

Author

Listed:
  • Foltas, Alexander

Abstract

I contribute to previous research on the efficient integration of forecasters' narratives into business cycle forecasts. Using a Bidirectional Encoder Representations from Transformers (BERT) model, I quantify 19,300 paragraphs from German business cycle reports (1998-2021) and classify the signs of institutes' consumption forecast errors. The correlation is strong for 12.8% of paragraphs with a predicted class probability of 85% or higher. Reviewing 150 of such high-probability paragraphs reveals recurring narratives. Underestimations of consumption growth often mention rising employment, increasing wages and transfer payments, low inflation, decreasing taxes, crisis-related fiscal support, and reduced relevance of marginal employment. Conversely, overestimated consumption forecasts present opposing narratives. Forecasters appear to particularly underestimate these factors when they disproportionately affect low-income households.

Suggested Citation

  • Foltas, Alexander, 2024. "Inefficient forecast narratives: A BERT-based approach," Working Papers 45, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
  • Handle: RePEc:zbw:pp1859:300847
    DOI: 10.18452/29133
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/300847/1/1897704313.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.18452/29133?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Dovern, Jonas & Weisser, Johannes, 2011. "Accuracy, unbiasedness and efficiency of professional macroeconomic forecasts: An empirical comparison for the G7," International Journal of Forecasting, Elsevier, vol. 27(2), pages 452-465.
    2. Clements, Michael P. & Reade, J. James, 2020. "Forecasting and forecast narratives: The Bank of England Inflation Reports," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1488-1500.
    3. Pierdzioch, Christian & Rülke, Jan-Christoph & Tillmann, Peter, 2016. "Using Forecasts To Uncover The Loss Function Of Federal Open Market Committee Members," Macroeconomic Dynamics, Cambridge University Press, vol. 20(3), pages 791-818, April.
    4. Gabe de Bondt & Arne Gieseck & Pablo Herrero & Zivile Zekaite, 2021. "Euro Area Income and Wealth Effects: Aggregation Issues," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(6), pages 1454-1474, December.
    5. Holtemöller, Oliver & Pohle, Felix, 2020. "Employment effects of introducing a minimum wage: The case of Germany," Economic Modelling, Elsevier, vol. 89(C), pages 108-121.
    6. Heinisch Katja & Behrens Christoph & Döpke Jörg & Foltas Alexander & Fritsche Ulrich & Köhler Tim & Müller Karsten & Puckelwald Johannes & Reichmayr Hannes, 2024. "The IWH Forecasting Dashboard: From Forecasts to Evaluation and Comparison," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 244(3), pages 277-288, June.
    7. Patton, Andrew J. & Timmermann, Allan, 2007. "Testing Forecast Optimality Under Unknown Loss," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1172-1184, December.
    8. Karl Brenke, 2009. "Real Wages in Germany: Numerous Years of Decline," Weekly Report, DIW Berlin, German Institute for Economic Research, vol. 5(28), pages 193-202.
    9. Krüger, Jens & Hoss, Julian, 2012. "German Business Cycle Forecasts, Asymmetric Loss and Financial Variables," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 63652, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    10. Alexander Foltas & Christian Pierdzioch, 2022. "Business-cycle reports and the efficiency of macroeconomic forecasts for Germany," Applied Economics Letters, Taylor & Francis Journals, vol. 29(10), pages 867-872, June.
    11. Krüger, Jens J. & Hoss, Julian, 2012. "German business cycle forecasts, asymmetric loss and financial variables," Economics Letters, Elsevier, vol. 114(3), pages 284-287.
    12. Bruno Deschamps & Paolo Bianchi, 2012. "An evaluation of Chinese macroeconomic forecasts," Journal of Chinese Economic and Business Studies, Taylor & Francis Journals, vol. 10(3), pages 229-246, December.
    13. Bruttel, Oliver, 2019. "The effects of the new statutory minimum wage in Germany: a first assessment of the evidence," Journal for Labour Market Research, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany], vol. 53(1), pages 1-10.
    14. repec:iab:iabjlr:v:53:i:1:p:art.10 is not listed on IDEAS
    15. Eicher, Theo S. & Kuenzel, David J. & Papageorgiou, Chris & Christofides, Charis, 2019. "Forecasts in times of crises," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1143-1159.
    16. Sharpe, Steven A. & Sinha, Nitish R. & Hollrah, Christopher A., 2023. "The power of narrative sentiment in economic forecasts," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1097-1121.
    17. Gabe Jacob de Bondt & Arne Gieseck & Zivile Zekaite, 2020. "Thick modelling income and wealth effects: a forecast application to euro area private consumption," Empirical Economics, Springer, vol. 58(1), pages 257-286, January.
    18. Harry X. Wu, 2007. "The Chinese GDP Growth Rate Puzzle: How Fast Has the Chinese Economy Grown?," Asian Economic Papers, MIT Press, vol. 6(1), pages 1-23, Winter.
    19. Stekler, Herman & Symington, Hilary, 2016. "Evaluating qualitative forecasts: The FOMC minutes, 2006–2010," International Journal of Forecasting, Elsevier, vol. 32(2), pages 559-570.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Behrens, Christoph & Pierdzioch, Christian & Risse, Marian, 2018. "Testing the optimality of inflation forecasts under flexible loss with random forests," Economic Modelling, Elsevier, vol. 72(C), pages 270-277.
    2. Foltas, Alexander, 2023. "Quantifying priorities in business cycle reports: Analysis of recurring textual patterns around peaks and troughs," Working Papers 44, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    3. Karsten Müller, 2022. "German forecasters’ narratives: How informative are German business cycle forecast reports?," Empirical Economics, Springer, vol. 62(5), pages 2373-2415, May.
    4. Müller, Karsten, 2020. "German forecasters' narratives: How informative are German business cycle forecast reports?," Working Papers 23, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    5. Ilias Filippou & James Mitchell & My T. Nguyen, 2023. "The FOMC versus the Staff: Do Policymakers Add Value in Their Tales?," Working Papers 23-20, Federal Reserve Bank of Cleveland.
    6. Giovannelli, Alessandro & Pericoli, Filippo Maria, 2020. "Are GDP forecasts optimal? Evidence on European countries," International Journal of Forecasting, Elsevier, vol. 36(3), pages 963-973.
    7. Jens J. Krüger, 2014. "A multivariate evaluation of German output growth and inflation forecasts," Economics Bulletin, AccessEcon, vol. 34(3), pages 1410-1418.
    8. Tsuchiya, Yoichi, 2016. "Asymmetric loss and rationality of Chinese renminbi forecasts: An implication for the trade between China and the US," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 44(C), pages 116-127.
    9. Frenkel, Michael & Rülke, Jan-Christoph & Zimmermann, Lilli, 2013. "Do private sector forecasters chase after IMF or OECD forecasts?," Journal of Macroeconomics, Elsevier, vol. 37(C), pages 217-229.
    10. Mihaela Simionescu, 2014. "What Type Of Social Capital Is Engaged By The French Dairy Stockbreeders? A Characterization Through Their Professional Identities," Romanian Journal of Regional Science, Romanian Regional Science Association, vol. 8(1), pages 87-102, JUNE.
    11. Dovern, Jonas & Jannsen, Nils, 2017. "Systematic errors in growth expectations over the business cycle," International Journal of Forecasting, Elsevier, vol. 33(4), pages 760-769.
    12. Jakub Rybacki & Michał Gniazdowski, 2023. "Macroeconomic forecasting in Poland: lessons from the external shocks," Bank i Kredyt, Narodowy Bank Polski, vol. 54(1), pages 45-64.
    13. Marcos Bujosa & Antonio García‐Ferrer & Aránzazu de Juan & Antonio Martín‐Arroyo, 2020. "Evaluating early warning and coincident indicators of business cycles using smooth trends," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(1), pages 1-17, January.
    14. Marco Caliendo & Nico Pestel & Rebecca Olthaus, 2023. "Long-Term Employment Effects of the Minimum Wage in Germany: New Data and Estimators," Papers 2310.15964, arXiv.org.
    15. Mihaela Bratu, 2012. "A Strategy to Improve the Survey of Professional Forecasters (SPF) Predictions Using Bias-Corrected-Accelerated (BCA) Bootstrap Forecast Intervals," International Journal of Synergy and Research, ToKnowPress, vol. 1(2), pages 45-59.
    16. Carlos Cañizares Martínez & Gabe J. de Bondt & Arne Gieseck, 2023. "Forecasting housing investment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(3), pages 543-565, April.
    17. Jones, Jacob T. & Sinclair, Tara M. & Stekler, Herman O., 2020. "A textual analysis of Bank of England growth forecasts," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1478-1487.
    18. Jennifer L. Castle & David F. Hendry & Andrew B. Martinez, 2017. "Evaluating Forecasts, Narratives and Policy Using a Test of Invariance," Econometrics, MDPI, vol. 5(3), pages 1-27, September.
    19. Tsuchiya, Yoichi, 2016. "Assessing macroeconomic forecasts for Japan under an asymmetric loss function," International Journal of Forecasting, Elsevier, vol. 32(2), pages 233-242.
    20. Strunz, Franziska & Gödl, Maximilian, 2023. "An Evaluation of Professional Forecasts for the German Economy," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277707, Verein für Socialpolitik / German Economic Association.

    More about this item

    Keywords

    Macroeconomic forecasting; Evaluating forecasts; Business cycles; Consumption forecasting; Natural language processing; Language Modeling; Machine learning; Judgemental forecasting;
    All these keywords.

    JEL classification:

    • E21 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Consumption; Saving; Wealth
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:pp1859:300847. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.experience-expectation.de/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.