IDEAS home Printed from https://ideas.repec.org/a/ijc/ijcjou/y2024q3a2.html
   My bibliography  Save this article

Order Matters: An Experimental Study on How Question Ordering Affects Survey-Based Inflation Forecasts

Author

Listed:
  • Maxime Phillot

    (Swiss National Bank)

  • Rina Rosenblatt-Wisch

    (Swiss National Bank)

Abstract

Policymakers often rely on survey data when gauging expectations. To know the limits of survey data is thus crucial. We look at inflation expectations as measured through the Deloitte CFO Survey Switzerland and respondents’ sensitivity to question ordering thereof. We investigate whether forecast inconsistencies—the discrepancies between point and density forecasts—as well as forecast accuracy change significantly depending on whether the point forecast or the density forecast is asked first. We find that forecast inconsistencies are sizable and order matters. Density forecasts seem to be less affected by question ordering than point forecasts and more accurate than point forecasts.

Suggested Citation

  • Maxime Phillot & Rina Rosenblatt-Wisch, 2024. "Order Matters: An Experimental Study on How Question Ordering Affects Survey-Based Inflation Forecasts," International Journal of Central Banking, International Journal of Central Banking, vol. 20(3), pages 63-114, July.
    • Handle: RePEc:ijc:ijcjou:y:2024:q:3:a:2
    as

    Download full text from publisher

    File URL: http://www.ijcb.org/journal/ijcb24q3a2.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Olivier Coibion & Yuriy Gorodnichenko & Saten Kumar, 2018. "How Do Firms Form Their Expectations? New Survey Evidence," American Economic Review, American Economic Association, vol. 108(9), pages 2671-2713, September.
    2. Charles F. Manski, 2018. "Survey Measurement of Probabilistic Macroeconomic Expectations: Progress and Promise," NBER Macroeconomics Annual, University of Chicago Press, vol. 32(1), pages 411-471.
    3. Manzanares, Andrés & Garcí­a, Juan Angel, 2007. "Reporting biases and survey results: evidence from European professional forecasters," Working Paper Series 836, European Central Bank.
    4. Anthony Tay & Kenneth F. Wallis, 2000. "Density Forecasting: A Survey," Econometric Society World Congress 2000 Contributed Papers 0370, Econometric Society.
    5. Michael Gideon & Brooke Helppie-McFall & Joanne W. Hsu, 2017. "Heaping at Round Numbers on Financial Questions : The Role of Satisficing," Finance and Economics Discussion Series 2017-006, Board of Governors of the Federal Reserve System (U.S.).
    6. Gianna Boero & Jeremy Smith & Kenneth F. Wallis, 2008. "Uncertainty and Disagreement in Economic Prediction: The Bank of England Survey of External Forecasters," Economic Journal, Royal Economic Society, vol. 118(530), pages 1107-1127, July.
    7. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    8. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    9. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    10. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:zbw:bofrdp:037 is not listed on IDEAS
    2. Harvey, David I. & Newbold, Paul, 2003. "The non-normality of some macroeconomic forecast errors," International Journal of Forecasting, Elsevier, vol. 19(4), pages 635-653.
    3. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    4. repec:zbw:bofrdp:2017_037 is not listed on IDEAS
    5. Hartmann, Matthias & Herwartz, Helmut & Ulm, Maren, 2017. "A comparative assessment of alternative ex ante measures of inflation uncertainty," International Journal of Forecasting, Elsevier, vol. 33(1), pages 76-89.
    6. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    7. Terasvirta, Timo, 2006. "Forecasting economic variables with nonlinear models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 8, pages 413-457, Elsevier.
    8. Binder, Carola C., 2017. "Measuring uncertainty based on rounding: New method and application to inflation expectations," Journal of Monetary Economics, Elsevier, vol. 90(C), pages 1-12.
    9. Rianne Legerstee & Philip Hans Franses, 2015. "Does Disagreement Amongst Forecasters Have Predictive Value?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(4), pages 290-302, July.
    10. Balcilar, Mehmet & Katzke, Nico & Gupta, Rangan, 2017. "Do precious metal prices help in forecasting South African inflation?," The North American Journal of Economics and Finance, Elsevier, vol. 40(C), pages 63-72.
    11. Lan Bai & Xiafei Li & Yu Wei & Guiwu Wei, 2022. "Does crude oil futures price really help to predict spot oil price? New evidence from density forecasting," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 27(3), pages 3694-3712, July.
    12. de Mendonça, Helder Ferreira & Simão Filho, José & Abreu, Vanessa Castro, 2023. "Central bank’s forecasts and lack of transparency: An assessment of the effect on private expectations in a large emerging economy," Economic Systems, Elsevier, vol. 47(2).
    13. Francesco D’Acunto & Daniel Hoang & Maritta Paloviita & Michael Weber, 2023. "IQ, Expectations, and Choice," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(5), pages 2292-2325.
    14. Fabian Krüger & Ingmar Nolte, 2011. "Disagreement, Uncertainty and the True Predictive Density," Working Paper Series of the Department of Economics, University of Konstanz 2011-43, Department of Economics, University of Konstanz.
    15. Krüger, Fabian & Nolte, Ingmar, 2016. "Disagreement versus uncertainty: Evidence from distribution forecasts," Journal of Banking & Finance, Elsevier, vol. 72(S), pages 172-186.
    16. Boero, Gianna & Marrocu, Emanuela, 2004. "The performance of SETAR models: a regime conditional evaluation of point, interval and density forecasts," International Journal of Forecasting, Elsevier, vol. 20(2), pages 305-320.
    17. Maxime Phillot & Dr. Rina Rosenblatt-Wisch, 2018. "Inflation Expectations: The Effect of Question Ordering on Forecast Inconsistencies," Working Papers 2018-11, Swiss National Bank.
    18. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    19. Brückbauer, Frank, 2022. "Do financial market experts know their theory? New evidence from survey data," ZEW Discussion Papers 20-092, ZEW - Leibniz Centre for European Economic Research, revised 2022.
    20. Oscar Claveria, 2021. "On the Aggregation of Survey-Based Economic Uncertainty Indicators Between Different Agents and Across Variables," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 17(1), pages 1-26, April.
    21. Hecq, A.W. & Götz, T.B. & Urbain, J.R.Y.J., 2012. "Real-time forecast density combinations (forecasting US GDP growth using mixed-frequency data)," Research Memorandum 021, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    22. Clements, Michael P., 2014. "Probability distributions or point predictions? Survey forecasts of US output growth and inflation," International Journal of Forecasting, Elsevier, vol. 30(1), pages 99-117.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ijc:ijcjou:y:2024:q:3:a:2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bank for International Settlements (email available below). General contact details of provider: https://www.ijcb.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.