IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v167y2012i2p358-369.html
   My bibliography  Save this article

Bayesian averaging, prediction and nonnested model selection

Author

Listed:
  • Hong, Han
  • Preston, Bruce

Abstract

This paper studies the asymptotic relationship between Bayesian model averaging and post-selection frequentist predictors in both nested and nonnested models. We derive conditions under which their difference is of a smaller order of magnitude than the inverse of the square root of the sample size in large samples. This result depends crucially on the relation between posterior odds and frequentist model selection criteria. Weak conditions are given under which consistent model selection is feasible, regardless of whether models are nested or nonnested and regardless of whether models are correctly specified or not, in the sense that they select the best model with the least number of parameters with probability converging to 1. Under these conditions, Bayesian posterior odds and BICs are consistent for selecting among nested models, but are not consistent for selecting among nonnested models and possibly overlapping models. These findings have important bearing for applied researchers who are frequent users of model selection tools for empirical investigation of model predictions.

Suggested Citation

  • Hong, Han & Preston, Bruce, 2012. "Bayesian averaging, prediction and nonnested model selection," Journal of Econometrics, Elsevier, vol. 167(2), pages 358-369.
  • Handle: RePEc:eee:econom:v:167:y:2012:i:2:p:358-369
    DOI: 10.1016/j.jeconom.2011.09.021
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407611002028
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jeconom.2011.09.021?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1059-1087.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Sin, Chor-Yiu & White, Halbert, 1996. "Information criteria for selecting possibly misspecified parametric models," Journal of Econometrics, Elsevier, vol. 71(1-2), pages 207-225.
    4. Gourieroux,Christian & Monfort,Alain, 1995. "Statistics and Econometric Models," Cambridge Books, Cambridge University Press, number 9780521471626, October.
    5. Carroll Christopher Dixon, 2001. "Death to the Log-Linearized Consumption Euler Equation! (And Very Poor Health to the Second-Order Approximation)," The B.E. Journal of Macroeconomics, De Gruyter, vol. 1(1), pages 1-38, April.
    6. Chernozhukov, Victor & Hong, Han, 2003. "An MCMC approach to classical estimation," Journal of Econometrics, Elsevier, vol. 115(2), pages 293-346, August.
    7. Lubik, Thomas A. & Schorfheide, Frank, 2007. "Do central banks respond to exchange rate movements? A structural investigation," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1069-1087, May.
    8. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    9. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
    10. Juan F. Rubio-Ramirez & Jesus Fernández-Villaverde, 2005. "Estimating dynamic equilibrium economies: linear versus nonlinear likelihood," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(7), pages 891-910.
    11. Wright, Jonathan H., 2008. "Bayesian Model Averaging and exchange rate forecasts," Journal of Econometrics, Elsevier, vol. 146(2), pages 329-341, October.
    12. Pakes, Ariel & Pollard, David, 1989. "Simulation and the Asymptotics of Optimization Estimators," Econometrica, Econometric Society, vol. 57(5), pages 1027-1057, September.
    13. Jesús Fernández-Villaverde & Juan F. Rubio-Ramirez, 2004. "Estimating nonlinear dynamic equilibrium economies: a likelihood approach," FRB Atlanta Working Paper 2004-1, Federal Reserve Bank of Atlanta.
    14. Justiniano, Alejandro & Preston, Bruce, 2010. "Can structural small open-economy models account for the influence of foreign disturbances?," Journal of International Economics, Elsevier, vol. 81(1), pages 61-74, May.
    15. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 4, pages 135-196, Elsevier.
    16. Orazio P. Attanasio & Hamish Low, 2004. "Estimating Euler Equations," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 7(2), pages 405-435, April.
    17. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    18. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    19. Hong, Han & Preston, Bruce & Shum, Matthew, 2003. "Generalized Empirical Likelihood–Based Model Selection Criteria For Moment Condition Models," Econometric Theory, Cambridge University Press, vol. 19(6), pages 923-943, December.
    20. Fan, Yanqin, 1994. "Testing the Goodness of Fit of a Parametric Density Function by Kernel Method," Econometric Theory, Cambridge University Press, vol. 10(2), pages 316-356, June.
    21. Powell, James L., 1986. "Censored regression quantiles," Journal of Econometrics, Elsevier, vol. 32(1), pages 143-155, June.
    22. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    23. Victor Chernozhukov & Christian Hansen, 2005. "An IV Model of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 73(1), pages 245-261, January.
    24. Newey, Whitney K, 1985. "Maximum Likelihood Specification Testing and Conditional Moment Tests," Econometrica, Econometric Society, vol. 53(5), pages 1047-1070, September.
    25. Andrews, Donald W. K. & Lu, Biao, 2001. "Consistent model and moment selection procedures for GMM estimation with application to dynamic panel data models," Journal of Econometrics, Elsevier, vol. 101(1), pages 123-164, March.
    26. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    27. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    28. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(2), pages 163-185, June.
    29. Fernandez-Villaverde, Jesus & Francisco Rubio-Ramirez, Juan, 2004. "Comparing dynamic equilibrium models to data: a Bayesian approach," Journal of Econometrics, Elsevier, vol. 123(1), pages 153-187, November.
    30. Pollard, David, 1991. "Asymptotics for Least Absolute Deviation Regression Estimators," Econometric Theory, Cambridge University Press, vol. 7(2), pages 186-199, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shimizu, Kenichi, 2023. "Asymptotic properties of Bayesian inference in linear regression with a structural break," Journal of Econometrics, Elsevier, vol. 235(1), pages 202-219.
    2. Huigang Chen & Mr. Alin T Mirestean & Mr. Charalambos G Tsangarides, 2011. "Limited Information Bayesian Model Averaging for Dynamic Panels with An Application to a Trade Gravity Model," IMF Working Papers 2011/230, International Monetary Fund.
    3. Erhard Reschenhofer & David Preinerstorfer & Lukas Steinberger, 2013. "Non-monotonic penalizing for the number of structural breaks," Computational Statistics, Springer, vol. 28(6), pages 2585-2598, December.
    4. Eric Hillebrand & Marcelo C. Medeiros, 2016. "Nonlinearity, Breaks, and Long-Range Dependence in Time-Series Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(1), pages 23-41, January.
    5. Carlos Carvalho & Stefano Eusepi & Emanuel Moench & Bruce Preston, 2023. "Anchored Inflation Expectations," American Economic Journal: Macroeconomics, American Economic Association, vol. 15(1), pages 1-47, January.
    6. Mao, Guangyu, 2015. "Model selection of M-estimation models using least squares approximation," Statistics & Probability Letters, Elsevier, vol. 99(C), pages 238-243.
    7. Alexander Vosseler & Enzo Weber, 2018. "Forecasting seasonal time series data: a Bayesian model averaging approach," Computational Statistics, Springer, vol. 33(4), pages 1733-1765, December.
    8. Mao, Guangyu, 2013. "Model selection for regression with heteroskedastic and autocorrelated errors," Economics Letters, Elsevier, vol. 118(3), pages 497-501.
    9. Koetter, Michael & Poghosyan, Tigran, 2010. "Real estate prices and bank stability," Journal of Banking & Finance, Elsevier, vol. 34(6), pages 1129-1138, June.
    10. Cui, Xiaomeng & Gafarov, Bulat & Ghanem, Dalia & Kuffner, Todd, 2024. "On model selection criteria for climate change impact studies," Journal of Econometrics, Elsevier, vol. 239(1).
    11. Li, Cheng & Jiang, Wenxin, 2016. "On oracle property and asymptotic validity of Bayesian generalized method of moments," Journal of Multivariate Analysis, Elsevier, vol. 145(C), pages 132-147.
    12. Siddharta Chib & Minchul Shin & Anna Simoni, 2016. "Bayesian Empirical Likelihood Estimation and Comparison of Moment Condition Models," Working Papers 2016-21, Center for Research in Economics and Statistics.
    13. Minerva Mukhopadhyay & Sourabh Bhattacharya, 2022. "Bayes factor asymptotics for variable selection in the Gaussian process framework," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(3), pages 581-613, June.
    14. Mr. Alin T Mirestean & Mr. Charalambos G Tsangarides & Huigang Chen, 2009. "Limited Information Bayesian Model Averaging for Dynamic Panels with Short Time Periods," IMF Working Papers 2009/074, International Monetary Fund.
    15. Barbaglia, Luca & Frattarolo, Lorenzo & Onorante, Luca & Pericoli, Filippo Maria & Ratto, Marco & Tiozzo Pezzoli, Luca, 2023. "Testing big data in a big crisis: Nowcasting under Covid-19," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1548-1563.
    16. Brück, Florian & Fermanian, Jean-David & Min, Aleksey, 2023. "A corrected Clarke test for model selection and beyond," Journal of Econometrics, Elsevier, vol. 235(1), pages 105-132.
    17. Javier García-Cicco & Roque Montero, 2011. "Modeling Copper Price: A Regime-Switching Approach," Working Papers Central Bank of Chile 613, Central Bank of Chile.
    18. Kira Alhorn & Holger Dette & Kirsten Schorning, 2021. "Optimal Designs for Model Averaging in non-nested Models," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(2), pages 745-778, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez & Manuel S. Santos, 2006. "Convergence Properties of the Likelihood of Computed Dynamic Models," Econometrica, Econometric Society, vol. 74(1), pages 93-119, January.
    2. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1059-1087.
    3. Parente, Paulo M.D.C. & Smith, Richard J., 2011. "Gel Methods For Nonsmooth Moment Indicators," Econometric Theory, Cambridge University Press, vol. 27(1), pages 74-113, February.
    4. Susanne M. Schennach & Daniel Wilhelm, 2017. "A Simple Parametric Model Selection Test," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1663-1674, October.
    5. Chen, Xiaohong & Hong, Han & Shum, Matthew, 2007. "Nonparametric likelihood ratio model selection tests between parametric likelihood and moment condition models," Journal of Econometrics, Elsevier, vol. 141(1), pages 109-140, November.
    6. Sungbae An & Frank Schorfheide, 2007. "Bayesian Analysis of DSGE Models," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 113-172.
    7. Jesús Fernández-Villaverde, 2010. "The econometrics of DSGE models," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 1(1), pages 3-49, March.
    8. Todd E. Clark & Michael W. McCracken, 2010. "Averaging forecasts from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 5-29, January.
    9. Antoine Mandel & Amir Sani, 2017. "A Machine Learning Approach to the Forecast Combination Puzzle," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01317974, HAL.
    10. Komunjer, Ivana, 2005. "Quasi-maximum likelihood estimation for conditional quantiles," Journal of Econometrics, Elsevier, vol. 128(1), pages 137-164, September.
    11. John Geweke & Joel Horowitz & M. Hashem Pesaran, 2006. "Econometrics: A Bird’s Eye View," CESifo Working Paper Series 1870, CESifo.
    12. Szafranek, Karol, 2019. "Bagged neural networks for forecasting Polish (low) inflation," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1042-1059.
    13. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    14. Maican, Florin G., 2012. "From Boom to Bust and Back Again: A dynamic analysis of IT services," Working Papers in Economics 543, University of Gothenburg, Department of Economics.
    15. Jesús Fernández-Villaverde & Juan F. Rubio-Ramirez, 2004. "Estimating nonlinear dynamic equilibrium economies: a likelihood approach," FRB Atlanta Working Paper 2004-1, Federal Reserve Bank of Atlanta.
    16. Rossi, Barbara & Sekhposyan, Tatevik, 2014. "Evaluating predictive densities of US output growth and inflation in a large macroeconomic data set," International Journal of Forecasting, Elsevier, vol. 30(3), pages 662-682.
    17. Samuels, Jon D. & Sekkel, Rodrigo M., 2017. "Model Confidence Sets and forecast combination," International Journal of Forecasting, Elsevier, vol. 33(1), pages 48-60.
    18. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2008. "How Structural Are Structural Parameters?," NBER Chapters, in: NBER Macroeconomics Annual 2007, Volume 22, pages 83-137, National Bureau of Economic Research, Inc.
    19. Christian Kascha & Francesco Ravazzolo, 2010. "Combining inflation density forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 231-250.
    20. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.

    More about this item

    Keywords

    Model selection criteria; Nonnested; Posterior odds; BIC;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:167:y:2012:i:2:p:358-369. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.