IDEAS home Printed from https://ideas.repec.org/a/eee/ecolet/v226y2023ics016517652300109x.html
   My bibliography  Save this article

Complete subset averaging approach for high-dimensional generalized linear models

Author

Listed:
  • Chen, Xingyi
  • Li, Haiqi
  • Zhang, Jing

Abstract

This study proposes a novel complete subset averaging (CSA) method for high-dimensional generalized linear models based on a penalized Kullback–Leibler (KL) loss. All models under consideration can be potentially misspecified, and the dimension of covariates is allowed to diverge to infinity. The uniform convergence rate and asymptotic normality of the proposed estimator are established. Moreover, it is asymptotically optimal in terms of achieving the lowest KL loss. To ease the computational burden, we randomly draw a fixed number of subsets from the complete subsets and show their asymptotic equivalence. The Monte Carlo simulation and empirical application demonstrate that the proposed CSA method outperforms popular model-averaging methods.

Suggested Citation

  • Chen, Xingyi & Li, Haiqi & Zhang, Jing, 2023. "Complete subset averaging approach for high-dimensional generalized linear models," Economics Letters, Elsevier, vol. 226(C).
    • Handle: RePEc:eee:ecolet:v:226:y:2023:i:c:s016517652300109x
    DOI: 10.1016/j.econlet.2023.111084
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016517652300109X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econlet.2023.111084?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gerda Claeskens & Christophe Croux & Johan Van Kerckhoven, 2006. "Variable Selection for Logistic Regression Using a Prediction-Focused Information Criterion," Biometrics, The International Biometric Society, vol. 62(4), pages 972-979, December.
    2. Fang, Fang & Yang, Qiwei & Tian, Wenling, 2022. "Cross-validation for selecting the penalty factor in least squares model averaging," Economics Letters, Elsevier, vol. 217(C).
    3. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    4. Xinyu Zhang & Dalei Yu & Guohua Zou & Hua Liang, 2016. "Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1775-1790, October.
    5. Xinyu Zhang & Guohua Zou & Hua Liang, 2014. "Model averaging and weight choice in linear mixed-effects models," Biometrika, Biometrika Trust, vol. 101(1), pages 205-218.
    6. Wang, Weiwei & Zhang, Qi & Zhang, Xinyu & Li, Xinmin, 2021. "Model averaging based on generalized method of moments," Economics Letters, Elsevier, vol. 200(C).
    7. Seojeong Lee & Youngki Shin, 2021. "Complete subset averaging with many instruments," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 290-314.
    8. Fang, Fang & Liu, Minhan, 2020. "Limit of the optimal weight in least squares model averaging with non-nested models," Economics Letters, Elsevier, vol. 196(C).
    9. Philip Oreopoulos, 2006. "Estimating Average and Local Average Treatment Effects of Education when Compulsory Schooling Laws Really Matter," American Economic Review, American Economic Association, vol. 96(1), pages 152-175, March.
    10. Wan, Alan T.K. & Zhang, Xinyu & Zou, Guohua, 2010. "Least squares model averaging by Mallows criterion," Journal of Econometrics, Elsevier, vol. 156(2), pages 277-283, June.
    11. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(4), pages 979-1014.
    12. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    13. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," The Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    14. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    15. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    16. Cheryl J. Flynn & Clifford M. Hurvich & Jeffrey S. Simonoff, 2013. "Efficiency for Regularization Parameter Selection in Penalized Likelihood Estimation of Misspecified Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(503), pages 1031-1043, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziwen Gao & Jiahui Zou & Xinyu Zhang & Yanyuan Ma, 2023. "Frequentist model averaging for envelope models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(3), pages 1325-1364, September.
    2. Shangwei Zhao & Jun Liao & Dalei Yu, 2020. "Model averaging estimator in ridge regression and its large sample properties," Statistical Papers, Springer, vol. 61(4), pages 1719-1739, August.
    3. Wenchao Xu & Xinyu Zhang, 2024. "On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included," Papers 2411.09258, arXiv.org.
    4. Xinyu Zhang & Dalei Yu & Guohua Zou & Hua Liang, 2016. "Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1775-1790, October.
    5. Haili Zhang & Guohua Zou, 2020. "Cross-Validation Model Averaging for Generalized Functional Linear Model," Econometrics, MDPI, vol. 8(1), pages 1-35, February.
    6. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    7. Yuting Wei & Qihua Wang & Wei Liu, 2021. "Model averaging for linear models with responses missing at random," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(3), pages 535-553, June.
    8. Lee, Ji Hyung & Shin, Youngki, 2023. "Complete Subset Averaging For Quantile Regressions," Econometric Theory, Cambridge University Press, vol. 39(1), pages 146-188, February.
    9. Rongjie Jiang & Liming Wang & Yang Bai, 2021. "Optimal model averaging estimator for semi-functional partially linear models," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 84(2), pages 167-194, February.
    10. Zhang, Xiaomeng & Zhang, Xinyu, 2023. "Optimal model averaging based on forward-validation," Journal of Econometrics, Elsevier, vol. 237(2).
    11. Hongwei Zhang & Qiang He & Ben Jacobsen & Fuwei Jiang, 2020. "Forecasting stock returns with model uncertainty and parameter instability," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 629-644, August.
    12. Seojeong Lee & Youngki Shin, 2018. "Optimal Estimation with Complete Subsets of Instruments," Department of Economics Working Papers 2018-15, McMaster University.
    13. Ren, Yu & Liang, Xuanxuan & Wang, Qin, 2021. "Short-term exchange rate forecasting: A panel combination approach," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 73(C).
    14. Wei, Yuting & Wang, Qihua, 2021. "Cross-validation-based model averaging in linear models with response missing at random," Statistics & Probability Letters, Elsevier, vol. 171(C).
    15. Yan, Xiaodong & Wang, Hongni & Wang, Wei & Xie, Jinhan & Ren, Yanyan & Wang, Xinjun, 2021. "Optimal model averaging forecasting in high-dimensional survival analysis," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1147-1155.
    16. Fang, Fang & Li, Jialiang & Xia, Xiaochao, 2022. "Semiparametric model averaging prediction for dichotomous response," Journal of Econometrics, Elsevier, vol. 229(2), pages 219-245.
    17. Seojeong Lee & Youngki Shin, 2021. "Complete subset averaging with many instruments," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 290-314.
    18. Jingwen Tu & Hu Yang & Chaohui Guo & Jing Lv, 2021. "Model averaging marginal regression for high dimensional conditional quantile prediction," Statistical Papers, Springer, vol. 62(6), pages 2661-2689, December.
    19. Zhang, Xinyu & Liu, Chu-An, 2023. "Model averaging prediction by K-fold cross-validation," Journal of Econometrics, Elsevier, vol. 235(1), pages 280-301.
    20. Yuan, Chaoxia & Fang, Fang & Ni, Lyu, 2022. "Mallows model averaging with effective model size in fragmentary data prediction," Computational Statistics & Data Analysis, Elsevier, vol. 173(C).

    More about this item

    Keywords

    Asymptotic optimality; Complete subset averaging; Kullback–Leibler loss; Generalized linear models;
    All these keywords.

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C35 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecolet:v:226:y:2023:i:c:s016517652300109x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ecolet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.