IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v152y2020ics0167947320300840.html
   My bibliography  Save this article

Model averaging assisted sufficient dimension reduction

Author

Listed:
  • Fang, Fang
  • Yu, Zhou

Abstract

Sufficient dimension reduction that replaces original predictors with their low- dimensional linear combinations without loss of information is a critical tool in modern statistics and has gained considerable research momentum in the past decades since the two pioneers sliced inverse regression and principal Hessian directions. The classical sufficient dimension reduction methods do not handle sparse case well since the estimated linear reductions involve all of the original predictors. Sparse sufficient dimension reduction methods rely on sparsity assumption which may not be true in practice. Motivated by the least squares formulation of the classical sliced inverse regression and principal Hessian directions, several model averaging assisted sufficient dimension reduction methods are proposed. They are applicable to both dense and sparse cases even with weak signals since model averaging adaptively assigns weights to different candidate models. Based on the model averaging assisted sufficient dimension reduction methods, how to estimate the structural dimension is further studied. Theoretical justifications are given and empirical results show that the proposed methods compare favorably with the classical sufficient dimension reduction methods and popular sparse sufficient dimension reduction methods.

Suggested Citation

  • Fang, Fang & Yu, Zhou, 2020. "Model averaging assisted sufficient dimension reduction," Computational Statistics & Data Analysis, Elsevier, vol. 152(C).
    • Handle: RePEc:eee:csdana:v:152:y:2020:i:c:s0167947320300840
    DOI: 10.1016/j.csda.2020.106993
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947320300840
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2020.106993?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kitagawa, Toru & Muris, Chris, 2016. "Model averaging in semiparametric estimation of treatment effects," Journal of Econometrics, Elsevier, vol. 193(1), pages 271-289.
    2. Zou, Changliang & Chen, Xin, 2012. "On the consistency of coordinate-independent sparse estimation with BIC," Journal of Multivariate Analysis, Elsevier, vol. 112(C), pages 248-255.
    3. Kean Ming Tan & Zhaoran Wang & Tong Zhang & Han Liu & R Dennis Cook, 2018. "A convex formulation for high-dimensional sparse sliced inverse regression," Biometrika, Biometrika Trust, vol. 105(4), pages 769-782.
    4. Lexin Li, 2007. "Sparse sufficient dimension reduction," Biometrika, Biometrika Trust, vol. 94(3), pages 603-613.
    5. Qian Lin & Zhigen Zhao & Jun S. Liu, 2019. "Sparse Sliced Inverse Regression via Lasso," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1726-1739, October.
    6. N. T. Longford, 2005. "Editorial: Model selection and efficiency—is ‘Which model …?’ the right question?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(3), pages 469-472, July.
    7. Liqiang Ni & R. Dennis Cook & Chih-Ling Tsai, 2005. "A note on shrinkage sliced inverse regression," Biometrika, Biometrika Trust, vol. 92(1), pages 242-247, March.
    8. Xiangrong Yin & R. Dennis Cook, 2002. "Dimension reduction for the conditional kth moment in regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(2), pages 159-175, May.
    9. Zhang, Xinyu & Liu, Chu-An, 2019. "Inference After Model Averaging In Linear Regression Models," Econometric Theory, Cambridge University Press, vol. 35(4), pages 816-841, August.
    10. Wei Qian & Shanshan Ding & R. Dennis Cook, 2019. "Sparse Minimum Discrepancy Approach to Sufficient Dimension Reduction with Simultaneous Variable Selection in Ultrahigh Dimension," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(527), pages 1277-1290, July.
    11. Xiangrong Yin & Haileab Hilafu, 2015. "Sequential sufficient dimension reduction for large p, small n problems," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(4), pages 879-892, September.
    12. Yin, Xiangrong & Li, Bing & Cook, R. Dennis, 2008. "Successive direction extraction for estimating the central subspace in a multiple-index regression," Journal of Multivariate Analysis, Elsevier, vol. 99(8), pages 1733-1757, September.
    13. Xinyu Zhang & Dalei Yu & Guohua Zou & Hua Liang, 2016. "Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1775-1790, October.
    14. Xinyu Zhang & Guohua Zou & Hua Liang, 2014. "Model averaging and weight choice in linear mixed-effects models," Biometrika, Biometrika Trust, vol. 101(1), pages 205-218.
    15. R. Dennis Cook & Liqiang Ni, 2006. "Using intraslice covariances for improved estimation of the central subspace in regression," Biometrika, Biometrika Trust, vol. 93(1), pages 65-74, March.
    16. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    17. Zhou Yu & Yuexiao Dong & Li-Xing Zhu, 2016. "Trace Pursuit: A General Framework for Model-Free Variable Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(514), pages 813-821, April.
    18. Tomohiro Ando & Ker-Chau Li, 2014. "A Model-Averaging Approach for High-Dimensional Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 254-265, March.
    19. Zhu, Lixing & Miao, Baiqi & Peng, Heng, 2006. "On Sliced Inverse Regression With High-Dimensional Covariates," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 630-643, June.
    20. Li, Degui & Linton, Oliver & Lu, Zudi, 2015. "A flexible semiparametric forecasting model for time series," Journal of Econometrics, Elsevier, vol. 187(1), pages 345-357.
    21. Yang Y., 2001. "Adaptive Regression by Mixing," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 574-588, June.
    22. Zhao, L. C. & Krishnaiah, P. R. & Bai, Z. D., 1986. "On detection of the number of signals in presence of white noise," Journal of Multivariate Analysis, Elsevier, vol. 20(1), pages 1-25, October.
    23. Li, Bing & Wang, Shaoli, 2007. "On Directional Regression for Dimension Reduction," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 997-1008, September.
    24. Jialiang Li & Xiaochao Xia & Weng Kee Wong & David Nott, 2018. "Varying‐coefficient semiparametric model averaging prediction," Biometrics, The International Biometric Society, vol. 74(4), pages 1417-1426, December.
    25. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    26. Lexin Li & Xiangrong Yin, 2008. "Sliced Inverse Regression with Regularizations," Biometrics, The International Biometric Society, vol. 64(1), pages 124-131, March.
    27. Rong Zhu & Alan T. K. Wan & Xinyu Zhang & Guohua Zou, 2019. "A Mallows-Type Model Averaging Estimator for the Varying-Coefficient Partially Linear Model," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 882-892, April.
    28. Wei Luo & Bing Li, 2016. "Combining eigenvalues and variation of eigenvectors for order determination," Biometrika, Biometrika Trust, vol. 103(4), pages 875-887.
    29. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Power, Michael Declan & Dong, Yuexiao, 2021. "Bayesian model averaging sliced inverse regression," Statistics & Probability Letters, Elsevier, vol. 174(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fang, Fang & Li, Jialiang & Xia, Xiaochao, 2022. "Semiparametric model averaging prediction for dichotomous response," Journal of Econometrics, Elsevier, vol. 229(2), pages 219-245.
    2. Wang, Qin & Xue, Yuan, 2021. "An ensemble of inverse moment estimators for sufficient dimension reduction," Computational Statistics & Data Analysis, Elsevier, vol. 161(C).
    3. Weng, Jiaying, 2022. "Fourier transform sparse inverse regression estimators for sufficient variable selection," Computational Statistics & Data Analysis, Elsevier, vol. 168(C).
    4. Jingwen Tu & Hu Yang & Chaohui Guo & Jing Lv, 2021. "Model averaging marginal regression for high dimensional conditional quantile prediction," Statistical Papers, Springer, vol. 62(6), pages 2661-2689, December.
    5. Yuan, Chaoxia & Fang, Fang & Ni, Lyu, 2022. "Mallows model averaging with effective model size in fragmentary data prediction," Computational Statistics & Data Analysis, Elsevier, vol. 173(C).
    6. Xiaochao Xia, 2021. "Model averaging prediction for nonparametric varying-coefficient models with B-spline smoothing," Statistical Papers, Springer, vol. 62(6), pages 2885-2905, December.
    7. Qin Wang & Yuan Xue, 2023. "A structured covariance ensemble for sufficient dimension reduction," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 777-800, September.
    8. Wang, Pei & Yin, Xiangrong & Yuan, Qingcong & Kryscio, Richard, 2021. "Feature filter for estimating central mean subspace and its sparse solution," Computational Statistics & Data Analysis, Elsevier, vol. 163(C).
    9. Yuting Wei & Qihua Wang & Wei Liu, 2021. "Model averaging for linear models with responses missing at random," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(3), pages 535-553, June.
    10. Wei, Yuting & Wang, Qihua, 2021. "Cross-validation-based model averaging in linear models with response missing at random," Statistics & Probability Letters, Elsevier, vol. 171(C).
    11. Yan, Xiaodong & Wang, Hongni & Wang, Wei & Xie, Jinhan & Ren, Yanyan & Wang, Xinjun, 2021. "Optimal model averaging forecasting in high-dimensional survival analysis," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1147-1155.
    12. Gao, Yan & Zhang, Xinyu & Wang, Shouyang & Zou, Guohua, 2016. "Model averaging based on leave-subject-out cross-validation," Journal of Econometrics, Elsevier, vol. 192(1), pages 139-151.
    13. Xiao, Zhen & Zhang, Qi, 2022. "Dimension reduction for block-missing data based on sparse sliced inverse regression," Computational Statistics & Data Analysis, Elsevier, vol. 167(C).
    14. Jialiang Li & Tonghui Yu & Jing Lv & Mei‐Ling Ting Lee, 2021. "Semiparametric model averaging prediction for lifetime data via hazards regression," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(5), pages 1187-1209, November.
    15. Zhang, Xinyu & Liu, Chu-An, 2023. "Model averaging prediction by K-fold cross-validation," Journal of Econometrics, Elsevier, vol. 235(1), pages 280-301.
    16. Sun, Yuying & Hong, Yongmiao & Wang, Shouyang & Zhang, Xinyu, 2023. "Penalized time-varying model averaging," Journal of Econometrics, Elsevier, vol. 235(2), pages 1355-1377.
    17. Wei Luo, 2022. "On efficient dimension reduction with respect to the interaction between two response variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 269-294, April.
    18. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    19. Haowen Bao & Zongwu Cai & Yuying Sun & Shouyang Wang, 2023. "Penalized Model Averaging for High Dimensional Quantile Regressions," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202302, University of Kansas, Department of Economics, revised Jan 2023.
    20. Sun, Yuying & Zhang, Xinyu & Wan, Alan T.K. & Wang, Shouyang, 2022. "Model averaging for interval-valued data," European Journal of Operational Research, Elsevier, vol. 301(2), pages 772-784.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:152:y:2020:i:c:s0167947320300840. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.