IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v146y2016icp7-17.html
   My bibliography  Save this article

Supervised singular value decomposition and its asymptotic properties

Author

Listed:
  • Li, Gen
  • Yang, Dan
  • Nobel, Andrew B.
  • Shen, Haipeng

Abstract

A supervised singular value decomposition (SupSVD) model has been developed for supervised dimension reduction where the low rank structure of the data of interest is potentially driven by additional variables measured on the same set of samples. The SupSVD model can make use of the information in the additional variables to accurately extract underlying structures that are more interpretable. The model is general and includes the principal component analysis model and the reduced rank regression model as two extreme cases. The model is formulated in a hierarchical fashion using latent variables, and a modified expectation–maximization algorithm for parameter estimation is developed, which is computationally efficient. The asymptotic properties for the estimated parameters are derived. We use comprehensive simulations and a real data example to illustrate the advantages of the SupSVD model.

Suggested Citation

  • Li, Gen & Yang, Dan & Nobel, Andrew B. & Shen, Haipeng, 2016. "Supervised singular value decomposition and its asymptotic properties," Journal of Multivariate Analysis, Elsevier, vol. 146(C), pages 7-17.
    • Handle: RePEc:eee:jmvana:v:146:y:2016:i:c:p:7-17
    DOI: 10.1016/j.jmva.2015.02.016
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047259X15000512
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jmva.2015.02.016?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shen, Haipeng & Huang, Jianhua Z., 2008. "Sparse principal component analysis via regularized low rank matrix approximation," Journal of Multivariate Analysis, Elsevier, vol. 99(6), pages 1015-1034, July.
    2. Shen, Dan & Shen, Haipeng & Marron, J.S., 2013. "Consistency of sparse PCA in High Dimension, Low Sample Size contexts," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 317-333.
    3. Aneiros, Germán & Vieu, Philippe, 2014. "Variable selection in infinite-dimensional problems," Statistics & Probability Letters, Elsevier, vol. 94(C), pages 12-20.
    4. R. D. Cook & I. S. Helland & Z. Su, 2013. "Envelopes and partial least squares regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(5), pages 851-877, November.
    5. Izenman, Alan Julian, 1975. "Reduced-rank regression for the multivariate linear model," Journal of Multivariate Analysis, Elsevier, vol. 5(2), pages 248-264, June.
    6. Cook, R. Dennis & Ni, Liqiang, 2005. "Sufficient Dimension Reduction via Inverse Regression: A Minimum Discrepancy Approach," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 410-428, June.
    7. R. Dennis Cook & Zhihua Su, 2013. "Scaled envelopes: scale-invariant and efficient estimation in multivariate linear regression," Biometrika, Biometrika Trust, vol. 100(4), pages 939-954.
    8. Bair, Eric & Hastie, Trevor & Paul, Debashis & Tibshirani, Robert, 2006. "Prediction by Supervised Principal Components," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 119-137, March.
    9. Huang, Jianhua Z. & Shen, Haipeng & Buja, Andreas, 2009. "The Analysis of Two-Way Functional Data Using Two-Way Regularized Singular Value Decompositions," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1609-1620.
    10. Zhihua Su & R. Dennis Cook, 2011. "Partial envelopes for efficient estimation in multivariate linear regression," Biometrika, Biometrika Trust, vol. 98(1), pages 133-146.
    11. Dozier, R. Brent & Silverstein, Jack W., 2007. "On the empirical distribution of eigenvalues of large dimensional information-plus-noise-type matrices," Journal of Multivariate Analysis, Elsevier, vol. 98(4), pages 678-694, April.
    12. Shabalin, Andrey A. & Nobel, Andrew B., 2013. "Reconstruction of a low-rank matrix in the presence of Gaussian noise," Journal of Multivariate Analysis, Elsevier, vol. 118(C), pages 67-76.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Junhui Cai & Dan Yang & Wu Zhu & Haipeng Shen & Linda Zhao, 2021. "Network regression and supervised centrality estimation," Papers 2111.12921, arXiv.org.
    2. Fan, Jianqing & Ke, Yuan & Liao, Yuan, 2021. "Augmented factor models with applications to validating market risk factors and forecasting bond risk premia," Journal of Econometrics, Elsevier, vol. 222(1), pages 269-294.
    3. Minji Lee & Zhihua Su, 2020. "A Review of Envelope Models," International Statistical Review, International Statistical Institute, vol. 88(3), pages 658-676, December.
    4. Phuong T. Vu & Timothy V. Larson & Adam A. Szpiro, 2020. "Probabilistic predictive principal component analysis for spatially misaligned and high‐dimensional air pollution data with missing observations," Environmetrics, John Wiley & Sons, Ltd., vol. 31(4), June.
    5. Wan, Runzhe & Li, Yingying & Lu, Wenbin & Song, Rui, 2024. "Mining the factor zoo: Estimation of latent factor models with sufficient proxies," Journal of Econometrics, Elsevier, vol. 239(2).
    6. Gen Li & Sungkyu Jung, 2017. "Incorporating covariates into integrated factor analysis of multi‐view data," Biometrics, The International Biometric Society, vol. 73(4), pages 1433-1442, December.
    7. Xing Gao & Sungwon Lee & Gen Li & Sungkyu Jung, 2021. "Covariate‐driven factorization by thresholding for multiblock data," Biometrics, The International Biometric Society, vol. 77(3), pages 1011-1023, September.
    8. Jung, Sungkyu, 2018. "Continuum directions for supervised dimension reduction," Computational Statistics & Data Analysis, Elsevier, vol. 125(C), pages 27-43.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Minji Lee & Zhihua Su, 2020. "A Review of Envelope Models," International Statistical Review, International Statistical Institute, vol. 88(3), pages 658-676, December.
    2. Zhang, Xin & Wang, Chong & Wu, Yichao, 2018. "Functional envelope for model-free sufficient dimension reduction," Journal of Multivariate Analysis, Elsevier, vol. 163(C), pages 37-50.
    3. Jung, Sungkyu, 2018. "Continuum directions for supervised dimension reduction," Computational Statistics & Data Analysis, Elsevier, vol. 125(C), pages 27-43.
    4. Iaci, Ross & Yin, Xiangrong & Zhu, Lixing, 2016. "The Dual Central Subspaces in dimension reduction," Journal of Multivariate Analysis, Elsevier, vol. 145(C), pages 178-189.
    5. Chen, Canyi & Xu, Wangli & Zhu, Liping, 2022. "Distributed estimation in heterogeneous reduced rank regression: With application to order determination in sufficient dimension reduction," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    6. Dennis Cook, R. & Forzani, Liliana, 2023. "On the role of partial least squares in path analysis for the social sciences," Journal of Business Research, Elsevier, vol. 167(C).
    7. Hojin Yang & Hongtu Zhu & Joseph G. Ibrahim, 2018. "MILFM: Multiple index latent factor model based on high‐dimensional features," Biometrics, The International Biometric Society, vol. 74(3), pages 834-844, September.
    8. Yue Zhao & Ingrid Van Keilegom & Shanshan Ding, 2022. "Envelopes for censored quantile regression," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(4), pages 1562-1585, December.
    9. Hongtu Zhu & Dan Shen & Xuewei Peng & Leo Yufeng Liu, 2017. "MWPCR: Multiscale Weighted Principal Component Regression for High-Dimensional Prediction," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1009-1021, July.
    10. Andrea Bergesio & María Eugenia Szretter Noste & Víctor J. Yohai, 2021. "A robust proposal of estimation for the sufficient dimension reduction problem," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 758-783, September.
    11. D. J. Eck & R. D. Cook, 2017. "Weighted envelope estimation to handle variability in model selection," Biometrika, Biometrika Trust, vol. 104(3), pages 743-749.
    12. Juan C. Laria & M. Carmen Aguilera-Morillo & Rosa E. Lillo, 2023. "Group linear algorithm with sparse principal decomposition: a variable selection and clustering method for generalized linear models," Statistical Papers, Springer, vol. 64(1), pages 227-253, February.
    13. Cook, R. Dennis & Forzani, Liliana & Su, Zhihua, 2016. "A note on fast envelope estimation," Journal of Multivariate Analysis, Elsevier, vol. 150(C), pages 42-54.
    14. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    15. Ekvall, Karl Oskar, 2022. "Targeted principal components regression," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    16. Yeonhee Park & Zhihua Su & Hongtu Zhu, 2017. "Groupwise envelope models for imaging genetic analysis," Biometrics, The International Biometric Society, vol. 73(4), pages 1243-1253, December.
    17. Jain Yashita & Ding Shanshan & Qiu Jing, 2019. "Sliced inverse regression for integrative multi-omics data analysis," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 18(1), pages 1-13, February.
    18. Forzani, Liliana & Rodriguez, Daniela & Smucler, Ezequiel & Sued, Mariela, 2019. "Sufficient dimension reduction and prediction in regression: Asymptotic results," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 339-349.
    19. Yang, Yang & Yang, Yanrong & Shang, Han Lin, 2022. "Feature extraction for functional time series: Theory and application to NIR spectroscopy data," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    20. Huang, Dashan & Jiang, Fuwei & Li, Kunpeng & Tong, Guoshi & Zhou, Guofu, 2023. "Are bond returns predictable with real-time macro data?," Journal of Econometrics, Elsevier, vol. 237(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:146:y:2016:i:c:p:7-17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.