IDEAS home Printed from https://ideas.repec.org/p/cor/louvco/2023005.html
   My bibliography  Save this paper

Direction Identification and Minimax Estimation by Generalized Eigenvalue Problem in High Dimensional Sparse Regression

Author

Listed:
  • Sauvenier, Mathieu

    (Université catholique de Louvain, LIDAM/CORE, Belgium)

  • Van Bellegem, Sébastien

    (Université catholique de Louvain, LIDAM/CORE, Belgium)

Abstract

In high-dimensional sparse linear regression, the selection and the estimation of the parameters are studied based on an L0−constraint on the direction of the vector of parameters. We first establish a general result for the direction of the vector of parameters, which is identified through the leading generalized eigenspace of measurable matrices. Based on this result, we suggest addressing the best subset selection problem from a new perspective by solving an empirical generalized eigenvalue problem to estimate the direction of the high-dimensional vector of parameters. We then study a new estimator based on the RIFLE algorithm and demonstrate a nonasymptotic bound of the L2 risk, the minimax convergence of the estimator and a central limit theorem. Simulations show the superiority of the proposed inference over some known l0 constrained estimators.

Suggested Citation

  • Sauvenier, Mathieu & Van Bellegem, Sébastien, 2023. "Direction Identification and Minimax Estimation by Generalized Eigenvalue Problem in High Dimensional Sparse Regression," LIDAM Discussion Papers CORE 2023005, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    • Handle: RePEc:cor:louvco:2023005
    as

    Download full text from publisher

    File URL: https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A272429/datastream/PDF_01/view
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lexin Li, 2007. "Sparse sufficient dimension reduction," Biometrika, Biometrika Trust, vol. 94(3), pages 603-613.
    2. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    3. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    4. Gold, David & Lederer, Johannes & Tao, Jing, 2020. "Inference for high-dimensional instrumental variables regression," Journal of Econometrics, Elsevier, vol. 217(1), pages 79-111.
    5. Xiaotong Shen & Wei Pan & Yunzhang Zhu & Hui Zhou, 2013. "On constrained and regularized high-dimensional regression," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 65(5), pages 807-832, October.
    6. Kean Ming Tan & Zhaoran Wang & Han Liu & Tong Zhang, 2018. "Sparse generalized eigenvalue problem: optimal statistical rates via truncated Rayleigh flow," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(5), pages 1057-1086, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sauvenier, Mathieu & Van Bellegem, Sébastien, 2023. "Goodness-of-fit test in high-dimensional linear sparse models," LIDAM Discussion Papers CORE 2023008, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Saulius Jokubaitis & Remigijus Leipus, 2022. "Asymptotic Normality in Linear Regression with Approximately Sparse Structure," Mathematics, MDPI, vol. 10(10), pages 1-28, May.
    2. Hung Hung & Su‐Yun Huang, 2019. "Sufficient dimension reduction via random‐partitions for the large‐p‐small‐n problem," Biometrics, The International Biometric Society, vol. 75(1), pages 245-255, March.
    3. Lee, Ji Hyung & Shi, Zhentao & Gao, Zhan, 2022. "On LASSO for predictive regression," Journal of Econometrics, Elsevier, vol. 229(2), pages 322-349.
    4. Medeiros, Marcelo C. & Mendes, Eduardo F., 2016. "ℓ1-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors," Journal of Econometrics, Elsevier, vol. 191(1), pages 255-271.
    5. Hojin Yang & Hongtu Zhu & Joseph G. Ibrahim, 2018. "MILFM: Multiple index latent factor model based on high‐dimensional features," Biometrics, The International Biometric Society, vol. 74(3), pages 834-844, September.
    6. Wang, Pei & Yin, Xiangrong & Yuan, Qingcong & Kryscio, Richard, 2021. "Feature filter for estimating central mean subspace and its sparse solution," Computational Statistics & Data Analysis, Elsevier, vol. 163(C).
    7. Alexandre Belloni & Victor Chernozhukov & Lie Wang, 2013. "Pivotal estimation via square-root lasso in nonparametric regression," CeMMAP working papers CWP62/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. Eric Gautier & Christiern Rose, 2022. "Fast, Robust Inference for Linear Instrumental Variables Models using Self-Normalized Moments," Papers 2211.02249, arXiv.org, revised Nov 2022.
    9. Dai, Linlin & Chen, Kani & Sun, Zhihua & Liu, Zhenqiu & Li, Gang, 2018. "Broken adaptive ridge regression and its asymptotic properties," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 334-351.
    10. Qinqin Hu & Lu Lin, 2022. "Feature Screening in High Dimensional Regression with Endogenous Covariates," Computational Economics, Springer;Society for Computational Economics, vol. 60(3), pages 949-969, October.
    11. He, Xin & Mao, Xiaojun & Wang, Zhonglei, 2024. "Nonparametric augmented probability weighting with sparsity," Computational Statistics & Data Analysis, Elsevier, vol. 191(C).
    12. Alena Skolkova, 2023. "Instrumental Variable Estimation with Many Instruments Using Elastic-Net IV," CERGE-EI Working Papers wp759, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    13. Qingliang Fan & Zijian Guo & Ziwei Mei, 2022. "A Heteroskedasticity-Robust Overidentifying Restriction Test with High-Dimensional Covariates," Papers 2205.00171, arXiv.org, revised May 2024.
    14. Helmut Farbmacher & Rebecca Groh & Michael Muhlegger & Gabriel Vollert, 2024. "Revisiting the Many Instruments Problem using Random Matrix Theory," Papers 2408.08580, arXiv.org.
    15. Hao Zeng & Wei Zhong & Xingbai Xu, 2024. "Transfer Learning for Spatial Autoregressive Models with Application to U.S. Presidential Election Prediction," Papers 2405.15600, arXiv.org, revised Sep 2024.
    16. Emmanuel Jordy Menvouta & Sven Serneels & Tim Verdonck, 2022. "Sparse dimension reduction based on energy and ball statistics," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(4), pages 951-975, December.
    17. Zhou Yu & Yuexiao Dong & Li-Xing Zhu, 2016. "Trace Pursuit: A General Framework for Model-Free Variable Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(514), pages 813-821, April.
    18. Jooyoung Cha & Harold D. Chiang & Yuya Sasaki, 2021. "Inference in high-dimensional regression models without the exact or $L^p$ sparsity," Papers 2108.09520, arXiv.org, revised Dec 2022.
    19. Kock, Anders Bredahl & Callot, Laurent, 2015. "Oracle inequalities for high dimensional vector autoregressions," Journal of Econometrics, Elsevier, vol. 186(2), pages 325-344.
    20. Caner, Mehmet & Kock, Anders Bredahl, 2018. "Asymptotically honest confidence regions for high dimensional parameters by the desparsified conservative Lasso," Journal of Econometrics, Elsevier, vol. 203(1), pages 143-168.

    More about this item

    Keywords

    High-dimensional model ; sparsity ; generalized eigenvalue problem ; identification ; best subset selection ; minimax L0 estimation ; central limit theorem;
    All these keywords.

    JEL classification:

    • C30 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - General
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C59 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Other

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cor:louvco:2023005. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Alain GILLIS (email available below). General contact details of provider: https://edirc.repec.org/data/coreebe.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.