IDEAS home Printed from https://ideas.repec.org/a/spr/aistmt/v72y2020i2d10.1007_s10463-018-0693-6.html
   My bibliography  Save this article

Sharp oracle inequalities for low-complexity priors

Author

Listed:
  • Tung Duy Luu

    (Normandie Univ)

  • Jalal Fadili

    (Normandie Univ)

  • Christophe Chesneau

    (Normandie Univ, Université de Caen Normandie)

Abstract

In this paper, we consider a high-dimensional statistical estimation problem in which the number of parameters is comparable or larger than the sample size. We present a unified analysis of the performance guarantees of exponential weighted aggregation and penalized estimators with a general class of data losses and priors which encourage objects which conform to some notion of simplicity/complexity. More precisely, we show that these two estimators satisfy sharp oracle inequalities for prediction ensuring their good theoretical performances. We also highlight the differences between them. When the noise is random, we provide oracle inequalities in probability using concentration inequalities. These results are then applied to several instances including the Lasso, the group Lasso, their analysis-type counterparts, the $$\ell _\infty $$ℓ∞ and the nuclear norm penalties. All our estimators can be efficiently implemented using proximal splitting algorithms.

Suggested Citation

  • Tung Duy Luu & Jalal Fadili & Christophe Chesneau, 2020. "Sharp oracle inequalities for low-complexity priors," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(2), pages 353-397, April.
    • Handle: RePEc:spr:aistmt:v:72:y:2020:i:2:d:10.1007_s10463-018-0693-6
    DOI: 10.1007/s10463-018-0693-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10463-018-0693-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10463-018-0693-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arnak S. Dalalyan & Mohamed Hebiri & Johannes Lederer, 2014. "On the Prediction Performance of the Lasso," Working Papers 2014-05, Center for Research in Economics and Statistics.
    2. Samuel Vaiter & Charles Deledalle & Jalal Fadili & Gabriel Peyré & Charles Dossal, 2017. "The degrees of freedom of partly smooth regularizers," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 69(4), pages 791-832, August.
    3. Sara Geer, 2014. "Weakly decomposable regularization penalties and structured sparsity," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(1), pages 72-86, March.
    4. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    5. Tingni Sun & Cun-Hui Zhang, 2012. "Scaled sparse linear regression," Biometrika, Biometrika Trust, vol. 99(4), pages 879-898.
    6. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laura Freijeiro‐González & Manuel Febrero‐Bande & Wenceslao González‐Manteiga, 2022. "A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates," International Statistical Review, International Statistical Institute, vol. 90(1), pages 118-145, April.
    2. Victor Chernozhukov & Christian Hansen & Yuan Liao, 2015. "A lava attack on the recovery of sums of dense and sparse signals," CeMMAP working papers CWP56/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    3. Jacob Bien & Irina Gaynanova & Johannes Lederer & Christian L. Müller, 2019. "Prediction error bounds for linear regression with the TREX," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 28(2), pages 451-474, June.
    4. Tung Duy Luu & Jalal Fadili & Christophe Chesneau, 2021. "Sampling from Non-smooth Distributions Through Langevin Diffusion," Methodology and Computing in Applied Probability, Springer, vol. 23(4), pages 1173-1201, December.
    5. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    6. Yize Zhao & Matthias Chung & Brent A. Johnson & Carlos S. Moreno & Qi Long, 2016. "Hierarchical Feature Selection Incorporating Known and Novel Biological Information: Identifying Genomic Features Related to Prostate Cancer Recurrence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1427-1439, October.
    7. Caner, Mehmet, 2023. "Generalized linear models with structured sparsity estimators," Journal of Econometrics, Elsevier, vol. 236(2).
    8. Tomáš Plíhal, 2021. "Scheduled macroeconomic news announcements and Forex volatility forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(8), pages 1379-1397, December.
    9. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    10. Murat Genç & M. Revan Özkale, 2021. "Usage of the GO estimator in high dimensional linear models," Computational Statistics, Springer, vol. 36(1), pages 217-239, March.
    11. Luu, Tung Duy & Fadili, Jalal & Chesneau, Christophe, 2019. "PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 209-233.
    12. Wang, Shixuan & Syntetos, Aris A. & Liu, Ying & Di Cairano-Gilfedder, Carla & Naim, Mohamed M., 2023. "Improving automotive garage operations by categorical forecasts using a large number of variables," European Journal of Operational Research, Elsevier, vol. 306(2), pages 893-908.
    13. Takumi Saegusa & Tianzhou Ma & Gang Li & Ying Qing Chen & Mei-Ling Ting Lee, 2020. "Variable Selection in Threshold Regression Model with Applications to HIV Drug Adherence Data," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 12(3), pages 376-398, December.
    14. Ngai Hang Chan & Chun Yip Yau & Rong-Mao Zhang, 2014. "Group LASSO for Structural Break Time Series," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 590-599, June.
    15. Korobilis, Dimitris, 2013. "Hierarchical shrinkage priors for dynamic regressions with many predictors," International Journal of Forecasting, Elsevier, vol. 29(1), pages 43-59.
    16. Xiaoquan Wen, 2014. "Bayesian model selection in complex linear systems, as illustrated in genetic association studies," Biometrics, The International Biometric Society, vol. 70(1), pages 73-83, March.
    17. Yuanyuan Shen & Katherine P. Liao & Tianxi Cai, 2015. "Sparse kernel machine regression for ordinal outcomes," Biometrics, The International Biometric Society, vol. 71(1), pages 63-70, March.
    18. Wei Tang & Steven L Bressler & Chad M Sylvester & Gordon L Shulman & Maurizio Corbetta, 2012. "Measuring Granger Causality between Cortical Regions from Voxelwise fMRI BOLD Signals with LASSO," PLOS Computational Biology, Public Library of Science, vol. 8(5), pages 1-14, May.
    19. Massimiliano Caporin & Francesco Poli, 2017. "Building News Measures from Textual Data and an Application to Volatility Forecasting," Econometrics, MDPI, vol. 5(3), pages 1-46, August.
    20. Pei Wang & Shunjie Chen & Sijia Yang, 2022. "Recent Advances on Penalized Regression Models for Biological Data," Mathematics, MDPI, vol. 10(19), pages 1-24, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aistmt:v:72:y:2020:i:2:d:10.1007_s10463-018-0693-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.