IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0148620.html
   My bibliography  Save this article

An Adaptive Ridge Procedure for L0 Regularization

Author

Listed:
  • Florian Frommlet
  • Grégory Nuel

Abstract

Penalized selection criteria like AIC or BIC are among the most popular methods for variable selection. Their theoretical properties have been studied intensively and are well understood, but making use of them in case of high-dimensional data is difficult due to the non-convex optimization problem induced by L0 penalties. In this paper we introduce an adaptive ridge procedure (AR), where iteratively weighted ridge problems are solved whose weights are updated in such a way that the procedure converges towards selection with L0 penalties. After introducing AR its specific shrinkage properties are studied in the particular case of orthogonal linear regression. Based on extensive simulations for the non-orthogonal case as well as for Poisson regression the performance of AR is studied and compared with SCAD and adaptive LASSO. Furthermore an efficient implementation of AR in the context of least-squares segmentation is presented. The paper ends with an illustrative example of applying AR to analyze GWAS data.

Suggested Citation

  • Florian Frommlet & Grégory Nuel, 2016. "An Adaptive Ridge Procedure for L0 Regularization," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-23, February.
    • Handle: RePEc:plo:pone00:0148620
    DOI: 10.1371/journal.pone.0148620
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148620
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0148620&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0148620?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ziqi Chen & Man-Lai Tang & Wei Gao & Ning-Zhong Shi, 2014. "New Robust Variable Selection Methods for Linear Regression Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(3), pages 725-741, September.
    2. Karl W. Broman & Terence P. Speed, 2002. "A model selection approach for the identification of quantitative trait loci in experimental crosses," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 641-656, October.
    3. Xiaoguang Wang & Junhui Fan, 2014. "Variable selection for multivariate generalized linear models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(2), pages 393-406, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jian Huang & Yuling Jiao & Lican Kang & Jin Liu & Yanyan Liu & Xiliang Lu, 2022. "GSDAR: a fast Newton algorithm for $$\ell _0$$ ℓ 0 regularized generalized linear models with statistical guarantee," Computational Statistics, Springer, vol. 37(1), pages 507-533, March.
    2. Takumi Saegusa & Tianzhou Ma & Gang Li & Ying Qing Chen & Mei-Ling Ting Lee, 2020. "Variable Selection in Threshold Regression Model with Applications to HIV Drug Adherence Data," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 12(3), pages 376-398, December.
    3. Philippe Goulet Coulombe, 2020. "Time-Varying Parameters as Ridge Regressions," Papers 2009.00401, arXiv.org, revised Apr 2023.
    4. Sascha A. Keweloh, 2023. "Uncertain Short-Run Restrictions and Statistically Identified Structural Vector Autoregressions," Papers 2303.13281, arXiv.org, revised Apr 2024.
    5. Ersin Yılmaz & Syed Ejaz Ahmed & Dursun Aydın, 2020. "A-Spline Regression for Fitting a Nonparametric Regression Function with Censored Data," Stats, MDPI, vol. 3(2), pages 1-17, May.
    6. Luke Smallman & William Underwood & Andreas Artemiou, 2020. "Simple Poisson PCA: an algorithm for (sparse) feature extraction with simultaneous dimension determination," Computational Statistics, Springer, vol. 35(2), pages 559-577, June.
    7. Zhao, Hui & Sun, Dayu & Li, Gang & Sun, Jianguo, 2019. "Simultaneous estimation and variable selection for incomplete event history studies," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 350-361.
    8. Goepp, Vivien & van de Kassteele, Jan, 2024. "Graph-based spatial segmentation of areal data," Computational Statistics & Data Analysis, Elsevier, vol. 192(C).
    9. Antonis Christou & Andreas Artemiou, 2023. "Adaptive L0 Regularization for Sparse Support Vector Regression," Mathematics, MDPI, vol. 11(13), pages 1-12, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frommlet, Florian & Ruhaltinger, Felix & Twaróg, Piotr & Bogdan, Małgorzata, 2012. "Modified versions of Bayesian Information Criterion for genome-wide association studies," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1038-1051.
    2. Zak-Szatkowska, Malgorzata & Bogdan, Malgorzata, 2011. "Modified versions of the Bayesian Information Criterion for sparse Generalized Linear Models," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 2908-2924, November.
    3. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    4. Frommlet Florian & Ljubic Ivana & Arnardóttir Helga Björk & Bogdan Malgorzata, 2012. "QTL Mapping Using a Memetic Algorithm with Modifications of BIC as Fitness Function," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-26, May.
    5. Zehua Chen & Jianbin Liu, 2009. "Mixture Generalized Linear Models for Multiple Interval Mapping of Quantitative Trait Loci in Experimental Crosses," Biometrics, The International Biometric Society, vol. 65(2), pages 470-477, June.
    6. Gilmour, A.R., 2007. "Mixed model regression mapping for QTL detection in experimental crosses," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3749-3764, May.
    7. Huang, B. Emma & Shah, Rohan & George, Andrew W., 2012. "dlmap: An R Package for Mixed Model QTL and Association Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(i06).
    8. Marit Ackermann & Mathieu Clément-Ziza & Jacob J Michaelson & Andreas Beyer, 2012. "Teamwork: Improved eQTL Mapping Using Combinations of Machine Learning Methods," PLOS ONE, Public Library of Science, vol. 7(7), pages 1-8, July.
    9. C. M. Kendziorski & M. Chen & M. Yuan & H. Lan & A. D. Attie, 2006. "Statistical Methods for Expression Quantitative Trait Loci (eQTL) Mapping," Biometrics, The International Biometric Society, vol. 62(1), pages 19-27, March.
    10. Yawei He & Zehua Chen, 2016. "The EBIC and a sequential procedure for feature selection in interactive linear models with high-dimensional data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 68(1), pages 155-180, February.
    11. Chun Wang, 2021. "Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 167-189, March.
    12. repec:jss:jstsof:28:i02 is not listed on IDEAS
    13. Małgorzata Bogdan & Florian Frommlet & Przemysław Biecek & Riyan Cheng & Jayanta K. Ghosh & R.W. Doerge, 2008. "Extending the Modified Bayesian Information Criterion (mBIC) to Dense Markers and Multiple Interval Mapping," Biometrics, The International Biometric Society, vol. 64(4), pages 1162-1169, December.
    14. Ziqi Chen & Man†Lai Tang & Wei Gao, 2018. "A profile likelihood approach for longitudinal data analysis," Biometrics, The International Biometric Society, vol. 74(1), pages 220-228, March.
    15. Erhardt Vinzenz & Bogdan Malgorzata & Czado Claudia, 2010. "Locating Multiple Interacting Quantitative Trait Loci with the Zero-Inflated Generalized Poisson Regression," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 9(1), pages 1-27, June.
    16. Baierl, Andreas & Futschik, Andreas & Bogdan, Malgorzata & Biecek, Przemyslaw, 2007. "Locating multiple interacting quantitative trait loci using robust model selection," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 6423-6434, August.
    17. Jiming Jiang & Thuan Nguyen & J. Sunil Rao, 2015. "The E-MS Algorithm: Model Selection With Incomplete Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 1136-1147, September.
    18. Sun, Hanmei & Nguyen, Thuan & Luan, Yihui & Jiang, Jiming, 2018. "Classified mixed logistic model prediction," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 63-74.
    19. McClosky Benjamin & Ma Xiwen & Tanksley Steven D., 2011. "Quantifying the Relative Contribution of the Heterozygous Class to QTL Detection Power," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-19, May.
    20. Ziqi Chen & Jing Ning & Yu Shen & Jing Qin, 2021. "Combining primary cohort data with external aggregate information without assuming comparability," Biometrics, The International Biometric Society, vol. 77(3), pages 1024-1036, September.
    21. Zehua Chen, 2005. "The Full EM Algorithm for the MLEs of QTL Effects and Positions and Their Estimated Variances in Multiple-Interval Mapping," Biometrics, The International Biometric Society, vol. 61(2), pages 474-480, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0148620. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.