IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v100y2009i9p2100-2111.html
   My bibliography  Save this article

Automatic model selection for partially linear models

Author

Listed:
  • Ni, Xiao
  • Zhang, Hao Helen
  • Zhang, Daowen

Abstract

We propose and study a unified procedure for variable selection in partially linear models. A new type of double-penalized least squares is formulated, using the smoothing spline to estimate the nonparametric part and applying a shrinkage penalty on parametric components to achieve model parsimony. Theoretically we show that, with proper choices of the smoothing and regularization parameters, the proposed procedure can be as efficient as the oracle estimator [J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of American Statistical Association 96 (2001) 1348-1360]. We also study the asymptotic properties of the estimator when the number of parametric effects diverges with the sample size. Frequentist and Bayesian estimates of the covariance and confidence intervals are derived for the estimators. One great advantage of this procedure is its linear mixed model (LMM) representation, which greatly facilitates its implementation by using standard statistical software. Furthermore, the LMM framework enables one to treat the smoothing parameter as a variance component and hence conveniently estimate it together with other regression coefficients. Extensive numerical studies are conducted to demonstrate the effective performance of the proposed procedure.

Suggested Citation

  • Ni, Xiao & Zhang, Hao Helen & Zhang, Daowen, 2009. "Automatic model selection for partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 100(9), pages 2100-2111, October.
    • Handle: RePEc:eee:jmvana:v:100:y:2009:i:9:p:2100-2111
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047-259X(09)00117-1
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, November.
    2. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    3. Liang, Hua, 2006. "Estimation in partially linear models and numerical comparisons," Computational Statistics & Data Analysis, Elsevier, vol. 50(3), pages 675-687, February.
    4. Jianqing Fan & Runze Li, 2004. "New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 710-723, January.
    5. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, November.
    6. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Guozhi Hu & Weihu Cheng & Jie Zeng, 2023. "Optimal Model Averaging for Semiparametric Partially Linear Models with Censored Data," Mathematics, MDPI, vol. 11(3), pages 1-21, February.
    2. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    3. Hu Yang & Ning Li & Jing Yang, 2020. "A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates," Statistical Papers, Springer, vol. 61(5), pages 1911-1937, October.
    4. Mingqiu Wang & Guo-Liang Tian, 2016. "Robust group non-convex estimations for high-dimensional partially linear models," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 28(1), pages 49-67, March.
    5. Du, Pang & Cheng, Guang & Liang, Hua, 2012. "Semiparametric regression models with additive nonparametric components and high dimensional parametric components," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 2006-2017.
    6. Cui, Wenquan & Cheng, Haoyang & Sun, Jiajing, 2018. "An RKHS-based approach to double-penalized regression in high-dimensional partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 201-210.
    7. Guang Cheng & Hao Zhang & Zuofeng Shang, 2015. "Sparse and efficient estimation for partial spline models with increasing dimension," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(1), pages 93-127, February.
    8. Feng Li & Lu Lin & Yuxia Su, 2013. "Variable selection and parameter estimation for partially linear models via Dantzig selector," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(2), pages 225-238, February.
    9. Aifen Feng & Xiaogai Chang & Youlin Shang & Jingya Fan, 2022. "Application of the ADMM Algorithm for a High-Dimensional Partially Linear Model," Mathematics, MDPI, vol. 10(24), pages 1-13, December.
    10. Arfan Raheen Afzal & Jing Yang & Xuewen Lu, 2021. "Variable selection in partially linear additive hazards model with grouped covariates and a diverging number of parameters," Computational Statistics, Springer, vol. 36(2), pages 829-855, June.
    11. Germán Aneiros & Philippe Vieu, 2015. "Partial linear modelling with multi-functional covariates," Computational Statistics, Springer, vol. 30(3), pages 647-671, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guang Cheng & Hao Zhang & Zuofeng Shang, 2015. "Sparse and efficient estimation for partial spline models with increasing dimension," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(1), pages 93-127, February.
    2. Feng Li & Lu Lin & Yuxia Su, 2013. "Variable selection and parameter estimation for partially linear models via Dantzig selector," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(2), pages 225-238, February.
    3. Yazhao Lv & Riquan Zhang & Weihua Zhao & Jicai Liu, 2015. "Quantile regression and variable selection of partial linear single-index model," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(2), pages 375-409, April.
    4. Gerhard Tutz & Margret-Ruth Oelker, 2017. "Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures," International Statistical Review, International Statistical Institute, vol. 85(2), pages 204-227, August.
    5. Degao Li & Guodong Li & Jinhong You, 2014. "Significant Variable Selection And Autoregressive Order Determination For Time-Series Partially Linear Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 35(5), pages 478-490, August.
    6. Xiao Ni & Daowen Zhang & Hao Helen Zhang, 2010. "Variable Selection for Semiparametric Mixed Models in Longitudinal Studies," Biometrics, The International Biometric Society, vol. 66(1), pages 79-88, March.
    7. Liu, Jingyuan & Lou, Lejia & Li, Runze, 2018. "Variable selection for partially linear models via partial correlation," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 418-434.
    8. Shan Luo & Zehua Chen, 2014. "Sequential Lasso Cum EBIC for Feature Selection With Ultra-High Dimensional Feature Space," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1229-1240, September.
    9. Lai, Peng & Wang, Qihua & Lian, Heng, 2012. "Bias-corrected GEE estimation and smooth-threshold GEE variable selection for single-index models with clustered data," Journal of Multivariate Analysis, Elsevier, vol. 105(1), pages 422-432.
    10. Xingwei Tong & Xin He & Liuquan Sun & Jianguo Sun, 2009. "Variable Selection for Panel Count Data via Non‐Concave Penalized Estimating Function," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(4), pages 620-635, December.
    11. Cui, Wenquan & Cheng, Haoyang & Sun, Jiajing, 2018. "An RKHS-based approach to double-penalized regression in high-dimensional partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 201-210.
    12. Takuma Yoshida, 2019. "Two stage smoothing in additive models with missing covariates," Statistical Papers, Springer, vol. 60(6), pages 1803-1826, December.
    13. Peixin Zhao & Liugen Xue, 2012. "Variable selection in semiparametric regression analysis for longitudinal data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(1), pages 213-231, February.
    14. Gerhard Tutz & Jan Gertheiss, 2014. "Rating Scales as Predictors—The Old Question of Scale Level and Some Answers," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 357-376, July.
    15. Raheem, S.M. Enayetur & Ahmed, S. Ejaz & Doksum, Kjell A., 2012. "Absolute penalty and shrinkage estimation in partially linear models," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 874-891.
    16. Zhao, Peixin & Xue, Liugen, 2010. "Variable selection for semiparametric varying coefficient partially linear errors-in-variables models," Journal of Multivariate Analysis, Elsevier, vol. 101(8), pages 1872-1883, September.
    17. Yawei He & Zehua Chen, 2016. "The EBIC and a sequential procedure for feature selection in interactive linear models with high-dimensional data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 68(1), pages 155-180, February.
    18. Gao, Yan & Zhang, Xinyu & Wang, Shouyang & Zou, Guohua, 2016. "Model averaging based on leave-subject-out cross-validation," Journal of Econometrics, Elsevier, vol. 192(1), pages 139-151.
    19. Li, Haocheng & Shu, Di & He, Wenqing & Yi, Grace Y., 2019. "Variable selection via the composite likelihood method for multilevel longitudinal data with missing responses and covariates," Computational Statistics & Data Analysis, Elsevier, vol. 135(C), pages 25-34.
    20. Gao, Jiti, 2007. "Nonlinear time series: semiparametric and nonparametric methods," MPRA Paper 39563, University Library of Munich, Germany, revised 01 Sep 2007.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:100:y:2009:i:9:p:2100-2111. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.