IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v81y2011i8p1056-1062.html
   My bibliography  Save this article

Sparse variational analysis of linear mixed models for large data sets

Author

Listed:
  • Armagan, Artin
  • Dunson, David

Abstract

It is increasingly common to be faced with longitudinal or multi-level data sets that have large numbers of predictors and/or a large sample size. Current methods of fitting and inference for mixed effects models tend to perform poorly in such settings. When there are many variables, it is appealing to allow uncertainty in subset selection and to obtain a sparse characterization of the data. Bayesian methods are available to address these goals using Markov chain Monte Carlo (MCMC), but MCMC is very computationally expensive and can be infeasible in large p and/or large n problems. As a fast approximate Bayes solution, we recommend a novel approximation to the posterior relying on variational methods. Variational methods are used to approximate the posterior of the parameters in a decomposition of the variance components, with priors chosen to obtain a sparse solution that allows selection of random effects. The method is evaluated through a simulation study, and applied to an epidemiological application.

Suggested Citation

  • Armagan, Artin & Dunson, David, 2011. "Sparse variational analysis of linear mixed models for large data sets," Statistics & Probability Letters, Elsevier, vol. 81(8), pages 1056-1062, August.
    • Handle: RePEc:eee:stapro:v:81:y:2011:i:8:p:1056-1062
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715211000757
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
    2. Zhen Chen & David B. Dunson, 2003. "Random Effects Selection in Linear Mixed Models," Biometrics, The International Biometric Society, vol. 59(4), pages 762-769, December.
    3. Park, Trevor & Casella, George, 2008. "The Bayesian Lasso," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 681-686, June.
    4. Satkartar K. Kinney & David B. Dunson, 2007. "Fixed and Random Effects Selection in Linear and Logistic Models," Biometrics, The International Biometric Society, vol. 63(3), pages 690-698, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xiao Su & Yuguo Chen, 2021. "Variational approximation for importance sampling," Computational Statistics, Springer, vol. 36(3), pages 1901-1930, September.
    2. F. S. Nathoo & A. Babul & A. Moiseev & N. Virji-Babul & M. F. Beg, 2014. "A variational Bayes spatiotemporal model for electromagnetic brain mapping," Biometrics, The International Biometric Society, vol. 70(1), pages 132-143, March.
    3. Patrick O. Perry, 2017. "Fast moment-based estimation for hierarchical models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(1), pages 267-291, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mingan Yang & Min Wang & Guanghui Dong, 2020. "Bayesian variable selection for mixed effects model with shrinkage prior," Computational Statistics, Springer, vol. 35(1), pages 227-243, March.
    2. Howard D. Bondell & Arun Krishna & Sujit K. Ghosh, 2010. "Joint Variable Selection for Fixed and Random Effects in Linear Mixed-Effects Models," Biometrics, The International Biometric Society, vol. 66(4), pages 1069-1077, December.
    3. Wang, Hao, 2010. "Sparse seemingly unrelated regression modelling: Applications in finance and econometrics," Computational Statistics & Data Analysis, Elsevier, vol. 54(11), pages 2866-2877, November.
    4. Dimitris Korobilis, 2013. "Var Forecasting Using Bayesian Variable Selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 204-230, March.
    5. Mingan Yang, 2020. "Bayesian Mixed Effects Model with Variable Selection," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 10(2), pages 27-29, August.
    6. Joyee Ghosh & Amy H. Herring & Anna Maria Siega-Riz, 2011. "Bayesian Variable Selection for Latent Class Models," Biometrics, The International Biometric Society, vol. 67(3), pages 917-925, September.
    7. Sylvia Fruhwirth-Schnatter & Peter Knaus, 2022. "Sparse Bayesian State-Space and Time-Varying Parameter Models," Papers 2207.12147, arXiv.org.
    8. Wagner, Helga & Duller, Christine, 2012. "Bayesian model selection for logistic regression models with random intercept," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1256-1274.
    9. Imori, Shinpei & Rosen, Dietrich von, 2015. "Covariance components selection in high-dimensional growth curve model with random coefficients," Journal of Multivariate Analysis, Elsevier, vol. 136(C), pages 86-94.
    10. Tsai-Hung Fan & Yi-Fu Wang & Yi-Chen Zhang, 2014. "Bayesian model selection in linear mixed effects models with autoregressive(p) errors using mixture priors," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(8), pages 1814-1829, August.
    11. Lee, Kyeong Eun & Kim, Yongku & Xu, Ronghui, 2014. "Bayesian variable selection under the proportional hazards mixed-effects model," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 53-65.
    12. Daniel R. Kowal, 2023. "Subset selection for linear mixed models," Biometrics, The International Biometric Society, vol. 79(3), pages 1853-1867, September.
    13. Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2010. "Stochastic model specification search for Gaussian and partial non-Gaussian state space models," Journal of Econometrics, Elsevier, vol. 154(1), pages 85-100, January.
    14. Dimitris Korobilis & Kenichi Shimizu, 2022. "Bayesian Approaches to Shrinkage and Sparse Estimation," Foundations and Trends(R) in Econometrics, now publishers, vol. 11(4), pages 230-354, June.
    15. Xinyu Zhang & Hua Liang & Anna Liu & David Ruppert & Guohua Zou, 2016. "Selection Strategy for Covariance Structure of Random Effects in Linear Mixed-effects Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(1), pages 275-291, March.
    16. Yang, Mingan, 2012. "Bayesian variable selection for logistic mixed model with nonparametric random effects," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2663-2674.
    17. Alhamzawi, Rahim, 2016. "Bayesian model selection in ordinal quantile regression," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 68-78.
    18. Li, Chunyu & Lou, Chenxin & Luo, Dan & Xing, Kai, 2021. "Chinese corporate distress prediction using LASSO: The role of earnings management," International Review of Financial Analysis, Elsevier, vol. 76(C).
    19. Lam, Clifford, 2008. "Estimation of large precision matrices through block penalization," LSE Research Online Documents on Economics 31543, London School of Economics and Political Science, LSE Library.
    20. Martin Feldkircher & Florian Huber & Gary Koop & Michael Pfarrhofer, 2022. "APPROXIMATE BAYESIAN INFERENCE AND FORECASTING IN HUGE‐DIMENSIONAL MULTICOUNTRY VARs," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 63(4), pages 1625-1658, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:81:y:2011:i:8:p:1056-1062. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.