IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v40y2015i2p136-157.html
   My bibliography  Save this article

Weakly Informative Prior for Point Estimation of Covariance Matrices in Hierarchical Models

Author

Listed:
  • Yeojin Chung

    (Kookmin University)

  • Andrew Gelman

    (Columbia University)

  • Sophia Rabe-Hesketh

    (University of California)

  • Jingchen Liu

    (Columbia University)

  • Vincent Dorie

    (New York University)

Abstract

When fitting hierarchical regression models, maximum likelihood (ML) estimation has computational (and, for some users, philosophical) advantages compared to full Bayesian inference, but when the number of groups is small, estimates of the covariance matrix (Σ) of group-level varying coefficients are often degenerate. One can do better, even from a purely point estimation perspective, by using a prior distribution or penalty function. In this article, we use Bayes modal estimation to obtain positive definite covariance matrix estimates. We recommend a class of Wishart (not inverse-Wishart) priors for Σ with a default choice of hyperparameters, that is, the degrees of freedom are set equal to the number of varying coefficients plus 2, and the scale matrix is the identity matrix multiplied by a value that is large relative to the scale of the problem. This prior is equivalent to independent gamma priors for the eigenvalues of Σ with shape parameter 1.5 and rate parameter close to 0. It is also equivalent to independent gamma priors for the variances with the same hyperparameters multiplied by a function of the correlation coefficients. With this default prior, the posterior mode for Σ is always strictly positive definite. Furthermore, the resulting uncertainty for the fixed coefficients is less underestimated than under classical ML or restricted maximum likelihood estimation. We also suggest an extension of our method that can be used when stronger prior information is available for some of the variances or correlations.

Suggested Citation

  • Yeojin Chung & Andrew Gelman & Sophia Rabe-Hesketh & Jingchen Liu & Vincent Dorie, 2015. "Weakly Informative Prior for Point Estimation of Covariance Matrices in Hierarchical Models," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 136-157, April.
  • Handle: RePEc:sae:jedbes:v:40:y:2015:i:2:p:136-157
    DOI: 10.3102/1076998615570945
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998615570945
    Download Restriction: no

    File URL: https://libkey.io/10.3102/1076998615570945?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Warton, David I., 2008. "Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 340-349, March.
    2. Li, Huilin & Lahiri, P., 2010. "An adjusted maximum likelihood method for solving small area estimation problems," Journal of Multivariate Analysis, Elsevier, vol. 101(4), pages 882-892, April.
    3. James Martin & Roderick McDonald, 1975. "Bayesian estimation in unrestricted factor analysis: A treatment for heywood cases," Psychometrika, Springer;The Psychometric Society, vol. 40(4), pages 505-517, December.
    4. E. Maris, 1999. "Estimating multiple classification latent class models," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 187-212, June.
    5. Hariharan Swaminathan & Janice Gifford, 1985. "Bayesian estimation in the two-parameter logistic model," Psychometrika, Springer;The Psychometric Society, vol. 50(3), pages 349-364, September.
    6. Gabriela Ciuperca & Andrea Ridolfi & Jérôme Idier, 2003. "Penalized Maximum Likelihood Estimator for Normal Mixtures," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 30(1), pages 45-59, March.
    7. Rabe-Hesketh, Sophia & Skrondal, Anders & Pickles, Andrew, 2005. "Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects," Journal of Econometrics, Elsevier, vol. 128(2), pages 301-323, October.
    8. Robert Mislevy, 1986. "Bayes modal estimation in item response models," Psychometrika, Springer;The Psychometric Society, vol. 51(2), pages 177-195, June.
    9. Yeojin Chung & Sophia Rabe-Hesketh & Vincent Dorie & Andrew Gelman & Jingchen Liu, 2013. "A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 685-709, October.
    10. Robert Tsutakawa & Hsin Lin, 1986. "Bayesian estimation of item response curves," Psychometrika, Springer;The Psychometric Society, vol. 51(2), pages 251-267, June.
    11. Stanislav Kolenikov & Kenneth A. Bollen, 2012. "Testing Negative Error Variances," Sociological Methods & Research, , vol. 41(1), pages 124-167, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Meager, Rachael, 2019. "Understanding the average impact of microcredit expansions: a Bayesian hierarchical analysis of seven randomized experiments," LSE Research Online Documents on Economics 88190, London School of Economics and Political Science, LSE Library.
    2. Hein, Maren & Kurz, Peter & Steiner, Winfried J., 2019. "On the effect of HB covariance matrix prior settings: A simulation study," Journal of choice modelling, Elsevier, vol. 31(C), pages 51-72.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yeojin Chung & Sophia Rabe-Hesketh & Vincent Dorie & Andrew Gelman & Jingchen Liu, 2013. "A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 685-709, October.
    2. Ogasawara, Haruhiko, 2013. "Asymptotic cumulants of ability estimators using fallible item parameters," Journal of Multivariate Analysis, Elsevier, vol. 119(C), pages 144-162.
    3. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    4. Sheng, Yanyan, 2008. "Markov Chain Monte Carlo Estimation of Normal Ogive IRT Models in MATLAB," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 25(i08).
    5. repec:jss:jstsof:25:i08 is not listed on IDEAS
    6. Pere Ferrando, 2007. "A Pearson-Type-VII item response model for assessing person fluctuation," Psychometrika, Springer;The Psychometric Society, vol. 72(1), pages 25-41, March.
    7. Vitoratou, Silia & Ntzoufras, Ioannis & Moustaki, Irini, 2016. "Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions," LSE Research Online Documents on Economics 57685, London School of Economics and Political Science, LSE Library.
    8. Pittau, Maria Grazia & Farcomeni, Alessio & Zelli, Roberto, 2016. "Has the attitude of US citizens towards redistribution changed over time?," Economic Modelling, Elsevier, vol. 52(PB), pages 714-724.
    9. Robert Jannarone & Kai Yu & James Laughlin, 1990. "Easy bayes estimation for rasch-type models," Psychometrika, Springer;The Psychometric Society, vol. 55(3), pages 449-460, September.
    10. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    11. Edgar C. Merkle & Daniel Furr & Sophia Rabe-Hesketh, 2019. "Bayesian Comparison of Latent Variable Models: Conditional Versus Marginal Likelihoods," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 802-829, September.
    12. Li Cai, 2010. "High-dimensional Exploratory Item Factor Analysis by A Metropolis–Hastings Robbins–Monro Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 33-57, March.
    13. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    14. Simon Gaechter & Chris Starmer & Fabio Tufano, 2022. "Measuring “group cohesion” to reveal the power of social relationships in team production," Discussion Papers 2022-12, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    15. Roberto Rocci & Stefano Antonio Gattone & Roberto Di Mari, 2018. "A data driven equivariant approach to constrained Gaussian mixture modeling," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 235-260, June.
    16. Saul Estrin & Julia Korosteleva & Tomasz Mickiewicz, 2022. "Schumpeterian Entry: Innovation, Exporting, and Growth Aspirations of Entrepreneurs," Entrepreneurship Theory and Practice, , vol. 46(2), pages 269-296, March.
    17. Godager, Geir & Iversen, Tor & Ma, Ching-to Albert, 2015. "Competition, gatekeeping, and health care access," Journal of Health Economics, Elsevier, vol. 39(C), pages 159-170.
    18. Namin, Aidin & Soysal, Gonca P. & Ratchford, Brian T., 2022. "Alleviating demand uncertainty for seasonal goods: An analysis of attribute-based markdown policy for fashion retailers," Journal of Business Research, Elsevier, vol. 145(C), pages 671-681.
    19. Lawrence Choo & Todd R. Kaplan & Ro’i Zultan, 2019. "Information aggregation in Arrow–Debreu markets: an experiment," Experimental Economics, Springer;Economic Science Association, vol. 22(3), pages 625-652, September.
    20. Hans-Friedrich Köhn & Chia-Yi Chiu, 2016. "A Proof of the Duality of the DINA Model and the DINO Model," Journal of Classification, Springer;The Classification Society, vol. 33(2), pages 171-184, July.
    21. Jan Pablo Burgard & Domingo Morales & Anna-Lena Wölwer, 2022. "Small area estimation of socioeconomic indicators for sampled and unsampled domains," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 106(2), pages 287-314, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:40:y:2015:i:2:p:136-157. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.