IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v137y2015icp161-172.html
   My bibliography  Save this article

Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions

Author

Listed:
  • Cai, T. Tony
  • Liang, Tengyuan
  • Zhou, Harrison H.

Abstract

Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high-dimensional setting optimal estimation of the differential entropy and the log-determinant of the covariance matrix. We first establish a central limit theorem for the log determinant of the sample covariance matrix in the high-dimensional setting where the dimension p(n) can grow with the sample size n. An estimator of the differential entropy and the log determinant is then considered. Optimal rate of convergence is obtained. It is shown that in the case p(n)/n→0 the estimator is asymptotically sharp minimax. The ultra-high-dimensional setting where p(n)>n is also discussed.

Suggested Citation

  • Cai, T. Tony & Liang, Tengyuan & Zhou, Harrison H., 2015. "Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions," Journal of Multivariate Analysis, Elsevier, vol. 137(C), pages 161-172.
    • Handle: RePEc:eee:jmvana:v:137:y:2015:i:c:p:161-172
    DOI: 10.1016/j.jmva.2015.02.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047259X1500038X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jmva.2015.02.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Misra, Neeraj & Singh, Harshinder & Demchuk, Eugene, 2005. "Estimation of the entropy of a multivariate normal distribution," Journal of Multivariate Analysis, Elsevier, vol. 92(2), pages 324-342, February.
    2. Jonsson, Dag, 1982. "Some limit theorems for the eigenvalues of a sample covariance matrix," Journal of Multivariate Analysis, Elsevier, vol. 12(1), pages 1-38, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrew Martinez, 2017. "Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter," Working Papers (Old Series) 1717, Federal Reserve Bank of Cleveland.
    2. Tengyuan Liang, 2020. "How Well Generative Adversarial Networks Learn Distributions," Working Papers 2020-154, Becker Friedman Institute for Research In Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Jiaqi & Zhang, Yangchun & Li, Weiming & Tian, Boping, 2018. "A supplement on CLT for LSS under a large dimensional generalized spiked covariance model," Statistics & Probability Letters, Elsevier, vol. 138(C), pages 57-65.
    2. Birke, Melanie & Dette, Holger, 2005. "A note on testing the covariance matrix for large dimension," Statistics & Probability Letters, Elsevier, vol. 74(3), pages 281-289, October.
    3. Kayal, Suchandan & Kumar, Somesh, 2013. "Estimation of the Shannon’s entropy of several shifted exponential populations," Statistics & Probability Letters, Elsevier, vol. 83(4), pages 1127-1135.
    4. Bender, Martin, 2008. "Global fluctuations in general [beta] Dyson's Brownian motion," Stochastic Processes and their Applications, Elsevier, vol. 118(6), pages 1022-1042, June.
    5. Friesen, Olga & Löwe, Matthias & Stolz, Michael, 2013. "Gaussian fluctuations for sample covariance matrices with dependent data," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 270-287.
    6. Jan Nagel, 2021. "A Functional CLT for Partial Traces of Random Matrices," Journal of Theoretical Probability, Springer, vol. 34(2), pages 953-974, June.
    7. Adhikari, Kartick & Saha, Koushik, 2018. "Universality in the fluctuation of eigenvalues of random circulant matrices," Statistics & Probability Letters, Elsevier, vol. 138(C), pages 1-8.
    8. Birke, Melanie & Dette, Holger, 2003. "A note on testing the covariance matrix for large dimension," Technical Reports 2004,02, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    9. Withers, Christopher S. & Nadarajah, Saralees, 2011. "Estimates of low bias for the multivariate normal," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1635-1647, November.
    10. Mohammed S. Kotb & Huda M. Alomari, 2024. "Estimating the entropy of a Rayleigh model under progressive first-failure censoring," Statistical Papers, Springer, vol. 65(5), pages 3135-3154, July.
    11. Tingting Zou & Shurong Zheng & Zhidong Bai & Jianfeng Yao & Hongtu Zhu, 2022. "CLT for linear spectral statistics of large dimensional sample covariance matrices with dependent data," Statistical Papers, Springer, vol. 63(2), pages 605-664, April.
    12. S. Chatterjee & A. Bose, 2004. "A New Method for Bounding Rates of Convergence of Empirical Spectral Distributions," Journal of Theoretical Probability, Springer, vol. 17(4), pages 1003-1019, October.
    13. Mansoor Sheikh & A. C. C. Coolen, 2020. "Accurate Bayesian Data Classification Without Hyperparameter Cross-Validation," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 277-297, July.
    14. Yao, Jianfeng, 2012. "A note on a Marčenko–Pastur type theorem for time series," Statistics & Probability Letters, Elsevier, vol. 82(1), pages 22-28.
    15. Yu, Philip L.H. & Wang, Xiaohang & Zhu, Yuanyuan, 2017. "High dimensional covariance matrix estimation by penalizing the matrix-logarithm transformed likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 114(C), pages 12-25.
    16. Dörnemann, Nina & Dette, Holger, 2023. "Fluctuations of the diagonal entries of a large sample precision matrix," Statistics & Probability Letters, Elsevier, vol. 198(C).
    17. Bai, Zhidong & Silverstein, Jack W., 2022. "A tribute to P.R. Krishnaiah," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    18. Ningning Xia & Zhidong Bai, 2015. "Functional CLT of eigenvectors for large sample covariance matrices," Statistical Papers, Springer, vol. 56(1), pages 23-60, February.
    19. Klein, Daniel & Pielaszkiewicz, Jolanta & Filipiak, Katarzyna, 2022. "Approximate normality in testing an exchangeable covariance structure under large- and high-dimensional settings," Journal of Multivariate Analysis, Elsevier, vol. 192(C).
    20. Srivastava, Muni S. & Kollo, Tõnu & von Rosen, Dietrich, 2011. "Some tests for the covariance matrix with fewer observations than the dimension under non-normality," Journal of Multivariate Analysis, Elsevier, vol. 102(6), pages 1090-1103, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:137:y:2015:i:c:p:161-172. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.