IDEAS home Printed from https://ideas.repec.org/p/upf/upfgen/691.html
   My bibliography  Save this paper

Honey, I shrunk the sample covariance matrix

Author

Listed:
  • Olivier Ledoit
  • Michael Wolf

Abstract

The central message of this paper is that nobody should be using the sample covariance matrix for the purpose of portfolio optimization. It contains estimation error of the kind most likely to perturb a mean-variance optimizer. In its place, we suggest using the matrix obtained from the sample covariance matrix through a transformation called shrinkage. This tends to pull the most extreme coefficients towards more central values, thereby systematically reducing estimation error where it matters most. Statistically, the challenge is to know the optimal shrinkage intensity, and we give the formula for that. Without changing any other step in the portfolio optimization process, we show on actual stock market data that shrinkage reduces tracking error relative to a benchmark index, and substantially increases the realized information ratio of the active portfolio manager.

Suggested Citation

  • Olivier Ledoit & Michael Wolf, 2003. "Honey, I shrunk the sample covariance matrix," Economics Working Papers 691, Department of Economics and Business, Universitat Pompeu Fabra.
    • Handle: RePEc:upf:upfgen:691
    as

    Download full text from publisher

    File URL: https://econ-papers.upf.edu/papers/691.pdf
    File Function: Whole Paper
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
    2. Connor, Gregory & Korajczyk, Robert A, 1993. "A Test for the Number of Factors in an Approximate Factor Model," Journal of Finance, American Finance Association, vol. 48(4), pages 1263-1291, September.
    3. Ravi Jagannathan & Tongshu Ma, 2003. "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps," Journal of Finance, American Finance Association, vol. 58(4), pages 1651-1683, August.
    4. Ledoit, Olivier & Wolf, Michael, 2003. "Improved estimation of the covariance matrix of stock returns with an application to portfolio selection," Journal of Empirical Finance, Elsevier, vol. 10(5), pages 603-621, December.
    5. Jorion, Philippe, 1986. "Bayes-Stein Estimation for Portfolio Analysis," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 21(3), pages 279-292, September.
    6. repec:bla:jfinan:v:58:y:2003:i:4:p:1651-1684 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael W. Brandt & Pedro Santa-Clara & Rossen Valkanov, 2009. "Parametric Portfolio Policies: Exploiting Characteristics in the Cross-Section of Equity Returns," The Review of Financial Studies, Society for Financial Studies, vol. 22(9), pages 3411-3447, September.
    2. Caicedo-Llano, Juliana & Dionysopoulos, Thomas, 2008. "Market integration: A risk-budgeting guide for pure alpha investors," Journal of Multinational Financial Management, Elsevier, vol. 18(4), pages 313-327, October.
    3. Michael Wolf, 2006. "Resampling vs. Shrinkage for Benchmarked Managers," IEW - Working Papers 263, Institute for Empirical Research in Economics - University of Zurich.
    4. Perreault, Samuel & Duchesne, Thierry & Nešlehová, Johanna G., 2019. "Detection of block-exchangeable structure in large-scale correlation matrices," Journal of Multivariate Analysis, Elsevier, vol. 169(C), pages 400-422.
    5. Daniel Espinoza & Eduardo Moreno, 2014. "A primal-dual aggregation algorithm for minimizing conditional value-at-risk in linear programs," Computational Optimization and Applications, Springer, vol. 59(3), pages 617-638, December.
    6. Popovic, Gordana C. & Hui, Francis K.C. & Warton, David I., 2018. "A general algorithm for covariance modeling of discrete data," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 86-100.
    7. Paolo Andreini & Donato Ceci, 2019. "A Horse Race in High Dimensional Space," CEIS Research Paper 452, Tor Vergata University, CEIS, revised 14 Feb 2019.
    8. Loriana Pelizzon & Massimiliano Caporin, 2012. "Market volatility, optimal portfolios and naive asset allocations," Working Papers 2012_08, Department of Economics, University of Venice "Ca' Foscari".
    9. Yuanrong Wang & Tomaso Aste, 2021. "Dynamic Portfolio Optimization with Inverse Covariance Clustering," Papers 2112.15499, arXiv.org, revised Jan 2022.
    10. Gopal K. Basak & Ravi Jagannathan & Tongshu Ma, 2004. "A Jackknife Estimator for Tracking Error Variance of Optimal Portfolios Constructed Using Estimated Inputs1," NBER Working Papers 10447, National Bureau of Economic Research, Inc.
    11. Rubio-García, Álvaro & Fernández-Lorenzo, Samuel & García-Ripoll, Juan José & Porras, Diego, 2024. "Accurate solution of the Index Tracking problem with a hybrid simulated annealing algorithm," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 639(C).
    12. Bessler, Wolfgang & Leonhardt, Alexander & Wolff, Dominik, 2016. "Analyzing hedging strategies for fixed income portfolios: A Bayesian approach for model selection," International Review of Financial Analysis, Elsevier, vol. 46(C), pages 239-256.
    13. Guidolin, Massimo & Wang, Kai, 2023. "The empirical performance of option implied volatility surface-driven optimal portfolios," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 618(C).
    14. Soufiane Hayou, 2017. "On the overestimation of the largest eigenvalue of a covariance matrix," Papers 1708.03551, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Candelon, B. & Hurlin, C. & Tokpavi, S., 2012. "Sampling error and double shrinkage estimation of minimum variance portfolios," Journal of Empirical Finance, Elsevier, vol. 19(4), pages 511-527.
    2. Mishra, Anil V., 2016. "Foreign bias in Australian-domiciled mutual fund holdings," Pacific-Basin Finance Journal, Elsevier, vol. 39(C), pages 101-123.
    3. Francisco Peñaranda & Enrique Sentana, 2024. "Portfolio management with big data," Working Papers wp2024_2411, CEMFI.
    4. Istvan Varga-Haszonits & Fabio Caccioli & Imre Kondor, 2016. "Replica approach to mean-variance portfolio optimization," Papers 1606.08679, arXiv.org.
    5. Mishra, Anil V., 2015. "Measures of equity home bias puzzle," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 293-312.
    6. Thomas Trier Bjerring & Omri Ross & Alex Weissensteiner, 2017. "Feature selection for portfolio optimization," Annals of Operations Research, Springer, vol. 256(1), pages 21-40, September.
    7. Fabio Caccioli & Imre Kondor & Matteo Marsili & Susanne Still, 2014. "$L_p$ regularized portfolio optimization," Papers 1404.4040, arXiv.org.
    8. Imre Kondor & G'abor Papp & Fabio Caccioli, 2016. "Analytic solution to variance optimization with no short-selling," Papers 1612.07067, arXiv.org, revised Jan 2017.
    9. Yan, Cheng & Zhang, Huazhu, 2017. "Mean-variance versus naïve diversification: The role of mispricing," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 48(C), pages 61-81.
    10. Long Zhao & Deepayan Chakrabarti & Kumar Muthuraman, 2019. "Portfolio Construction by Mitigating Error Amplification: The Bounded-Noise Portfolio," Operations Research, INFORMS, vol. 67(4), pages 965-983, July.
    11. Kourtis, Apostolos & Dotsis, George & Markellos, Raphael N., 2012. "Parameter uncertainty in portfolio selection: Shrinking the inverse covariance matrix," Journal of Banking & Finance, Elsevier, vol. 36(9), pages 2522-2531.
    12. Behr, Patrick & Guettler, Andre & Miebs, Felix, 2013. "On portfolio optimization: Imposing the right constraints," Journal of Banking & Finance, Elsevier, vol. 37(4), pages 1232-1242.
    13. André Alves Portela Santos, 2010. "The Out-of-Sample Performance of Robust Portfolio Optimization," Brazilian Review of Finance, Brazilian Society of Finance, vol. 8(2), pages 141-166.
    14. Lim Hao Shen Keith, 2024. "Covariance Matrix Analysis for Optimal Portfolio Selection," Papers 2407.08748, arXiv.org.
    15. Johannes Bock, 2018. "An updated review of (sub-)optimal diversification models," Papers 1811.08255, arXiv.org.
    16. Kremer, Philipp J. & Lee, Sangkyun & Bogdan, Małgorzata & Paterlini, Sandra, 2020. "Sparse portfolio selection via the sorted ℓ1-Norm," Journal of Banking & Finance, Elsevier, vol. 110(C).
    17. Yen, Yu-Min & Yen, Tso-Jung, 2014. "Solving norm constrained portfolio optimization via coordinate-wise descent algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 76(C), pages 737-759.
    18. Mishra, Anil V., 2017. "Foreign bias in Australia's international equity holdings," Review of Financial Economics, Elsevier, vol. 33(C), pages 41-54.
    19. Yuki Shigeta, 2016. "Optimality of Naive Investment Strategies in Dynamic MeanVariance Optimization Problems with Multiple Priors," Discussion papers e-16-004, Graduate School of Economics , Kyoto University.
    20. Aït-Sahalia, Yacine & Xiu, Dacheng, 2017. "Using principal component analysis to estimate a high dimensional factor model with high-frequency data," Journal of Econometrics, Elsevier, vol. 201(2), pages 384-399.

    More about this item

    Keywords

    Covariance matrix; Markovitz optimization; shrinkage; tracking error;
    All these keywords.

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G15 - Financial Economics - - General Financial Markets - - - International Financial Markets

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:upf:upfgen:691. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: http://www.econ.upf.edu/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.