IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v37y2017i8p1532-1549.html
   My bibliography  Save this article

Big Data Challenges of High‐Dimensional Continuous‐Time Mean‐Variance Portfolio Selection and a Remedy

Author

Listed:
  • Mei Choi Chiu
  • Chi Seng Pun
  • Hoi Ying Wong

Abstract

Investors interested in the global financial market must analyze financial securities internationally. Making an optimal global investment decision involves processing a huge amount of data for a high‐dimensional portfolio. This article investigates the big data challenges of two mean‐variance optimal portfolios: continuous‐time precommitment and constant‐rebalancing strategies. We show that both optimized portfolios implemented with the traditional sample estimates converge to the worst performing portfolio when the portfolio size becomes large. The crux of the problem is the estimation error accumulated from the huge dimension of stock data. We then propose a linear programming optimal (LPO) portfolio framework, which applies a constrained ℓ1 minimization to the theoretical optimal control to mitigate the risk associated with the dimensionality issue. The resulting portfolio becomes a sparse portfolio that selects stocks with a data‐driven procedure and hence offers a stable mean‐variance portfolio in practice. When the number of observations becomes large, the LPO portfolio converges to the oracle optimal portfolio, which is free of estimation error, even though the number of stocks grows faster than the number of observations. Our numerical and empirical studies demonstrate the superiority of the proposed approach.

Suggested Citation

  • Mei Choi Chiu & Chi Seng Pun & Hoi Ying Wong, 2017. "Big Data Challenges of High‐Dimensional Continuous‐Time Mean‐Variance Portfolio Selection and a Remedy," Risk Analysis, John Wiley & Sons, vol. 37(8), pages 1532-1549, August.
    • Handle: RePEc:wly:riskan:v:37:y:2017:i:8:p:1532-1549
    DOI: 10.1111/risa.12801
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/risa.12801
    Download Restriction: no
    File URL: https://libkey.io/10.1111/risa.12801?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    2. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    3. Ravi Jagannathan & Tongshu Ma, 2003. "Risk Reduction in Large Portfolios: Why Imposing the Wrong Constraints Helps," Journal of Finance, American Finance Association, vol. 58(4), pages 1651-1683, August.
    4. Victor DeMiguel & Lorenzo Garlappi & Francisco J. Nogales & Raman Uppal, 2009. "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms," Management Science, INFORMS, vol. 55(5), pages 798-812, May.
    5. Pun, Chi Seng & Wong, Hoi Ying, 2015. "Robust investment–reinsurance optimization with multiscale stochastic volatility," Insurance: Mathematics and Economics, Elsevier, vol. 62(C), pages 245-256.
    6. Andreas Tsanakas & Pietro Millossovich, 2016. "Sensitivity Analysis Using Risk Measures," Risk Analysis, John Wiley & Sons, vol. 36(1), pages 30-48, January.
    7. Seunggeun Lee & Fei Zou & Fred A. Wright, 2014. "Convergence of sample eigenvalues, eigenvectors, and principal component scores for ultra-high dimensional data," Biometrika, Biometrika Trust, vol. 101(2), pages 484-490.
    8. Jianqing Fan & Jingjin Zhang & Ke Yu, 2012. "Vast Portfolio Selection With Gross-Exposure Constraints," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 592-606, June.
    9. Lorenzo Garlappi & Raman Uppal & Tan Wang, 2007. "Portfolio Selection with Parameter and Model Uncertainty: A Multi-Prior Approach," The Review of Financial Studies, Society for Financial Studies, vol. 20(1), pages 41-81, January.
    10. John Y. Campbell & Luis M. Viceira, 1999. "Consumption and Portfolio Decisions when Expected Returns are Time Varying," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 433-495.
    11. Choi, Tsan-Ming & Li, Duan & Yan, Houmin, 2008. "Mean-variance analysis of a single supplier and retailer supply chain under a returns policy," European Journal of Operational Research, Elsevier, vol. 184(1), pages 356-376, January.
    12. Jianqing Fan & Yuan Liao & Han Liu, 2016. "An overview of the estimation of large covariance and precision matrices," Econometrics Journal, Royal Economic Society, vol. 19(1), pages 1-32, February.
    13. Duan Li & Wan‐Lung Ng, 2000. "Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation," Mathematical Finance, Wiley Blackwell, vol. 10(3), pages 387-406, July.
    14. Kolm, Petter N. & Tütüncü, Reha & Fabozzi, Frank J., 2014. "60 Years of portfolio optimization: Practical challenges and current trends," European Journal of Operational Research, Elsevier, vol. 234(2), pages 356-371.
    15. Dimitrina S. Dimitrova & Vladimir K. Kaishev & Shouqi Zhao, 2015. "Modeling Finite‐Time Failure Probabilities in Risk Analysis Applications," Risk Analysis, John Wiley & Sons, vol. 35(10), pages 1919-1939, October.
    16. Duan Li & Xiaoling Sun & Jun Wang, 2006. "Optimal Lot Solution To Cardinality Constrained Mean–Variance Formulation For Portfolio Selection," Mathematical Finance, Wiley Blackwell, vol. 16(1), pages 83-101, January.
    17. Jianjun Gao & Duan Li, 2013. "Optimal Cardinality Constrained Portfolio Selection," Operations Research, INFORMS, vol. 61(3), pages 745-761, June.
    18. repec:bla:jfinan:v:58:y:2003:i:4:p:1651-1684 is not listed on IDEAS
    19. Fan, Jianqing & Fan, Yingying & Lv, Jinchi, 2008. "High dimensional covariance matrix estimation using a factor model," Journal of Econometrics, Elsevier, vol. 147(1), pages 186-197, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pun, Chi Seng, 2018. "Time-consistent mean-variance portfolio selection with only risky assets," Economic Modelling, Elsevier, vol. 75(C), pages 281-292.
    2. Pun, Chi Seng & Wong, Hoi Ying, 2019. "A linear programming model for selection of sparse high-dimensional multiperiod portfolios," European Journal of Operational Research, Elsevier, vol. 273(2), pages 754-771.
    3. Pun, Chi Seng & Hadimaja, Matthew Zakharia, 2021. "A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions," Computational Statistics & Data Analysis, Elsevier, vol. 155(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pun, Chi Seng & Wong, Hoi Ying, 2019. "A linear programming model for selection of sparse high-dimensional multiperiod portfolios," European Journal of Operational Research, Elsevier, vol. 273(2), pages 754-771.
    2. Min Dai & Hanqing Jin & Steven Kou & Yuhong Xu, 2021. "A Dynamic Mean-Variance Analysis for Log Returns," Management Science, INFORMS, vol. 67(2), pages 1093-1108, February.
    3. Ding, Yi & Li, Yingying & Zheng, Xinghua, 2021. "High dimensional minimum variance portfolio estimation under statistical factor models," Journal of Econometrics, Elsevier, vol. 222(1), pages 502-515.
    4. Seyoung Park & Eun Ryung Lee & Sungchul Lee & Geonwoo Kim, 2019. "Dantzig Type Optimization Method with Applications to Portfolio Selection," Sustainability, MDPI, vol. 11(11), pages 1-32, June.
    5. De Nard, Gianluca & Zhao, Zhao, 2023. "Using, taming or avoiding the factor zoo? A double-shrinkage estimator for covariance matrices," Journal of Empirical Finance, Elsevier, vol. 72(C), pages 23-35.
    6. Kremer, Philipp J. & Lee, Sangkyun & Bogdan, Małgorzata & Paterlini, Sandra, 2020. "Sparse portfolio selection via the sorted ℓ1-Norm," Journal of Banking & Finance, Elsevier, vol. 110(C).
    7. Aït-Sahalia, Yacine & Xiu, Dacheng, 2017. "Using principal component analysis to estimate a high dimensional factor model with high-frequency data," Journal of Econometrics, Elsevier, vol. 201(2), pages 384-399.
    8. Chakrabarti, Deepayan, 2021. "Parameter-free robust optimization for the maximum-Sharpe portfolio problem," European Journal of Operational Research, Elsevier, vol. 293(1), pages 388-399.
    9. Liusha Yang & Romain Couillet & Matthew R. McKay, 2015. "A Robust Statistics Approach to Minimum Variance Portfolio Optimization," Papers 1503.08013, arXiv.org.
    10. Fan, Jianqing & Liao, Yuan & Shi, Xiaofeng, 2015. "Risks of large portfolios," Journal of Econometrics, Elsevier, vol. 186(2), pages 367-387.
    11. Maillet, Bertrand & Tokpavi, Sessi & Vaucher, Benoit, 2015. "Global minimum variance portfolio optimisation under some model risk: A robust regression-based approach," European Journal of Operational Research, Elsevier, vol. 244(1), pages 289-299.
    12. Hautsch, Nikolaus & Voigt, Stefan, 2019. "Large-scale portfolio allocation under transaction costs and model uncertainty," Journal of Econometrics, Elsevier, vol. 212(1), pages 221-240.
    13. Gianluca De Nard & Olivier Ledoit & Michael Wolf, 2018. "Factor models for portfolio selection in large dimensions: the good, the better and the ugly," ECON - Working Papers 290, Department of Economics - University of Zurich, revised Dec 2018.
    14. Cai, T. Tony & Hu, Jianchang & Li, Yingying & Zheng, Xinghua, 2020. "High-dimensional minimum variance portfolio estimation based on high-frequency data," Journal of Econometrics, Elsevier, vol. 214(2), pages 482-494.
    15. Thomas Conlon & John Cotter & Iason Kynigakis, 2021. "Machine Learning and Factor-Based Portfolio Optimization," Papers 2107.13866, arXiv.org.
    16. Fan, Qingliang & Wu, Ruike & Yang, Yanrong & Zhong, Wei, 2024. "Time-varying minimum variance portfolio," Journal of Econometrics, Elsevier, vol. 239(2).
    17. Meade, N. & Beasley, J.E. & Adcock, C.J., 2021. "Quantitative portfolio selection: Using density forecasting to find consistent portfolios," European Journal of Operational Research, Elsevier, vol. 288(3), pages 1053-1067.
    18. Maurizio Daniele & Winfried Pohlmeier & Aygul Zagidullina, 2018. "Sparse Approximate Factor Estimation for High-Dimensional Covariance Matrices," Working Paper Series of the Department of Economics, University of Konstanz 2018-07, Department of Economics, University of Konstanz.
    19. Philipp J. Kremer & Andreea Talmaciu & Sandra Paterlini, 2018. "Risk minimization in multi-factor portfolios: What is the best strategy?," Annals of Operations Research, Springer, vol. 266(1), pages 255-291, July.
    20. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2020. "Company classification using machine learning," Papers 2004.01496, arXiv.org, revised May 2020.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:37:y:2017:i:8:p:1532-1549. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.