IDEAS home Printed from https://ideas.repec.org/a/bla/jfinan/v79y2024i1p459-503.html
   My bibliography  Save this article

The Virtue of Complexity in Return Prediction

Author

Listed:
  • BRYAN KELLY
  • SEMYON MALAMUD
  • KANGYING ZHOU

Abstract

Much of the extant literature predicts market returns with “simple” models that use only a few parameters. Contrary to conventional wisdom, we theoretically prove that simple models severely understate return predictability compared to “complex” models in which the number of parameters exceeds the number of observations. We empirically document the virtue of complexity in U.S. equity market return prediction. Our findings establish the rationale for modeling expected returns through machine learning.

Suggested Citation

  • Bryan Kelly & Semyon Malamud & Kangying Zhou, 2024. "The Virtue of Complexity in Return Prediction," Journal of Finance, American Finance Association, vol. 79(1), pages 459-503, February.
    • Handle: RePEc:bla:jfinan:v:79:y:2024:i:1:p:459-503
    DOI: 10.1111/jofi.13298
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/jofi.13298
    Download Restriction: no
    File URL: https://libkey.io/10.1111/jofi.13298?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    2. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    3. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    4. Martin, Ian W.R. & Nagel, Stefan, 2022. "Market efficiency in the age of big data," Journal of Financial Economics, Elsevier, vol. 145(1), pages 154-177.
    5. Joachim Freyberger & Andreas Neuhierl & Michael Weber & Andrew KarolyiEditor, 2020. "Dissecting Characteristics Nonparametrically," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2326-2377.
    6. Ralph S.J. Koijen & Stijn Van Nieuwerburgh, 2011. "Predictability of Returns and Cash Flows," Annual Review of Financial Economics, Annual Reviews, vol. 3(1), pages 467-491, December.
    7. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    8. Jianqing Fan & Jianhua Guo & Shurong Zheng, 2022. "Estimating Number of Factors by Adjusted Eigenvalues Thresholding," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(538), pages 852-861, April.
    9. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    10. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," The Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    11. Wayne E. Ferson & Andrew F. Siegel, 2001. "The Efficient Use of Conditioning Information in Portfolios," Journal of Finance, American Finance Association, vol. 56(3), pages 967-982, June.
    12. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    13. John H. Cochrane, 2011. "Presidential Address: Discount Rates," Journal of Finance, American Finance Association, vol. 66(4), pages 1047-1108, August.
    14. Abhyankar, Abhay & Basu, Devraj & Stremme, Alexander, 2012. "The Optimal Use of Return Predictability: An Empirical Study," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 47(5), pages 973-1001, October.
    15. Cenesizoglu, Tolga & Timmermann, Allan, 2012. "Do return prediction models add economic value?," Journal of Banking & Finance, Elsevier, vol. 36(11), pages 2974-2987.
    16. Silverstein, J. W., 1995. "Strong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices," Journal of Multivariate Analysis, Elsevier, vol. 55(2), pages 331-339, November.
    17. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    18. Fan, Jianqing & Fan, Yingying & Lv, Jinchi, 2008. "High dimensional covariance matrix estimation using a factor model," Journal of Econometrics, Elsevier, vol. 147(1), pages 186-197, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yonghe Lu & Yanrong Yang & Terry Zhang, 2024. "Double Descent in Portfolio Optimization: Dance between Theoretical Sharpe Ratio and Estimation Accuracy," Papers 2411.18830, arXiv.org.
    2. Faria, Gonçalo & Verona, Fabio, 2024. "Enhancing forecast accuracy through frequencydomain combination: Applications to financial and economic indicators," Bank of Finland Research Discussion Papers 14/2024, Bank of Finland.
    3. Faria, Gonçalo & Verona, Fabio, 2024. "Unlocking predictive potential: the frequency-domain approach to equity premium forecasting," Bank of Finland Research Discussion Papers 10/2024, Bank of Finland.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Smith, Simon C., 2021. "International stock return predictability," International Review of Financial Analysis, Elsevier, vol. 78(C).
    3. Mykola Babiak & Jozef Barunik, 2020. "Deep Learning, Predictability, and Optimal Portfolio Returns," CERGE-EI Working Papers wp677, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    4. Yaojie Zhang & Mengxi He & Zhikai Zhang, 2024. "Forecasting stock returns with industry volatility concentration," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(7), pages 2705-2730, November.
    5. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    6. Christian Fieberg & Daniel Metko & Thorsten Poddig & Thomas Loy, 2023. "Machine learning techniques for cross-sectional equity returns’ prediction," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 45(1), pages 289-323, March.
    7. , & Stein, Tobias, 2021. "Equity premium predictability over the business cycle," CEPR Discussion Papers 16357, C.E.P.R. Discussion Papers.
    8. Kuppenheimer, Gregory & Shelly, Stuart & Strauss, Jack, 2023. "Can machine learning identify sector-level financial ratios that predict sector returns?," Finance Research Letters, Elsevier, vol. 57(C).
    9. Cakici, Nusret & Fieberg, Christian & Metko, Daniel & Zaremba, Adam, 2023. "Machine learning goes global: Cross-sectional return predictability in international stock markets," Journal of Economic Dynamics and Control, Elsevier, vol. 155(C).
    10. Lof, Matthijs & Nyberg, Henri, 2024. "Discount rates and cash flows: A local projection approach," Journal of Banking & Finance, Elsevier, vol. 162(C).
    11. Nusret Cakici & Christian Fieberg & Daniel Metko & Adam Zaremba, 2024. "Do Anomalies Really Predict Market Returns? New Data and New Evidence," Review of Finance, European Finance Association, vol. 28(1), pages 1-44.
    12. Huang, Dashan & Li, Jiangyuan & Wang, Liyao, 2021. "Are disagreements agreeable? Evidence from information aggregation," Journal of Financial Economics, Elsevier, vol. 141(1), pages 83-101.
    13. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    14. Dichtl, Hubert & Drobetz, Wolfgang & Otto, Tizian, 2023. "Forecasting Stock Market Crashes via Machine Learning," Journal of Financial Stability, Elsevier, vol. 65(C).
    15. Niu, Zibo & Wang, Chenlu & Zhang, Hongwei, 2023. "Forecasting stock market volatility with various geopolitical risks categories: New evidence from machine learning models," International Review of Financial Analysis, Elsevier, vol. 89(C).
    16. Pan, Zhiyuan & Zhong, Hao & Wang, Yudong & Huang, Juan, 2024. "Forecasting oil futures returns with news," Energy Economics, Elsevier, vol. 134(C).
    17. Lawrenz, Jochen & Zorn, Josef, 2017. "Predicting international stock returns with conditional price-to-fundamental ratios," Journal of Empirical Finance, Elsevier, vol. 43(C), pages 159-184.
    18. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    19. Giovannelli, Alessandro & Massacci, Daniele & Soccorsi, Stefano, 2021. "Forecasting stock returns with large dimensional factor models," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 252-269.
    20. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2023. "Gold risk premium estimation with machine learning methods," Journal of Commodity Markets, Elsevier, vol. 31(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jfinan:v:79:y:2024:i:1:p:459-503. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/afaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.