IDEAS home Printed from https://ideas.repec.org/a/eee/jfinec/v155y2024ics0304405x24000382.html
   My bibliography  Save this article

Missing values handling for machine learning portfolios

Author

Listed:
  • Chen, Andrew Y.
  • McCoy, Jack

Abstract

We characterize the structure and origins of missingness for 159 cross-sectional return predictors and study missing value handling for portfolios constructed using machine learning. Simply imputing with cross-sectional means performs well compared to rigorous expectation-maximization methods. This stems from three facts about predictor data: (1) missingness occurs in large blocks organized by time, (2) cross-sectional correlations are small, and (3) missingness tends to occur in blocks organized by the underlying data source. As a result, observed data provide little information about missing data. Sophisticated imputations introduce estimation noise that can lead to underperformance if machine learning is not carefully applied.

Suggested Citation

  • Chen, Andrew Y. & McCoy, Jack, 2024. "Missing values handling for machine learning portfolios," Journal of Financial Economics, Elsevier, vol. 155(C).
    • Handle: RePEc:eee:jfinec:v:155:y:2024:i:c:s0304405x24000382
    DOI: 10.1016/j.jfineco.2024.103815
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304405X24000382
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jfineco.2024.103815?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lewellen, Jonathan, 2015. "The Cross-section of Expected Stock Returns," Critical Finance Review, now publishers, vol. 4(1), pages 1-44, June.
    2. Bryan Kelly & Semyon Malamud & Lasse Heje Pedersen, 2023. "Principal Portfolios," Journal of Finance, American Finance Association, vol. 78(1), pages 347-387, February.
    3. Eugene F. Fama & Kenneth R. French, 2008. "Dissecting Anomalies," Journal of Finance, American Finance Association, vol. 63(4), pages 1653-1678, August.
    4. Michael E. Tipping & Christopher M. Bishop, 1999. "Probabilistic Principal Component Analysis," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 611-622.
    5. Joachim Freyberger & Andreas Neuhierl & Michael Weber & Andrew KarolyiEditor, 2020. "Dissecting Characteristics Nonparametrically," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2326-2377.
    6. Dashan Huang & Fuwei Jiang & Kunpeng Li & Guoshi Tong & Guofu Zhou, 2022. "Scaled PCA: A New Approach to Dimension Reduction," Management Science, INFORMS, vol. 68(3), pages 1678-1695, March.
    7. Banz, Rolf W., 1981. "The relationship between return and market value of common stocks," Journal of Financial Economics, Elsevier, vol. 9(1), pages 3-18, March.
    8. Andrew Y. Chen & Tom Zimmermann, 2022. "Open Source Cross-Sectional Asset Pricing," Critical Finance Review, now publishers, vol. 11(2), pages 207-264, May.
    9. Kozak, Serhiy & Nagel, Stefan & Santosh, Shrihari, 2020. "Shrinking the cross-section," Journal of Financial Economics, Elsevier, vol. 135(2), pages 271-292.
    10. R. David Mclean & Jeffrey Pontiff, 2016. "Does Academic Research Destroy Stock Return Predictability?," Journal of Finance, American Finance Association, vol. 71(1), pages 5-32, February.
    11. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    12. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    13. Asquith, Paul & Pathak, Parag A. & Ritter, Jay R., 2005. "Short interest, institutional ownership, and stock returns," Journal of Financial Economics, Elsevier, vol. 78(2), pages 243-276, November.
    14. Andrew Y Chen & Tom Zimmermann & Jeffrey Pontiff, 2020. "Publication Bias and the Cross-Section of Stock Returns," The Review of Asset Pricing Studies, Society for Financial Studies, vol. 10(2), pages 249-289.
    15. Xiong, Ruoxuan & Pelger, Markus, 2023. "Large dimensional latent factor modeling with missing observations and applications to causal inference," Journal of Econometrics, Elsevier, vol. 233(1), pages 271-301.
    16. Victor DeMiguel & Alberto Martín-Utrera & Francisco J Nogales & Raman Uppal & Andrew KarolyiEditor, 2020. "A Transaction-Cost Perspective on the Multitude of Firm Characteristics," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2180-2222.
    17. Fama, Eugene F & French, Kenneth R, 1992. "The Cross-Section of Expected Stock Returns," Journal of Finance, American Finance Association, vol. 47(2), pages 427-465, June.
    18. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    19. Martin Lettau & Markus Pelger & Stijn Van Nieuwerburgh, 2020. "Factors That Fit the Time Series and Cross-Section of Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2274-2325.
    20. Jeremiah Green & John R. M. Hand & X. Frank Zhang, 2017. "The Characteristics that Provide Independent Information about Average U.S. Monthly Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 30(12), pages 4389-4436.
    21. Theis Ingerslev Jensen & Bryan T. Kelly & Semyon Malamud & Lasse Heje Pedersen, 2022. "Machine Learning and the Implementable Efficient Frontier," Swiss Finance Institute Research Paper Series 22-63, Swiss Finance Institute.
    22. Martin Lettau & Markus Pelger, 2020. "Factors That Fit the Time Series and Cross-Section of Stock Returns," Review of Finance, European Finance Association, vol. 33(5), pages 2274-2325.
    23. King, Gary & Honaker, James & Joseph, Anne & Scheve, Kenneth, 2001. "Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation," American Political Science Review, Cambridge University Press, vol. 95(1), pages 49-69, March.
    24. Titman, Sheridan & Wei, K. C. John & Xie, Feixue, 2004. "Capital Investments and Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 39(4), pages 677-700, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clarke, Charles, 2022. "The level, slope, and curve factor model for stocks," Journal of Financial Economics, Elsevier, vol. 143(1), pages 159-187.
    2. Andrew Y. Chen & Jack McCoy, 2022. "Missing Values Handling for Machine Learning Portfolios," Papers 2207.13071, arXiv.org, revised Jan 2024.
    3. Baba-Yara, Fahiz & Boons, Martijn & Tamoni, Andrea, 2024. "Persistent and transitory components of firm characteristics: Implications for asset pricing," Journal of Financial Economics, Elsevier, vol. 154(C).
    4. Christian Fieberg & Daniel Metko & Thorsten Poddig & Thomas Loy, 2023. "Machine learning techniques for cross-sectional equity returns’ prediction," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 45(1), pages 289-323, March.
    5. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    6. Söhnke M. Bartram & Harald Lohre & Peter F. Pope & Ananthalakshmi Ranganathan, 2021. "Navigating the factor zoo around the world: an institutional investor perspective," Journal of Business Economics, Springer, vol. 91(5), pages 655-703, July.
    7. Yan, Jingda & Yu, Jialin, 2023. "Cross-stock momentum and factor momentum," Journal of Financial Economics, Elsevier, vol. 150(2).
    8. Cakici, Nusret & Fieberg, Christian & Metko, Daniel & Zaremba, Adam, 2023. "Machine learning goes global: Cross-sectional return predictability in international stock markets," Journal of Economic Dynamics and Control, Elsevier, vol. 155(C).
    9. Doron Avramov & Guy Kaplanski & Avanidhar Subrahmanyam, 2022. "Postfundamentals Price Drift in Capital Markets: A Regression Regularization Perspective," Management Science, INFORMS, vol. 68(10), pages 7658-7681, October.
    10. Tobek, Ondrej & Hronec, Martin, 2021. "Does it pay to follow anomalies research? Machine learning approach with international evidence," Journal of Financial Markets, Elsevier, vol. 56(C).
    11. Bryzgalova, Svetlana & Huang, Jiantao & Julliard, Christian, 2023. "Bayesian solutions for the factor zoo: we just ran two quadrillion models," LSE Research Online Documents on Economics 126151, London School of Economics and Political Science, LSE Library.
    12. Lioui, Abraham & Tarelli, Andrea, 2022. "Chasing the ESG factor," Journal of Banking & Finance, Elsevier, vol. 139(C).
    13. Sun, Chuanping, 2024. "Factor correlation and the cross section of asset returns: A correlation-robust machine learning approach," Journal of Empirical Finance, Elsevier, vol. 77(C).
    14. Bagnara, Matteo, 2024. "The economic value of cross-predictability: A performance-based measure," SAFE Working Paper Series 424, Leibniz Institute for Financial Research SAFE.
    15. Joachim Freyberger & Andreas Neuhierl & Michael Weber, 2020. "Dissecting Characteristics Nonparametrically," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2326-2377.
    16. De Nard, Gianluca & Zhao, Zhao, 2023. "Using, taming or avoiding the factor zoo? A double-shrinkage estimator for covariance matrices," Journal of Empirical Finance, Elsevier, vol. 72(C), pages 23-35.
    17. Vitor Azevedo & Georg Sebastian Kaiser & Sebastian Mueller, 2023. "Stock market anomalies and machine learning across the globe," Journal of Asset Management, Palgrave Macmillan, vol. 24(5), pages 419-441, September.
    18. Beckmeyer, Heiner & Wiedemann, Timo, 2022. "Recovering Missing Firm Characteristics with Attention-Based Machine Learning," VfS Annual Conference 2022 (Basel): Big Data in Economics 264135, Verein für Socialpolitik / German Economic Association.
    19. Caldeira, João F. & Santos, André A.P. & Torrent, Hudson S., 2023. "Semiparametric portfolios: Improving portfolio performance by exploiting non-linearities in firm characteristics," Economic Modelling, Elsevier, vol. 122(C).
    20. Wolfgang Drobetz & Tizian Otto, 2021. "Empirical asset pricing via machine learning: evidence from the European stock market," Journal of Asset Management, Palgrave Macmillan, vol. 22(7), pages 507-538, December.

    More about this item

    Keywords

    Stock market predictability; Stock market anomalies; Missing values; Machine learning;
    All these keywords.

    JEL classification:

    • G0 - Financial Economics - - General
    • G1 - Financial Economics - - General Financial Markets

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jfinec:v:155:y:2024:i:c:s0304405x24000382. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/505576 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.