IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2105.09264.html
   My bibliography  Save this paper

Robo-Advising: Enhancing Investment with Inverse Optimization and Deep Reinforcement Learning

Author

Listed:
  • Haoran Wang
  • Shi Yu

Abstract

Machine Learning (ML) has been embraced as a powerful tool by the financial industry, with notable applications spreading in various domains including investment management. In this work, we propose a full-cycle data-driven investment robo-advising framework, consisting of two ML agents. The first agent, an inverse portfolio optimization agent, infers an investor's risk preference and expected return directly from historical allocation data using online inverse optimization. The second agent, a deep reinforcement learning (RL) agent, aggregates the inferred sequence of expected returns to formulate a new multi-period mean-variance portfolio optimization problem that can be solved using deep RL approaches. The proposed investment pipeline is applied on real market data from April 1, 2016 to February 1, 2021 and has shown to consistently outperform the S&P 500 benchmark portfolio that represents the aggregate market optimal allocation. The outperformance may be attributed to the the multi-period planning (versus single-period planning) and the data-driven RL approach (versus classical estimation approach).

Suggested Citation

  • Haoran Wang & Shi Yu, 2021. "Robo-Advising: Enhancing Investment with Inverse Optimization and Deep Reinforcement Learning," Papers 2105.09264, arXiv.org.
  • Handle: RePEc:arx:papers:2105.09264
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2105.09264
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dimitris Bertsimas & Vishal Gupta & Ioannis Ch. Paschalidis, 2012. "Inverse Optimization: A New Perspective on the Black-Litterman Model," Operations Research, INFORMS, vol. 60(6), pages 1389-1403, December.
    2. Harry Markowitz, 1952. "Portfolio Selection," Journal of Finance, American Finance Association, vol. 7(1), pages 77-91, March.
    3. Miguel Lobo & Maryam Fazel & Stephen Boyd, 2007. "Portfolio optimization with linear and fixed transaction costs," Annals of Operations Research, Springer, vol. 152(1), pages 341-365, July.
    4. Best, Michael J & Grauer, Robert R, 1991. "On the Sensitivity of Mean-Variance-Efficient Portfolios to Changes in Asset Means: Some Analytical and Computational Results," The Review of Financial Studies, Society for Financial Studies, vol. 4(2), pages 315-342.
    5. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    6. Duan Li & Wan‐Lung Ng, 2000. "Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation," Mathematical Finance, Wiley Blackwell, vol. 10(3), pages 387-406, July.
    7. Luigi Guiso & Monica Paiella, 2008. "Risk Aversion, Wealth, and Background Risk," Journal of the European Economic Association, MIT Press, vol. 6(6), pages 1109-1150, December.
    8. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    9. William F. Sharpe, 1964. "Capital Asset Prices: A Theory Of Market Equilibrium Under Conditions Of Risk," Journal of Finance, American Finance Association, vol. 19(3), pages 425-442, September.
    10. Robert B. Barsky & F. Thomas Juster & Miles S. Kimball & Matthew D. Shapiro, 1997. "Preference Parameters and Behavioral Heterogeneity: An Experimental Approach in the Health and Retirement Study," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 537-579.
    11. Alessandro Bucciol & Raffaele Miniaci, 2018. "Financial Risk Propensity, Business Cycles and Perceived Risk Exposure," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 80(1), pages 160-183, February.
    12. Mannor, Shie & Tsitsiklis, John N., 2013. "Algorithmic aspects of mean–variance optimization in Markov decision processes," European Journal of Operational Research, Elsevier, vol. 231(3), pages 645-653.
    13. J. Tobin, 1958. "Liquidity Preference as Behavior Towards Risk," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 25(2), pages 65-86.
    14. Haoran Wang, 2019. "Large scale continuous-time mean-variance portfolio allocation via reinforcement learning," Papers 1907.11718, arXiv.org, revised Aug 2019.
    15. Nesreen Ahmed & Amir Atiya & Neamat El Gayar & Hisham El-Shishiny, 2010. "An Empirical Comparison of Machine Learning Models for Time Series Forecasting," Econometric Reviews, Taylor & Francis Journals, vol. 29(5-6), pages 594-621.
    16. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    17. Alain Cohn & Jan Engelmann & Ernst Fehr & Michel André Maréchal, 2015. "Evidence for Countercyclical Risk Aversion: An Experiment with Financial Professionals," American Economic Review, American Economic Association, vol. 105(2), pages 860-885, February.
    18. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    19. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    20. Merton, Robert C., 1980. "On estimating the expected return on the market : An exploratory investigation," Journal of Financial Economics, Elsevier, vol. 8(4), pages 323-361, December.
    21. Charness, Gary & Gneezy, Uri & Imas, Alex, 2013. "Experimental methods: Eliciting risk preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 87(C), pages 43-51.
    22. D. Goldfarb & G. Iyengar, 2003. "Robust Portfolio Selection Problems," Mathematics of Operations Research, INFORMS, vol. 28(1), pages 1-38, February.
    23. Mikhail Beketov & Kevin Lehmann & Manuel Wittke, 2018. "Robo Advisors: quantitative methods inside the robots," Journal of Asset Management, Palgrave Macmillan, vol. 19(6), pages 363-370, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yu, Shi & Wang, Haoran & Dong, Chaosheng, 2023. "Learning risk preferences from investment portfolios using inverse optimization," Research in International Business and Finance, Elsevier, vol. 64(C).
    2. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    3. Jaydip Sen & Subhasis Dasgupta, 2023. "Portfolio Optimization: A Comparative Study," Papers 2307.05048, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuki Shigeta, 2016. "Optimality of Naive Investment Strategies in Dynamic MeanVariance Optimization Problems with Multiple Priors," Discussion papers e-16-004, Graduate School of Economics , Kyoto University.
    2. Yu, Shi & Wang, Haoran & Dong, Chaosheng, 2023. "Learning risk preferences from investment portfolios using inverse optimization," Research in International Business and Finance, Elsevier, vol. 64(C).
    3. Erindi Allaj, 2020. "The Black–Litterman model and views from a reverse optimization procedure: an out-of-sample performance evaluation," Computational Management Science, Springer, vol. 17(3), pages 465-492, October.
    4. Hsu, Po-Hsuan & Han, Qiheng & Wu, Wensheng & Cao, Zhiguang, 2018. "Asset allocation strategies, data snooping, and the 1 / N rule," Journal of Banking & Finance, Elsevier, vol. 97(C), pages 257-269.
    5. Agostino Capponi & Sveinn Olafsson & Thaleia Zariphopoulou, 2019. "Personalized Robo-Advising: Enhancing Investment through Client Interaction," Papers 1911.01391, arXiv.org, revised Nov 2020.
    6. Behr, Patrick & Guettler, Andre & Truebenbach, Fabian, 2012. "Using industry momentum to improve portfolio performance," Journal of Banking & Finance, Elsevier, vol. 36(5), pages 1414-1423.
    7. Seyoung Park & Eun Ryung Lee & Sungchul Lee & Geonwoo Kim, 2019. "Dantzig Type Optimization Method with Applications to Portfolio Selection," Sustainability, MDPI, vol. 11(11), pages 1-32, June.
    8. Thomas J. Brennan & Andrew W. Lo, 2010. "Impossible Frontiers," Management Science, INFORMS, vol. 56(6), pages 905-923, June.
    9. Karagiannidis, Iordanis & Vozlyublennaia, Nadia, 2016. "Limits to mutual funds' ability to rely on mean/variance optimization," Journal of Empirical Finance, Elsevier, vol. 37(C), pages 282-292.
    10. Thomas Trier Bjerring & Omri Ross & Alex Weissensteiner, 2017. "Feature selection for portfolio optimization," Annals of Operations Research, Springer, vol. 256(1), pages 21-40, September.
    11. Giorgio Costa & Roy Kwon, 2020. "A robust framework for risk parity portfolios," Journal of Asset Management, Palgrave Macmillan, vol. 21(5), pages 447-466, September.
    12. Yan, Cheng & Zhang, Huazhu, 2017. "Mean-variance versus naïve diversification: The role of mispricing," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 48(C), pages 61-81.
    13. Guiso, Luigi & Sodini, Paolo, 2013. "Household Finance: An Emerging Field," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1397-1532, Elsevier.
    14. Eom, Cheoljun & Park, Jong Won, 2018. "A new method for better portfolio investment: A case of the Korean stock market," Pacific-Basin Finance Journal, Elsevier, vol. 49(C), pages 213-231.
    15. Cisil Sarisoy & Bas J.M. Werker, 2024. "Linear Factor Models and the Estimation of Expected Returns," Finance and Economics Discussion Series 2024-014, Board of Governors of the Federal Reserve System (U.S.).
    16. Hetschko, Clemens & Preuss, Malte, 2020. "Income in jeopardy: How losing employment affects the willingness to take risks," Journal of Economic Psychology, Elsevier, vol. 79(C).
    17. John Y. Campbell, 2000. "Asset Pricing at the Millennium," Journal of Finance, American Finance Association, vol. 55(4), pages 1515-1567, August.
    18. Sonntag, Dominik, 2018. "Die Theorie der fairen geometrischen Rendite [The Theory of Fair Geometric Returns]," MPRA Paper 87082, University Library of Munich, Germany.
    19. Giorgio Costa & Roy H. Kwon, 2020. "Generalized risk parity portfolio optimization: an ADMM approach," Journal of Global Optimization, Springer, vol. 78(1), pages 207-238, September.
    20. Branger, Nicole & Lučivjanská, Katarína & Weissensteiner, Alex, 2019. "Optimal granularity for portfolio choice," Journal of Empirical Finance, Elsevier, vol. 50(C), pages 125-146.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2105.09264. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.