IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1909.03278.html
   My bibliography  Save this paper

Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning

Author

Listed:
  • Wonsup Shin
  • Seok-Jun Bu
  • Sung-Bae Cho

Abstract

The autonomous trading agent is one of the most actively studied areas of artificial intelligence to solve the capital market portfolio management problem. The two primary goals of the portfolio management problem are maximizing profit and restrainting risk. However, most approaches to this problem solely take account of maximizing returns. Therefore, this paper proposes a deep reinforcement learning based trading agent that can manage the portfolio considering not only profit maximization but also risk restraint. We also propose a new target policy to allow the trading agent to learn to prefer low-risk actions. The new target policy can be reflected in the update by adjusting the greediness for the optimal action through the hyper parameter. The proposed trading agent verifies the performance through the data of the cryptocurrency market. The Cryptocurrency market is the best test-ground for testing our trading agents because of the huge amount of data accumulated every minute and the market volatility is extremely large. As a experimental result, during the test period, our agents achieved a return of 1800% and provided the least risky investment strategy among the existing methods. And, another experiment shows that the agent can maintain robust generalized performance even if market volatility is large or training period is short.

Suggested Citation

  • Wonsup Shin & Seok-Jun Bu & Sung-Bae Cho, 2019. "Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning," Papers 1909.03278, arXiv.org.
    • Handle: RePEc:arx:papers:1909.03278
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1909.03278
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jacob Boudoukh & Matthew Richardson & Robert F. Whitelaw, 1997. "Nonlinearities in the Relation Between the Equity Risk Premium and the Term Structure," Management Science, INFORMS, vol. 43(3), pages 371-385, March.
    2. Whitelaw, Robert F, 1994. "Time Variations and Covariations in the Expectation and Volatility of Stock Market Returns," Journal of Finance, American Finance Association, vol. 49(2), pages 515-541, June.
    3. Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. "On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
    4. David P. Helmbold & Robert E. Schapire & Yoram Singer & Manfred K. Warmuth, 1998. "On‐Line Portfolio Selection Using Multiplicative Updates," Mathematical Finance, Wiley Blackwell, vol. 8(4), pages 325-347, October.
    5. Zhengyao Jiang & Dixing Xu & Jinjun Liang, 2017. "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," Papers 1706.10059, arXiv.org, revised Jul 2017.
    6. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert F. Whitelaw, 1997. "Time-Varying Sharpe Ratios and Market Timing," New York University, Leonard N. Stern School Finance Department Working Paper Seires 98-074, New York University, Leonard N. Stern School of Business-.
    2. Hong, Seok Young & Linton, Oliver, 2020. "Nonparametric estimation of infinite order regression and its application to the risk-return tradeoff," Journal of Econometrics, Elsevier, vol. 219(2), pages 389-424.
    3. Ludvigson, Sydney C. & Ng, Serena, 2007. "The empirical risk-return relation: A factor analysis approach," Journal of Financial Economics, Elsevier, vol. 83(1), pages 171-222, January.
    4. Kiseok Nam & Joshua Krausz & Augustine C. Arize, 2014. "Revisiting the intertemporal risk-return relation: asymmetrical effect of unexpected volatility shocks," Quantitative Finance, Taylor & Francis Journals, vol. 14(12), pages 2193-2203, December.
    5. Osman Kilic & Joseph M. Marks & Kiseok Nam, 2022. "Predictable asset price dynamics, risk-return tradeoff, and investor behavior," Review of Quantitative Finance and Accounting, Springer, vol. 59(2), pages 749-791, August.
    6. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    7. Chelikani, Surya & Marks, Joseph M. & Nam, Kiseok, 2023. "Volatility feedback effect and risk-return tradeoff," The Quarterly Review of Economics and Finance, Elsevier, vol. 92(C), pages 49-65.
    8. Wang, Zijun & Khan, M. Moosa, 2017. "Market states and the risk-return tradeoff," The Quarterly Review of Economics and Finance, Elsevier, vol. 65(C), pages 314-327.
    9. Li, Yuming, 1998. "Expected stock returns, risk premiums and volatilities of economic factors1," Journal of Empirical Finance, Elsevier, vol. 5(2), pages 69-97, June.
    10. Chiang, Thomas C., 2019. "Empirical analysis of intertemporal relations between downside risks and expected returns—Evidence from Asian markets," Research in International Business and Finance, Elsevier, vol. 47(C), pages 264-278.
    11. Dufour, Jean-Marie & García, René, 2008. "Measuring causality between volatility and returns with high-frequency data," UC3M Working papers. Economics we084422, Universidad Carlos III de Madrid. Departamento de Economía.
    12. Ajeet Jain & Sascha Strobl, 2017. "The effect of volatility persistence on excess returns," Review of Financial Economics, John Wiley & Sons, vol. 32(1), pages 58-63, January.
    13. Yuming Li & Ko Wang, 1995. "The Predictability of REIT Returns and Market Segmentatio," Journal of Real Estate Research, American Real Estate Society, vol. 10(4), pages 471-482.
    14. Weidong Tian & Murray Carlson & David A. Chapman & Ron Kaniel & Hong Yan, 2017. "Specification Error, Estimation Risk, and Conditional Portfolio Rules," International Review of Finance, International Review of Finance Ltd., vol. 17(2), pages 263-288, June.
    15. Thomas C. Chiang & Jiandong Li, 2012. "Stock Returns and Risk: Evidence from Quantile," JRFM, MDPI, vol. 5(1), pages 1-39, December.
    16. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep equal risk pricing of financial derivatives with non-translation invariant risk measures," Papers 2107.11340, arXiv.org.
    17. Guo, Hui & Savickas, Robert & Wang, Zijun & Yang, Jian, 2009. "Is the Value Premium a Proxy for Time-Varying Investment Opportunities? Some Time-Series Evidence," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 44(1), pages 133-154, February.
    18. Kim, Eung-Bin & Byun, Suk-Joon, 2021. "Risk, ambiguity, and equity premium: International evidence," International Review of Economics & Finance, Elsevier, vol. 76(C), pages 321-335.
    19. Wang, Wenzhao & Duxbury, Darren, 2021. "Institutional investor sentiment and the mean-variance relationship: Global evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 415-441.
    20. Appiah-Kusi, Joe & Menyah, Kojo, 2003. "Return predictability in African stock markets," Review of Financial Economics, Elsevier, vol. 12(3), pages 247-270.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1909.03278. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.