IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2311.13326.html
   My bibliography  Save this paper

Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series

Author

Listed:
  • Woosung Koh
  • Insu Choi
  • Yuntae Jang
  • Gimin Kang
  • Woo Chang Kim

Abstract

Curriculum learning and imitation learning have been leveraged extensively in the robotics domain. However, minimal research has been done on leveraging these ideas on control tasks over highly stochastic time-series data. Here, we theoretically and empirically explore these approaches in a representative control task over complex time-series data. We implement the fundamental ideas of curriculum learning via data augmentation, while imitation learning is implemented via policy distillation from an oracle. Our findings reveal that curriculum learning should be considered a novel direction in improving control-task performance over complex time-series. Our ample random-seed out-sample empirics and ablation studies are highly encouraging for curriculum learning for time-series control. These findings are especially encouraging as we tune all overlapping hyperparameters on the baseline -- giving an advantage to the baseline. On the other hand, we find that imitation learning should be used with caution.

Suggested Citation

  • Woosung Koh & Insu Choi & Yuntae Jang & Gimin Kang & Woo Chang Kim, 2023. "Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series," Papers 2311.13326, arXiv.org, revised Jan 2024.
  • Handle: RePEc:arx:papers:2311.13326
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2311.13326
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    2. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    3. Yunan Ye & Hengzhi Pei & Boxin Wang & Pin-Yu Chen & Yada Zhu & Jun Xiao & Bo Li, 2020. "Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States," Papers 2002.05780, arXiv.org.
    4. Thomas M. Cover, 1991. "Universal Portfolios," Mathematical Finance, Wiley Blackwell, vol. 1(1), pages 1-29, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wu, Bo & Li, Lingfei, 2024. "Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market," Journal of Economic Dynamics and Control, Elsevier, vol. 158(C).
    2. Haoren Zhu & Pengfei Zhao & Wilfred Siu Hung NG & Dik Lun Lee, 2024. "Financial Assets Dependency Prediction Utilizing Spatiotemporal Patterns," Papers 2406.11886, arXiv.org.
    3. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    4. Xiangyu Cui & Xun Li & Yun Shi & Si Zhao, 2023. "Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning," Papers 2312.15385, arXiv.org.
    5. Horikawa, Hiroaki & Nakagawa, Kei, 2024. "Relationship between deep hedging and delta hedging: Leveraging a statistical arbitrage strategy," Finance Research Letters, Elsevier, vol. 62(PA).
    6. Yuheng Zheng & Zihan Ding, 2024. "Reinforcement Learning in High-frequency Market Making," Papers 2407.21025, arXiv.org, revised Aug 2024.
    7. Xianhua Peng & Chenyin Gong & Xue Dong He, 2023. "Reinforcement Learning for Financial Index Tracking," Papers 2308.02820, arXiv.org, revised Nov 2024.
    8. Pascal Franc{c}ois & Genevi`eve Gauthier & Fr'ed'eric Godin & Carlos Octavio P'erez Mendoza, 2024. "Is the difference between deep hedging and delta hedging a statistical arbitrage?," Papers 2407.14736, arXiv.org, revised Oct 2024.
    9. Reilly Pickard & Yuri Lawryshyn, 2023. "Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review," Mathematics, MDPI, vol. 11(24), pages 1-19, December.
    10. Foster, Dean P. & Vohra, Rakesh, 1999. "Regret in the On-Line Decision Problem," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 7-35, October.
    11. Jin’an He & Shicheng Yin & Fangping Peng, 2024. "Weak aggregating specialist algorithm for online portfolio selection," Computational Economics, Springer;Society for Computational Economics, vol. 63(6), pages 2405-2434, June.
    12. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay & Jamal Atif, 2020. "AAMDRL: Augmented Asset Management with Deep Reinforcement Learning," Papers 2010.08497, arXiv.org.
    13. Frensi Zejnullahu & Maurice Moser & Joerg Osterrieder, 2022. "Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network," Papers 2206.14267, arXiv.org.
    14. Seung-Hyun Moon & Yong-Hyuk Kim & Byung-Ro Moon, 2019. "Empirical investigation of state-of-the-art mean reversion strategies for equity markets," Papers 1909.04327, arXiv.org.
    15. Man Yiu Tsang & Tony Sit & Hoi Ying Wong, 2022. "Adaptive Robust Online Portfolio Selection," Papers 2206.01064, arXiv.org.
    16. Luo, Yong & Zhu, Bo & Tang, Yong, 2014. "Simulated annealing algorithm for optimal capital growth," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 408(C), pages 10-18.
    17. Fabio Stella & Alfonso Ventura, 2011. "Defensive online portfolio selection," International Journal of Financial Markets and Derivatives, Inderscience Enterprises Ltd, vol. 2(1/2), pages 88-105.
    18. Erhan Bayraktar & Donghan Kim & Abhishek Tilva, 2024. "Quantifying dimensional change in stochastic portfolio theory," Mathematical Finance, Wiley Blackwell, vol. 34(3), pages 977-1021, July.
    19. Ormos, Mihály & Urbán, András & Zoltán, Tamás, 2009. "Logoptimális portfóliók empirikus vizsgálata [Empirical analysis of log-optimal portfolios]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(1), pages 1-18.
    20. Fereydooni, Ali & Barak, Sasan & Asaad Sajadi, Seyed Mehrzad, 2024. "A novel online portfolio selection approach based on pattern matching and ESG factors," Omega, Elsevier, vol. 123(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2311.13326. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.