IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2301.10724.html
   My bibliography  Save this paper

Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning

Author

Listed:
  • Weiguang Han
  • Boyi Zhang
  • Qianqian Xie
  • Min Peng
  • Yanzhao Lai
  • Jimin Huang

Abstract

Pair trading is one of the most effective statistical arbitrage strategies which seeks a neutral profit by hedging a pair of selected assets. Existing methods generally decompose the task into two separate steps: pair selection and trading. However, the decoupling of two closely related subtasks can block information propagation and lead to limited overall performance. For pair selection, ignoring the trading performance results in the wrong assets being selected with irrelevant price movements, while the agent trained for trading can overfit to the selected assets without any historical information of other assets. To address it, in this paper, we propose a paradigm for automatic pair trading as a unified task rather than a two-step pipeline. We design a hierarchical reinforcement learning framework to jointly learn and optimize two subtasks. A high-level policy would select two assets from all possible combinations and a low-level policy would then perform a series of trading actions. Experimental results on real-world stock data demonstrate the effectiveness of our method on pair trading compared with both existing pair selection and trading methods.

Suggested Citation

  • Weiguang Han & Boyi Zhang & Qianqian Xie & Min Peng & Yanzhao Lai & Jimin Huang, 2023. "Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning," Papers 2301.10724, arXiv.org, revised Feb 2023.
    • Handle: RePEc:arx:papers:2301.10724
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2301.10724
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Krauss, Christopher & Do, Xuan Anh & Huck, Nicolas, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," European Journal of Operational Research, Elsevier, vol. 259(2), pages 689-702.
    2. Huafeng (Jason) Chen & Shaojun (Jenny) Chen & Zhuo Chen & Feng Li, 2019. "Empirical Investigation of an Equity Pairs Trading Strategy," Management Science, INFORMS, vol. 65(1), pages 370-389, January.
    3. Binh Do & Robert Faff, 2012. "Are Pairs Trading Profits Robust To Trading Costs?," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 35(2), pages 261-287, June.
    4. Christopher Krauss & Anh Do & Nicolas Huck, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," Post-Print hal-01768895, HAL.
    5. Nicolas Huck & Komivi Afawubo, 2015. "Pairs trading and selection methods: is cointegration superior?," Applied Economics, Taylor & Francis Journals, vol. 47(6), pages 599-613, February.
    6. Perlin, M., 2007. "M of a kind: A Multivariate Approach at Pairs Trading," MPRA Paper 8309, University Library of Munich, Germany.
    7. Evan Gatev & William N. Goetzmann & K. Geert Rouwenhorst, 2006. "Pairs Trading: Performance of a Relative-Value Arbitrage Rule," The Review of Financial Studies, Society for Financial Studies, vol. 19(3), pages 797-827.
    8. Christopher Krauss, 2017. "Statistical Arbitrage Pairs Trading Strategies: Review And Outlook," Journal of Economic Surveys, Wiley Blackwell, vol. 31(2), pages 513-545, April.
    9. Taewook Kim & Ha Young Kim, 2019. "Optimizing the Pairs-Trading Strategy Using Deep Reinforcement Learning with Trading and Stop-Loss Boundaries," Complexity, Hindawi, vol. 2019, pages 1-20, November.
    10. Nicolas Huck & Komivi Afawubo, 2015. "Pairs trading and selection methods: is cointegration superior?," Post-Print hal-01369852, HAL.
    11. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    12. Ian Martin, 2021. "On the Autocorrelation of the Stock Market [X-CAPM: An Extrapolative Capital Asset Pricing Model]," Journal of Financial Econometrics, Oxford University Press, vol. 19(1), pages 39-52.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weiguang Han & Jimin Huang & Qianqian Xie & Boyi Zhang & Yanzhao Lai & Min Peng, 2023. "Mastering Pair Trading with Risk-Aware Recurrent Reinforcement Learning," Papers 2304.00364, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    2. Han, Chulwoo & He, Zhaodong & Toh, Alenson Jun Wei, 2023. "Pairs trading via unsupervised learning," European Journal of Operational Research, Elsevier, vol. 307(2), pages 929-947.
    3. Stübinger, Johannes & Endres, Sylvia, 2017. "Pairs trading with a mean-reverting jump-diffusion model on high-frequency data," FAU Discussion Papers in Economics 10/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    4. Krauss, Christopher, 2015. "Statistical arbitrage pairs trading strategies: Review and outlook," FAU Discussion Papers in Economics 09/2015, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    5. Clegg, Matthew & Krauss, Christopher, 2016. "Pairs trading with partial cointegration," FAU Discussion Papers in Economics 05/2016, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    6. Erdinc Akyildirim & Ahmet Goncu & Alper Hekimoglu & Duc Khuong Nguyen & Ahmet Sensoy, 2023. "Statistical arbitrage: factor investing approach," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 45(4), pages 1295-1331, December.
    7. Danni Chen & Jing Cui & Yan Gao & Leilei Wu, 2017. "Pairs trading in Chinese commodity futures markets: an adaptive cointegration approach," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 57(5), pages 1237-1264, December.
    8. GholamReza Keshavarz Haddad & Hassan Talebi, 2023. "The profitability of pair trading strategy in stock markets: Evidence from Toronto stock exchange," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(1), pages 193-207, January.
    9. Fernando Caneo & Werner Kristjanpoller, 2021. "Improving statistical arbitrage investment strategy: Evidence from Latin American stock markets," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4424-4440, July.
    10. Johannes Stübinger & Sylvia Endres, 2018. "Pairs trading with a mean-reverting jump–diffusion model on high-frequency data," Quantitative Finance, Taylor & Francis Journals, vol. 18(10), pages 1735-1751, October.
    11. Johannes St binger & Jens Bredthauer, 2017. "Statistical Arbitrage Pairs Trading with High-frequency Data," International Journal of Economics and Financial Issues, Econjournals, vol. 7(4), pages 650-662.
    12. Boming Ning & Prakash Chakraborty & Kiseop Lee, 2023. "Optimal Entry and Exit with Signature in Statistical Arbitrage," Papers 2309.16008, arXiv.org, revised Mar 2024.
    13. Marianna Brunetti & Roberta de Luca, 2022. "Sensitivity of profitability in cointegration-based pairs trading," Centro Studi di Banca e Finanza (CEFIN) (Center for Studies in Banking and Finance) 0090, Universita di Modena e Reggio Emilia, Dipartimento di Economia "Marco Biagi".
    14. Alexander Lipton & Marcos Lopez de Prado, 2020. "A closed-form solution for optimal mean-reverting trading strategies," Papers 2003.10502, arXiv.org.
    15. Marianna Brunetti & Roberta De Luca, 2023. "Pre-selection in cointegration-based pairs trading," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(5), pages 1611-1640, December.
    16. Sánchez-Granero, M.A. & Balladares, K.A. & Ramos-Requena, J.P. & Trinidad-Segovia, J.E., 2020. "Testing the efficient market hypothesis in Latin American stock markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 540(C).
    17. Guo, Minjia & Liu, Jianhe & Luo, Ziping & Han, Xiao, 2024. "Deep reinforcement learning for pairs trading: Evidence from China black series futures," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 981-993.
    18. Yen-Wu Ti & Tian-Shyr Dai & Kuan-Lun Wang & Hao-Han Chang & You-Jia Sun, 2024. "Improving Cointegration-Based Pairs Trading Strategy with Asymptotic Analyses and Convergence Rate Filters," Computational Economics, Springer;Society for Computational Economics, vol. 64(5), pages 2717-2745, November.
    19. Matthew Clegg & Christopher Krauss, 2018. "Pairs trading with partial cointegration," Quantitative Finance, Taylor & Francis Journals, vol. 18(1), pages 121-138, January.
    20. Ariel Neufeld & Julian Sester & Daiying Yin, 2022. "Detecting data-driven robust statistical arbitrage strategies with deep neural networks," Papers 2203.03179, arXiv.org, revised Feb 2024.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2301.10724. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.