IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2411.06389.html
   My bibliography  Save this paper

Optimal Execution with Reinforcement Learning

Author

Listed:
  • Yadh Hafsi
  • Edoardo Vittori

Abstract

This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a limited time frame. Our proposed model leverages input features derived from the current state of the limit order book. To simulate this environment and overcome the limitations associated with relying on historical data, we utilize the multi-agent market simulator ABIDES, which provides a diverse range of depth levels within the limit order book. We present a custom MDP formulation followed by the results of our methodology and benchmark the performance against standard execution strategies. Our findings suggest that the reinforcement learning-based approach demonstrates significant potential.

Suggested Citation

  • Yadh Hafsi & Edoardo Vittori, 2024. "Optimal Execution with Reinforcement Learning," Papers 2411.06389, arXiv.org.
    • Handle: RePEc:arx:papers:2411.06389
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2411.06389
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. A. Chakraborti & I. Muni-Toke & M. Patriarca & F. Abergel, 2011. "Econophysics Review : II. Agent-based models," Post-Print hal-03332946, HAL.
    2. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frederic Abergel, 2011. "Econophysics review: II. Agent-based models," Quantitative Finance, Taylor & Francis Journals, vol. 11(7), pages 1013-1041.
    3. Rama Cont & Sasha Stoikov & Rishi Talreja, 2010. "A Stochastic Model for Order Book Dynamics," Operations Research, INFORMS, vol. 58(3), pages 549-563, June.
    4. Marina Di Giacinto & Claudio Tebaldi & Tai-Ho Wang, 2021. "Optimal order execution under price impact: A hybrid model," Papers 2112.02228, arXiv.org, revised Aug 2022.
    5. V. Alfi & M. Cristelli & L. Pietronero & A. Zaccaria, 2009. "Minimal agent based model for financial markets II," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 67(3), pages 399-417, February.
    6. Tucker Hybinette Balch & Mahmoud Mahfouz & Joshua Lockhart & Maria Hybinette & David Byrd, 2019. "How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?," Papers 1906.12010, arXiv.org.
    7. Eric Smith & J Doyne Farmer & Laszlo Gillemot & Supriya Krishnamurthy, 2003. "Statistical theory of the continuous double auction," Quantitative Finance, Taylor & Francis Journals, vol. 3(6), pages 481-514.
    8. Etienne Chevalier & Yadh Hafsi & Vathana Ly Vath, 2023. "Uncovering Market Disorder and Liquidity Trends Detection," Papers 2310.09273, arXiv.org.
    9. Garman, Mark B., 1976. "Market microstructure," Journal of Financial Economics, Elsevier, vol. 3(3), pages 257-275, June.
    10. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frédéric Abergel, 2011. "Econophysics review: II. Agent-based models," Post-Print hal-00621059, HAL.
    11. Bertsimas, Dimitris & Lo, Andrew W., 1998. "Optimal control of execution costs," Journal of Financial Markets, Elsevier, vol. 1(1), pages 1-50, April.
    12. Tommaso Mariotti & Fabrizio Lillo & Giacomo Toscano, 2023. "From zero-intelligence to queue-reactive: limit-order-book modeling for high-frequency volatility estimation and optimal execution," Quantitative Finance, Taylor & Francis Journals, vol. 23(3), pages 367-388, March.
    13. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ban Zheng & François Roueff & Frédéric Abergel, 2014. "Ergodicity and scaling limit of a constrained multivariate Hawkes process," Post-Print hal-00777941, HAL.
    2. repec:hal:wpaper:hal-00777941 is not listed on IDEAS
    3. Julius Bonart & Martin D. Gould, 2017. "Latency and liquidity provision in a limit order book," Quantitative Finance, Taylor & Francis Journals, vol. 17(10), pages 1601-1616, October.
    4. Ban Zheng & Franc{c}ois Roueff & Fr'ed'eric Abergel, 2013. "Ergodicity and scaling limit of a constrained multivariate Hawkes process," Papers 1301.5007, arXiv.org, revised Feb 2014.
    5. Martin Šmíd, 2016. "Estimation of zero-intelligence models by L1 data," Quantitative Finance, Taylor & Francis Journals, vol. 16(9), pages 1423-1444, September.
    6. Hong Guo & Jianwu Lin & Fanlin Huang, 2023. "Market Making with Deep Reinforcement Learning from Limit Order Books," Papers 2305.15821, arXiv.org.
    7. Alexander Lykov & Stepan Muzychka & Kirill Vaninsky, 2016. "Investor'S Sentiment In Multi-Agent Model Of The Continuous Double Auction," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 19(06), pages 1-29, September.
    8. A. Sadoghi & J. Vecer, 2015. "Optimum Liquidation Problem Associated with the Poisson Cluster Process," Papers 1507.06514, arXiv.org, revised Dec 2015.
    9. Aleksejus Kononovicius & Julius Ruseckas, 2018. "Order book model with herd behavior exhibiting long-range memory," Papers 1809.02772, arXiv.org, revised Apr 2019.
    10. Konark Jain & Nick Firoozye & Jonathan Kochems & Philip Treleaven, 2024. "Limit Order Book Simulations: A Review," Papers 2402.17359, arXiv.org, revised Mar 2024.
    11. Kononovicius, Aleksejus & Ruseckas, Julius, 2019. "Order book model with herd behavior exhibiting long-range memory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 525(C), pages 171-191.
    12. Christoph J. Borner & Ingo Hoffmann & John H. Stiebel, 2024. "A closer look at the chemical potential of an ideal agent system," Papers 2401.09233, arXiv.org.
    13. Kiran Sharma & Subhradeep Das & Anirban Chakraborti, 2017. "Global Income Inequality and Savings: A Data Science Perspective," Papers 1801.00253, arXiv.org, revised Aug 2018.
    14. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of a point-process market-model with a matching engine," Papers 2105.02211, arXiv.org, revised Aug 2021.
    15. A. Lykov & S. Muzychka & K. Vaninsky, 2012. "Investor's sentiment in multi-agent model of the continuous double auction," Papers 1208.3083, arXiv.org, revised Feb 2016.
    16. A. O. Glekin & A. Lykov & K. L. Vaninsky, 2014. "On Simulation of Various Effects in Consolidated Order Book," Papers 1402.4150, arXiv.org.
    17. Fei Cao & Sebastien Motsch, 2021. "Derivation of wealth distributions from biased exchange of money," Papers 2105.07341, arXiv.org.
    18. Pierre Gosselin & Aïleen Lotz & Marc Wambst, 2020. "A path integral approach to business cycle models with large number of agents," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 15(4), pages 899-942, October.
    19. Dias, Thiago & Gonçalves, Sebastián, 2024. "Effectiveness of wealth-based vs exchange-based tax systems in reducing inequality," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 641(C).
    20. Anatoliy Swishchuk & Nelson Vadori, 2016. "A Semi-Markovian Modeling of Limit Order Markets," Papers 1601.01710, arXiv.org.
    21. Pierre Gosselin & Aïleen Lotz & Marc Wambst, 2019. "A Statistical Field Approach to Capital Accumulation," Working Papers hal-02280634, HAL.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2411.06389. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.