IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2405.16449.html
   My bibliography  Save this paper

Reinforcement Learning for Jump-Diffusions, with Financial Applications

Author

Listed:
  • Xuefeng Gao
  • Lingfei Li
  • Xun Yu Zhou

Abstract

We study continuous-time reinforcement learning (RL) for stochastic control in which system dynamics are governed by jump-diffusion processes. We formulate an entropy-regularized exploratory control problem with stochastic policies to capture the exploration--exploitation balance essential for RL. Unlike the pure diffusion case initially studied by Wang et al. (2020), the derivation of the exploratory dynamics under jump-diffusions calls for a careful formulation of the jump part. Through a theoretical analysis, we find that one can simply use the same policy evaluation and $q$-learning algorithms in Jia and Zhou (2022a, 2023), originally developed for controlled diffusions, without needing to check a priori whether the underlying data come from a pure diffusion or a jump-diffusion. However, we show that the presence of jumps ought to affect parameterizations of actors and critics in general. We investigate as an application the mean--variance portfolio selection problem with stock price modelled as a jump-diffusion, and show that both RL algorithms and parameterizations are invariant with respect to jumps. Finally, we present a detailed study on applying the general theory to option hedging.

Suggested Citation

  • Xuefeng Gao & Lingfei Li & Xun Yu Zhou, 2024. "Reinforcement Learning for Jump-Diffusions, with Financial Applications," Papers 2405.16449, arXiv.org, revised Aug 2024.
  • Handle: RePEc:arx:papers:2405.16449
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2405.16449
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Das, Sanjiv R., 2002. "The surprise element: jumps in interest rates," Journal of Econometrics, Elsevier, vol. 106(1), pages 27-65, January.
    2. S. G. Kou, 2002. "A Jump-Diffusion Model for Option Pricing," Management Science, INFORMS, vol. 48(8), pages 1086-1101, August.
    3. Bates, David S, 1991. "The Crash of '87: Was It Expected? The Evidence from Options Markets," Journal of Finance, American Finance Association, vol. 46(3), pages 1009-1044, July.
    4. Torben G. Andersen & Luca Benzoni & Jesper Lund, 2002. "An Empirical Investigation of Continuous‐Time Equity Return Models," Journal of Finance, American Finance Association, vol. 57(3), pages 1239-1284, June.
    5. Ning Cai & S. G. Kou, 2011. "Option Pricing Under a Mixed-Exponential Jump Diffusion Model," Management Science, INFORMS, vol. 57(11), pages 2067-2081, November.
    6. Haoran Wang & Xun Yu Zhou, 2020. "Continuous‐time mean–variance portfolio selection: A reinforcement learning framework," Mathematical Finance, Wiley Blackwell, vol. 30(4), pages 1273-1308, October.
    7. Wang, Bin & Zheng, Xu, 2022. "Testing for the presence of jump components in jump diffusion models," Journal of Econometrics, Elsevier, vol. 230(2), pages 483-509.
    8. Merton, Robert C., 1976. "Option pricing when underlying stock returns are discontinuous," Journal of Financial Economics, Elsevier, vol. 3(1-2), pages 125-144.
    9. Aït-Sahalia, Yacine & Jacod, Jean & Li, Jia, 2012. "Testing for jumps in noisy high frequency data," Journal of Econometrics, Elsevier, vol. 168(2), pages 207-222.
    10. Yanwei Jia & Xun Yu Zhou, 2022. "q-Learning in Continuous Time," Papers 2207.00713, arXiv.org, revised Apr 2023.
    11. Bates, David S, 1996. "Jumps and Stochastic Volatility: Exchange Rate Processes Implicit in Deutsche Mark Options," The Review of Financial Studies, Society for Financial Studies, vol. 9(1), pages 69-107.
    12. Wu, Bo & Li, Lingfei, 2024. "Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market," Journal of Economic Dynamics and Control, Elsevier, vol. 158(C).
    13. Bedartha Goswami & Niklas Boers & Aljoscha Rheinwalt & Norbert Marwan & Jobst Heitzig & Sebastian F. M. Breitenbach & Jürgen Kurths, 2018. "Abrupt transitions in time series with uncertainties," Nature Communications, Nature, vol. 9(1), pages 1-10, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frances Shaw & Finbarr Murphy & Fergal G. O’Brien, 2016. "Jumps in Euribor and the effect of ECB monetary policy announcements," Environment Systems and Decisions, Springer, vol. 36(2), pages 142-157, June.
    2. Dario Alitab & Giacomo Bormetti & Fulvio Corsi & Adam A. Majewski, 2019. "A realized volatility approach to option pricing with continuous and jump variance components," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 42(2), pages 639-664, December.
    3. Li, Chenxu & Chen, Dachuan, 2016. "Estimating jump–diffusions using closed-form likelihood expansions," Journal of Econometrics, Elsevier, vol. 195(1), pages 51-70.
    4. Kozarski, R., 2013. "Pricing and hedging in the VIX derivative market," Other publications TiSEM 221fefe0-241e-4914-b6bd-c, Tilburg University, School of Economics and Management.
    5. Jin-Yu Zhang & Wen-Bo Wu & Yong Li & Zhu-Sheng Lou, 2021. "Pricing Exotic Option Under Jump-Diffusion Models by the Quadrature Method," Computational Economics, Springer;Society for Computational Economics, vol. 58(3), pages 867-884, October.
    6. Michael C. Fu & Bingqing Li & Guozhen Li & Rongwen Wu, 2017. "Option Pricing for a Jump-Diffusion Model with General Discrete Jump-Size Distributions," Management Science, INFORMS, vol. 63(11), pages 3961-3977, November.
    7. Michael C. Fu & Bingqing Li & Rongwen Wu & Tianqi Zhang, 2020. "Option Pricing Under a Discrete-Time Markov Switching Stochastic Volatility with Co-Jump Model," Papers 2006.15054, arXiv.org.
    8. Carr, Peter & Wu, Liuren, 2004. "Time-changed Levy processes and option pricing," Journal of Financial Economics, Elsevier, vol. 71(1), pages 113-141, January.
    9. Liu, Yi & Liu, Huifang & Zhang, Lei, 2019. "Modeling and forecasting return jumps using realized variation measures," Economic Modelling, Elsevier, vol. 76(C), pages 63-80.
    10. Moreno, Manuel & Serrano, Pedro & Stute, Winfried, 2011. "Statistical properties and economic implications of jump-diffusion processes with shot-noise effects," European Journal of Operational Research, Elsevier, vol. 214(3), pages 656-664, November.
    11. Kim, Kyong-Hui & Yun, Sim & Kim, Nam-Ung & Ri, Ju-Hyuang, 2019. "Pricing formula for European currency option and exchange option in a generalized jump mixed fractional Brownian motion with time-varying coefficients," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 522(C), pages 215-231.
    12. Chan, Tat Lung (Ron), 2019. "Efficient computation of european option prices and their sensitivities with the complex fourier series method," The North American Journal of Economics and Finance, Elsevier, vol. 50(C).
    13. Bruti-Liberati Nicola & Nikitopoulos-Sklibosios Christina & Platen Eckhard, 2006. "First Order Strong Approximations of Jump Diffusions," Monte Carlo Methods and Applications, De Gruyter, vol. 12(3), pages 191-209, October.
    14. Carr, Peter & Wu, Liuren, 2007. "Stochastic skew in currency options," Journal of Financial Economics, Elsevier, vol. 86(1), pages 213-247, October.
    15. Torben G. Andersen & Nicola Fusari & Viktor Todorov, 2017. "Short-Term Market Risks Implied by Weekly Options," Journal of Finance, American Finance Association, vol. 72(3), pages 1335-1386, June.
    16. Bollerslev, Tim & Todorov, Viktor, 2014. "Time-varying jump tails," Journal of Econometrics, Elsevier, vol. 183(2), pages 168-180.
    17. Paweł Kliber, 2019. "Continuous and jump changes in prices processes in the selected stock markets," Collegium of Economic Analysis Annals, Warsaw School of Economics, Collegium of Economic Analysis, issue 54, pages 333-344.
    18. Timothy Sharp & Steven Li & David Allen, 2010. "Empirical performance of affine option pricing models: evidence from the Australian index options market," Applied Financial Economics, Taylor & Francis Journals, vol. 20(6), pages 501-514.
    19. Guan, Lim Kian & Ting, Christopher & Warachka, Mitch, 2005. "The implied jump risk of LIBOR rates," Journal of Banking & Finance, Elsevier, vol. 29(10), pages 2503-2522, October.
    20. Peter Carr & Liuren Wu, 2014. "Static Hedging of Standard Options," Journal of Financial Econometrics, Oxford University Press, vol. 12(1), pages 3-46.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2405.16449. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.