IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2211.07622.html
   My bibliography  Save this paper

Exploratory Control with Tsallis Entropy for Latent Factor Models

Author

Listed:
  • Ryan Donnelly
  • Sebastian Jaimungal

Abstract

We study optimal control in models with latent factors where the agent controls the distribution over actions, rather than actions themselves, in both discrete and continuous time. To encourage exploration of the state space, we reward exploration with Tsallis Entropy and derive the optimal distribution over states - which we prove is $q$-Gaussian distributed with location characterized through the solution of an FBS$\Delta$E and FBSDE in discrete and continuous time, respectively. We discuss the relation between the solutions of the optimal exploration problems and the standard dynamic optimal control solution. Finally, we develop the optimal policy in a model-agnostic setting along the lines of soft $Q$-learning. The approach may be applied in, e.g., developing more robust statistical arbitrage trading strategies.

Suggested Citation

  • Ryan Donnelly & Sebastian Jaimungal, 2022. "Exploratory Control with Tsallis Entropy for Latent Factor Models," Papers 2211.07622, arXiv.org, revised Jan 2024.
    • Handle: RePEc:arx:papers:2211.07622
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2211.07622
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. N. El Karoui & S. Peng & M. C. Quenez, 1997. "Backward Stochastic Differential Equations in Finance," Mathematical Finance, Wiley Blackwell, vol. 7(1), pages 1-71, January.
    2. Sebastian Jaimungal, 2022. "Reinforcement learning and stochastic optimisation," Finance and Stochastics, Springer, vol. 26(1), pages 103-129, January.
    3. Gobet, Emmanuel & Labart, Céline, 2007. "Error expansion for the discretization of backward stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 117(7), pages 803-829, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chol-Kyu Pak & Mun-Chol Kim & Chang-Ho Rim, 2018. "Adapted $\theta$-Scheme and Its Error Estimates for Backward Stochastic Differential Equations," Papers 1808.02173, arXiv.org.
    2. Ioannis Exarchos & Evangelos Theodorou & Panagiotis Tsiotras, 2019. "Stochastic Differential Games: A Sampling Approach via FBSDEs," Dynamic Games and Applications, Springer, vol. 9(2), pages 486-505, June.
    3. Wei Zhang & Hui Min, 2021. "Weak Convergence Analysis and Improved Error Estimates for Decoupled Forward-Backward Stochastic Differential Equations," Mathematics, MDPI, vol. 9(8), pages 1-15, April.
    4. Guangbao Guo, 2018. "Finite Difference Methods for the BSDEs in Finance," IJFS, MDPI, vol. 6(1), pages 1-15, March.
    5. Dirk Becherer & Plamen Turkedjiev, 2014. "Multilevel approximation of backward stochastic differential equations," Papers 1412.3140, arXiv.org.
    6. Chol-Kyu Pak & Mun-Chol Kim & O Hun, 2018. "A generalized scheme for BSDEs based on derivative approximation and its error estimates," Papers 1808.02478, arXiv.org.
    7. Gnameho Kossi & Stadje Mitja & Pelsser Antoon, 2024. "A gradient method for high-dimensional BSDEs," Monte Carlo Methods and Applications, De Gruyter, vol. 30(2), pages 183-203.
    8. Pagès, Gilles & Sagna, Abass, 2018. "Improved error bounds for quantization based numerical schemes for BSDE and nonlinear filtering," Stochastic Processes and their Applications, Elsevier, vol. 128(3), pages 847-883.
    9. Crisan, D. & Manolarakis, K. & Touzi, N., 2010. "On the Monte Carlo simulation of BSDEs: An improvement on the Malliavin weights," Stochastic Processes and their Applications, Elsevier, vol. 120(7), pages 1133-1158, July.
    10. Fujii, Masaaki & Takahashi, Akihiko, 2019. "Solving backward stochastic differential equations with quadratic-growth drivers by connecting the short-term expansions," Stochastic Processes and their Applications, Elsevier, vol. 129(5), pages 1492-1532.
    11. Bouchard Bruno & Tan Xiaolu & Warin Xavier & Zou Yiyi, 2017. "Numerical approximation of BSDEs using local polynomial drivers and branching processes," Monte Carlo Methods and Applications, De Gruyter, vol. 23(4), pages 241-263, December.
    12. Fan, ShengJun, 2016. "Existence of solutions to one-dimensional BSDEs with semi-linear growth and general growth generators," Statistics & Probability Letters, Elsevier, vol. 109(C), pages 7-15.
    13. Masaaki Fujii & Akihiko Takahashi, 2015. "Perturbative Expansion Technique for Non-linear FBSDEs with Interacting Particle Method," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 22(3), pages 283-304, September.
    14. Kupper, Michael & Luo, Peng & Tangpi, Ludovic, 2019. "Multidimensional Markovian FBSDEs with super-quadratic growth," Stochastic Processes and their Applications, Elsevier, vol. 129(3), pages 902-923.
    15. Mingyu Xu, 2007. "Reflected Backward SDEs with Two Barriers Under Monotonicity and General Increasing Conditions," Journal of Theoretical Probability, Springer, vol. 20(4), pages 1005-1039, December.
    16. Gobet, Emmanuel & Labart, Céline, 2007. "Error expansion for the discretization of backward stochastic differential equations," Stochastic Processes and their Applications, Elsevier, vol. 117(7), pages 803-829, July.
    17. Han, Xingyu, 2018. "Pricing and hedging vulnerable option with funding costs and collateral," Chaos, Solitons & Fractals, Elsevier, vol. 112(C), pages 103-115.
    18. Giorgia Callegaro & Alessandro Gnoatto & Martino Grasselli, 2021. "A Fully Quantization-based Scheme for FBSDEs," Working Papers 07/2021, University of Verona, Department of Economics.
    19. Dirk Becherer & Wilfried Kuissi-Kamdem & Olivier Menoukeu-Pamen, 2023. "Optimal consumption with labor income and borrowing constraints for recursive preferences," Working Papers hal-04017143, HAL.
    20. Qi Zeng & Hae Won (Henny) Jung, 2014. "Optimal Contract, Ownership Structure and Asset Pricing," 2014 Meeting Papers 911, Society for Economic Dynamics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2211.07622. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.