IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2023i1p38-d1305724.html
   My bibliography  Save this article

Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation

Author

Listed:
  • Xianggang Lu

    (School of Mathematics and Statistics, Guangdong University of Technology, Guangzhou 510006, China)

  • Lin Sun

    (School of Mathematics and Statistics, Guangdong University of Technology, Guangzhou 510006, China)

Abstract

This work considers the infinite horizon discounted risk-sensitive optimal control problem for the switching diffusions with a compact control space and controlled through the drift; thus, the the generator of the switching diffusions also depends on the controls. Note that the running cost of interest can be unbounded, so a decent estimation on the value function is obtained, under suitable conditions. To solve such a risk-sensitive optimal control problem, we adopt the viscosity solution methods and propose a numerical approximation scheme. We can verify that the value function of the optimal control problem solves the optimality equation as the unique viscosity solution. The optimality equation is also called the Hamilton–Jacobi–Bellman (HJB) equation, which is a second-order partial differential equation (PDE). Since, the explicit solutions to such PDEs are usually difficult to obtain, the finite difference approximation scheme is derived to approximate the value function. As a byproduct, the ϵ -optimal control of finite difference type is also obtained.

Suggested Citation

  • Xianggang Lu & Lin Sun, 2023. "Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation," Mathematics, MDPI, vol. 12(1), pages 1-24, December.
    • Handle: RePEc:gam:jmathe:v:12:y:2023:i:1:p:38-:d:1305724
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/1/38/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/1/38/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nguyen, Dang Hai & Yin, George & Zhu, Chao, 2017. "Certain properties related to well posedness of switching diffusions," Stochastic Processes and their Applications, Elsevier, vol. 127(10), pages 3135-3158.
    2. Xianggang Lu, 2019. "Constrained optimality for controlled switching diffusions with an application to stock purchasing," Quantitative Finance, Taylor & Francis Journals, vol. 19(12), pages 2069-2085, December.
    3. Peter Grandits & Friedrich Hubalek & Walter Schachermayer & Mislav Žigo, 2007. "Optimal expected exponential utility of dividend payments in a Brownian risk model," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2007(2), pages 73-107.
    4. V. S. Borkar, 2002. "Q-Learning for Risk-Sensitive Control," Mathematics of Operations Research, INFORMS, vol. 27(2), pages 294-311, May.
    5. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    6. Arapostathis, Ari & Biswas, Anup, 2018. "Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions," Stochastic Processes and their Applications, Elsevier, vol. 128(5), pages 1485-1524.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arnab Basu & Tirthankar Bhattacharyya & Vivek S. Borkar, 2008. "A Learning Algorithm for Risk-Sensitive Cost," Mathematics of Operations Research, INFORMS, vol. 33(4), pages 880-898, November.
    2. Pestien, Victor & Wang, Xiaobo, 1998. "Markov-achievable payoffs for finite-horizon decision models," Stochastic Processes and their Applications, Elsevier, vol. 73(1), pages 101-118, January.
    3. Basu, Arnab & Ghosh, Mrinal Kanti, 2014. "Zero-sum risk-sensitive stochastic games on a countable state space," Stochastic Processes and their Applications, Elsevier, vol. 124(1), pages 961-983.
    4. Bhabak, Arnab & Saha, Subhamay, 2022. "Risk-sensitive semi-Markov decision problems with discounted cost and general utilities," Statistics & Probability Letters, Elsevier, vol. 184(C).
    5. Liu, Meng & Bai, Chuanzhi, 2020. "Optimal harvesting of a stochastic mutualism model with regime-switching," Applied Mathematics and Computation, Elsevier, vol. 373(C).
    6. Lucy Gongtao Chen & Daniel Zhuoyu Long & Melvyn Sim, 2015. "On Dynamic Decision Making to Meet Consumption Targets," Operations Research, INFORMS, vol. 63(5), pages 1117-1130, October.
    7. Bäuerle, Nicole & Rieder, Ulrich, 2017. "Zero-sum risk-sensitive stochastic games," Stochastic Processes and their Applications, Elsevier, vol. 127(2), pages 622-642.
    8. Guglielmo D’Amico & Fulvio Gismondi & Jacques Janssen & Raimondo Manca, 2015. "Discrete Time Homogeneous Markov Processes for the Study of the Basic Risk Processes," Methodology and Computing in Applied Probability, Springer, vol. 17(4), pages 983-998, December.
    9. Zeynep Erkin & Matthew D. Bailey & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2010. "Eliciting Patients' Revealed Preferences: An Inverse Markov Decision Process Approach," Decision Analysis, INFORMS, vol. 7(4), pages 358-365, December.
    10. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    11. Nelson Vadori & Sumitra Ganesh & Prashant Reddy & Manuela Veloso, 2020. "Risk-Sensitive Reinforcement Learning: a Martingale Approach to Reward Uncertainty," Papers 2006.12686, arXiv.org, revised Sep 2020.
    12. Carlos Camilo-Garay & Rolando Cavazos-Cadena & Hugo Cruz-Suárez, 2022. "Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space," Journal of Optimization Theory and Applications, Springer, vol. 192(1), pages 271-291, January.
    13. C. Barz & K. Waldmann, 2007. "Risk-sensitive capacity control in revenue management," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 65(3), pages 565-579, June.
    14. Monahan, George E. & Sobel, Matthew J., 1997. "Risk-Sensitive Dynamic Market Share Attraction Games," Games and Economic Behavior, Elsevier, vol. 20(2), pages 149-160, August.
    15. V. S. Borkar & S. P. Meyn, 2002. "Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost," Mathematics of Operations Research, INFORMS, vol. 27(1), pages 192-209, February.
    16. HuiChen Chiang, 2007. "Financial intermediary's choice of borrowing," Applied Economics, Taylor & Francis Journals, vol. 40(2), pages 251-260.
    17. Jelito, Damian & Pitera, Marcin & Stettner, Łukasz, 2021. "Risk sensitive optimal stopping," Stochastic Processes and their Applications, Elsevier, vol. 136(C), pages 125-144.
    18. Kang Boda & Jerzy Filar, 2006. "Time Consistent Dynamic Risk Measures," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 63(1), pages 169-186, February.
    19. Erick Delage & Shie Mannor, 2010. "Percentile Optimization for Markov Decision Processes with Parameter Uncertainty," Operations Research, INFORMS, vol. 58(1), pages 203-213, February.
    20. Naci Saldi & Tamer Bas¸ ar & Maxim Raginsky, 2020. "Approximate Markov-Nash Equilibria for Discrete-Time Risk-Sensitive Mean-Field Games," Mathematics of Operations Research, INFORMS, vol. 45(4), pages 1596-1620, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2023:i:1:p:38-:d:1305724. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.