IDEAS home Printed from https://ideas.repec.org/a/spr/mathme/v74y2011i3p361-379.html
   My bibliography  Save this article

Markov Decision Processes with Average-Value-at-Risk criteria

Author

Listed:
  • Nicole Bäuerle
  • Jonathan Ott

Abstract

We investigate the problem of minimizing the Average-Value-at-Risk (AVaR τ ) of the discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). We show that this problem can be reduced to an ordinary MDP with extended state space and give conditions under which an optimal policy exists. We also give a time-consistent interpretation of the AVaR τ . At the end we consider a numerical example which is a simple repeated casino game. It is used to discuss the influence of the risk aversion parameter τ of the AVaR τ -criterion. Copyright Springer-Verlag 2011

Suggested Citation

  • Nicole Bäuerle & Jonathan Ott, 2011. "Markov Decision Processes with Average-Value-at-Risk criteria," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 74(3), pages 361-379, December.
    • Handle: RePEc:spr:mathme:v:74:y:2011:i:3:p:361-379
    DOI: 10.1007/s00186-011-0367-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00186-011-0367-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s00186-011-0367-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jocelyne Bion-Nadal, 2008. "Dynamic risk measures: Time consistency and risk measures from BMO martingales," Finance and Stochastics, Springer, vol. 12(2), pages 219-244, April.
    2. Acerbi, Carlo & Tasche, Dirk, 2002. "On the coherence of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1487-1503, July.
    3. Duan Li & Wan‐Lung Ng, 2000. "Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation," Mathematical Finance, Wiley Blackwell, vol. 10(3), pages 387-406, July.
    4. Kang Boda & Jerzy Filar, 2006. "Time Consistent Dynamic Risk Measures," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 63(1), pages 169-186, February.
    5. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    6. Rockafellar, R. Tyrrell & Uryasev, Stanislav, 2002. "Conditional value-at-risk for general loss distributions," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1443-1471, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicole Bäuerle & Alexander Glauner, 2021. "Minimizing spectral risk measures applied to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 94(1), pages 35-69, August.
    2. Christopher W. Miller & Insoon Yang, 2015. "Optimal Control of Conditional Value-at-Risk in Continuous Time," Papers 1512.05015, arXiv.org, revised Jan 2017.
    3. Seungki Min & Ciamac C. Moallemi & Costis Maglaras, 2022. "Risk-Sensitive Optimal Execution via a Conditional Value-at-Risk Objective," Papers 2201.11962, arXiv.org.
    4. Vamsi K. Potluru & Daniel Borrajo & Andrea Coletta & Niccol`o Dalmasso & Yousef El-Laham & Elizabeth Fons & Mohsen Ghassemi & Sriram Gopalakrishnan & Vikesh Gosai & Eleonora Kreav{c}i'c & Ganapathy Ma, 2023. "Synthetic Data Applications in Finance," Papers 2401.00081, arXiv.org, revised Mar 2024.
    5. Juri Hinz & Tanya Tarnopolskaya & Jeremy Yee, 2020. "Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations," Annals of Operations Research, Springer, vol. 286(1), pages 583-615, March.
    6. Charilaos Mertzanis, 2013. "Risk Management Challenges after the Financial Crisis," Economic Notes, Banca Monte dei Paschi di Siena SpA, vol. 42(3), pages 285-320, November.
    7. Vito Alessandro Monaco & Antonio Riva & Luca Sabbioni & Lorenzo Bisi & Edoardo Vittori & Marco Pinciroli & Michele Trapletti & Marcello Restelli, 2024. "Exploiting Risk-Aversion and Size-dependent fees in FX Trading with Fitted Natural Actor-Critic," Papers 2410.23294, arXiv.org.
    8. Bäuerle, Nicole & Glauner, Alexander, 2022. "Markov decision processes with recursive risk measures," European Journal of Operational Research, Elsevier, vol. 296(3), pages 953-966.
    9. So, Mee Chi & Thomas, Lyn C. & Huang, Bo, 2016. "Lending decisions with limits on capital available: The polygamous marriage problem," European Journal of Operational Research, Elsevier, vol. 249(2), pages 407-416.
    10. Nicole Bauerle & Alexander Glauner, 2020. "Minimizing Spectral Risk Measures Applied to Markov Decision Processes," Papers 2012.04521, arXiv.org.
    11. Julio Backhoff Veraguas & A. Max Reppen & Ludovic Tangpi, 2020. "Stochastic control of optimized certainty equivalents," Papers 2001.10108, arXiv.org, revised Jun 2022.
    12. Nicole Bauerle & Alexander Glauner, 2020. "Distributionally Robust Markov Decision Processes and their Connection to Risk Measures," Papers 2007.13103, arXiv.org.
    13. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    14. Li Xia & Luyao Zhang & Peter W. Glynn, 2023. "Risk‐sensitive Markov decision processes with long‐run CVaR criterion," Production and Operations Management, Production and Operations Management Society, vol. 32(12), pages 4049-4067, December.
    15. Constantin Waubert de Puiseau & Richard Meyes & Tobias Meisen, 2022. "On reliability of reinforcement learning based production scheduling systems: a comparative survey," Journal of Intelligent Manufacturing, Springer, vol. 33(4), pages 911-927, April.
    16. Qiuli Liu & Wai-Ki Ching & Xianping Guo, 2023. "Zero-sum stochastic games with the average-value-at-risk criterion," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(3), pages 618-647, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhiping Chen & Jia Liu & Gang Li & Zhe Yan, 2016. "Composite time-consistent multi-period risk measure and its application in optimal portfolio selection," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(3), pages 515-540, October.
    2. Strub, Moris S. & Li, Duan & Cui, Xiangyu & Gao, Jianjun, 2019. "Discrete-time mean-CVaR portfolio selection and time-consistency induced term structure of the CVaR," Journal of Economic Dynamics and Control, Elsevier, vol. 108(C).
    3. Özlem Çavuş & Andrzej Ruszczyński, 2014. "Computational Methods for Risk-Averse Undiscounted Transient Markov Models," Operations Research, INFORMS, vol. 62(2), pages 401-417, April.
    4. Gao, Jianjun & Xiong, Yan & Li, Duan, 2016. "Dynamic mean-risk portfolio selection with multiple risk measures in continuous-time," European Journal of Operational Research, Elsevier, vol. 249(2), pages 647-656.
    5. Xiangyu Cui & Xun Li & Duan Li & Yun Shi, 2014. "Time Consistent Behavior Portfolio Policy for Dynamic Mean-Variance Formulation," Papers 1408.6070, arXiv.org, revised Aug 2015.
    6. Kull, Andreas, 2009. "Sharing Risk – An Economic Perspective," ASTIN Bulletin, Cambridge University Press, vol. 39(2), pages 591-613, November.
    7. Brian Tomlin & Yimin Wang, 2005. "On the Value of Mix Flexibility and Dual Sourcing in Unreliable Newsvendor Networks," Manufacturing & Service Operations Management, INFORMS, vol. 7(1), pages 37-57, June.
    8. Fermanian, Jean-David & Scaillet, Olivier, 2005. "Sensitivity analysis of VaR and Expected Shortfall for portfolios under netting agreements," Journal of Banking & Finance, Elsevier, vol. 29(4), pages 927-958, April.
    9. Maziar Sahamkhadam, 2021. "Dynamic copula-based expectile portfolios," Journal of Asset Management, Palgrave Macmillan, vol. 22(3), pages 209-223, May.
    10. Martin Herdegen & Cosimo Munari, 2023. "An elementary proof of the dual representation of Expected Shortfall," Papers 2306.14506, arXiv.org.
    11. Karma, Otto & Sander, Priit, 2006. "The impact of financial leverage on risk of equity measured by loss-oriented risk measures: An option pricing approach," European Journal of Operational Research, Elsevier, vol. 175(3), pages 1340-1356, December.
    12. Wolfgang Kürsten & Mario Brandtner, 2009. "Kohärente Risikomessung versus individuelle Akzeptanzmengen — Anmerkungen zum impliziten Risikoverständnis des “Conditional Value-at-Risk”," Schmalenbach Journal of Business Research, Springer, vol. 61(4), pages 358-381, June.
    13. Cui, Xiangyu & Li, Duan & Shi, Yun, 2017. "Self-coordination in time inconsistent stochastic decision problems: A planner–doer game framework," Journal of Economic Dynamics and Control, Elsevier, vol. 75(C), pages 91-113.
    14. Alessandro Staino & Emilio Russo & Massimo Costabile & Arturo Leccadito, 2023. "Minimum capital requirement and portfolio allocation for non-life insurance: a semiparametric model with Conditional Value-at-Risk (CVaR) constraint," Computational Management Science, Springer, vol. 20(1), pages 1-32, December.
    15. Markus Leippold & Nikola Vasiljević, 2020. "Option-Implied Intrahorizon Value at Risk," Management Science, INFORMS, vol. 66(1), pages 397-414, January.
    16. Geissel Sebastian & Sass Jörn & Seifried Frank Thomas, 2018. "Optimal expected utility risk measures," Statistics & Risk Modeling, De Gruyter, vol. 35(1-2), pages 73-87, January.
    17. Daouia, Abdelaati & Girard, Stéphane & Stupfler, Gilles, 2018. "Tail expectile process and risk assessment," TSE Working Papers 18-944, Toulouse School of Economics (TSE).
    18. Dan A. Iancu & Marek Petrik & Dharmashankar Subramanian, 2015. "Tight Approximations of Dynamic Risk Measures," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 655-682, March.
    19. Dominique Guegan & Bertrand K Hassani, 2014. "Distortion Risk Measures or the Transformation of Unimodal Distributions into Multimodal Functions," Documents de travail du Centre d'Economie de la Sorbonne 14008, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    20. Brandtner, Mario, 2018. "Expected Shortfall, spectral risk measures, and the aggravating effect of background risk, or: risk vulnerability and the problem of subadditivity," Journal of Banking & Finance, Elsevier, vol. 89(C), pages 138-149.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:mathme:v:74:y:2011:i:3:p:361-379. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.