IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v290y2021i2p640-656.html
   My bibliography  Save this article

Statistical learning for probability-constrained stochastic optimal control

Author

Listed:
  • Balata, Alessandro
  • Ludkovski, Michael
  • Maheshwari, Aditya
  • Palczewski, Jan

Abstract

We investigate Monte Carlo based algorithms for solving stochastic control problems with local probabilistic constraints. Our motivation comes from microgrid management, where the controller tries to optimally dispatch a diesel generator while maintaining low probability of blackouts at each step. The key question we investigate are empirical simulation procedures for learning the state-dependent admissible control set that is specified implicitly through a probability constraint on the system state. We propose a variety of relevant statistical tools including logistic regression, Gaussian process regression, quantile regression and support vector machines, which we then incorporate into an overall Regression Monte Carlo (RMC) framework for approximate dynamic programming. Our results indicate that using logistic or Gaussian process regression to estimate the admissibility probability outperforms the other options. Our algorithms offer an efficient and reliable extension of RMC to probability-constrained control. We illustrate our findings with two case studies for the microgrid problem.

Suggested Citation

  • Balata, Alessandro & Ludkovski, Michael & Maheshwari, Aditya & Palczewski, Jan, 2021. "Statistical learning for probability-constrained stochastic optimal control," European Journal of Operational Research, Elsevier, vol. 290(2), pages 640-656.
    • Handle: RePEc:eee:ejores:v:290:y:2021:i:2:p:640-656
    DOI: 10.1016/j.ejor.2020.08.041
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221720307542
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2020.08.041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," The Review of Financial Studies, Society for Financial Studies, vol. 14(1), pages 113-147.
    2. Andrieu, L. & Henrion, R. & Römisch, W., 2010. "A model for dynamic chance constraints in hydro power reservoir management," European Journal of Operational Research, Elsevier, vol. 207(2), pages 579-589, December.
    3. Abebe Geletu & Michael Klöppel & Hui Zhang & Pu Li, 2013. "Advances and applications of chance-constrained approaches to systems optimisation under uncertainty," International Journal of Systems Science, Taylor & Francis Journals, vol. 44(7), pages 1209-1232.
    4. repec:dau:papers:123456789/4273 is not listed on IDEAS
    5. Jun Xu & J. Scott Long, 2005. "Confidence intervals for predicted outcomes in regression models for categorical outcomes," Stata Journal, StataCorp LP, vol. 5(4), pages 537-559, December.
    6. Roustant, Olivier & Ginsbourger, David & Deville, Yves, 2012. "DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i01).
    7. repec:dau:papers:123456789/11531 is not listed on IDEAS
    8. Rene Carmona & Michael Ludkovski, 2010. "Valuation of energy storage: an optimal switching approach," Quantitative Finance, Taylor & Francis Journals, vol. 10(4), pages 359-374.
    9. Ying Jiao & Olivier Klopfenstein & Peter Tankov, 2017. "Hedging under multiple risk constraints," Finance and Stochastics, Springer, vol. 21(2), pages 361-396, April.
    10. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Maximilien Germain & Huy^en Pham & Xavier Warin, 2021. "A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection," Papers 2112.11059, arXiv.org, revised Nov 2022.
    2. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection," Working Papers hal-03498263, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alessandro Balata & Michael Ludkovski & Aditya Maheshwari & Jan Palczewski, 2019. "Statistical Learning for Probability-Constrained Stochastic Optimal Control," Papers 1905.00107, arXiv.org, revised Aug 2020.
    2. Hanfeld, Marc & Schlüter, Stephan, 2016. "Operating a swing option on today's gas markets: How least squares Monte Carlo works and why it is beneficial," FAU Discussion Papers in Economics 10/2016, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    3. Nemat Safarov & Colin Atkinson, 2017. "Natural Gas-Fired Power Plants Valuation And Optimization Under Lévy Copulas And Regime Switching," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 20(01), pages 1-38, February.
    4. Bastian Felix, 2012. "Gas Storage Valuation: A Comparative Simulation Study," EWL Working Papers 1201, University of Duisburg-Essen, Chair for Management Science and Energy Economics, revised Apr 2014.
    5. Nemat Safarov & Colin Atkinson, 2016. "Natural gas-fired power plants valuation and optimisation under Levy copulas and regime-switching," Papers 1607.01207, arXiv.org, revised Jul 2016.
    6. W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
    7. Trotter, Ian M. & Gomes, Marília Fernandes Maciel & Braga, Marcelo José & Brochmann, Bjørn & Lie, Ole Nikolai, 2016. "Optimal LNG (liquefied natural gas) regasification scheduling for import terminals with storage," Energy, Elsevier, vol. 105(C), pages 80-88.
    8. Nadarajah, Selvaprabu & Margot, François & Secomandi, Nicola, 2017. "Comparison of least squares Monte Carlo methods with applications to energy real options," European Journal of Operational Research, Elsevier, vol. 256(1), pages 196-204.
    9. Mike Ludkovski, 2020. "mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms," Papers 2012.00729, arXiv.org, revised Oct 2022.
    10. Thompson, Matt, 2016. "Natural gas storage valuation, optimization, market and credit risk management," Journal of Commodity Markets, Elsevier, vol. 2(1), pages 26-44.
    11. Cortazar, Gonzalo & Naranjo, Lorenzo & Sainz, Felipe, 2021. "Optimal decision policy for real options under general Markovian dynamics," European Journal of Operational Research, Elsevier, vol. 288(2), pages 634-647.
    12. R. Mark Reesor & T. James Marshall, 2020. "Forest of Stochastic Trees: A Method for Valuing Multiple Exercise Options," JRFM, MDPI, vol. 13(5), pages 1-31, May.
    13. Fabozzi, Frank J. & Paletta, Tommaso & Tunaru, Radu, 2017. "An improved least squares Monte Carlo valuation method based on heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 263(2), pages 698-706.
    14. Michael Ludkovski, 2015. "Kriging Metamodels and Experimental Design for Bermudan Option Pricing," Papers 1509.02179, arXiv.org, revised Oct 2016.
    15. Ruimeng Hu, 2019. "Deep Learning for Ranking Response Surfaces with Applications to Optimal Stopping Problems," Papers 1901.03478, arXiv.org, revised Mar 2020.
    16. Juri Hinz & Tanya Tarnopolskaya & Jeremy Yee, 2020. "Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations," Annals of Operations Research, Springer, vol. 286(1), pages 583-615, March.
    17. Michael Ludkovski & Aditya Maheshwari, 2018. "Simulation Methods for Stochastic Storage Problems: A Statistical Learning Perspective," Papers 1803.11309, arXiv.org.
    18. Kitapbayev, Yerkin & Moriarty, John & Mancarella, Pierluigi, 2015. "Stochastic control and real options valuation of thermal storage-enabled demand response from flexible district energy systems," Applied Energy, Elsevier, vol. 137(C), pages 823-831.
    19. Giovanni Mottola, 2014. "A stochastic switching control model arising in general OTC contracts with contingent CSA in presence of CVA, collateral and funding," Papers 1412.1469, arXiv.org.
    20. Alessio Trivella & Selvaprabu Nadarajah & Stein-Erik Fleten & Denis Mazieres & David Pisinger, 2021. "Managing Shutdown Decisions in Merchant Commodity and Energy Production: A Social Commerce Perspective," Manufacturing & Service Operations Management, INFORMS, vol. 23(2), pages 311-330, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:290:y:2021:i:2:p:640-656. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.