IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v49y2001i3p398-412.html
   My bibliography  Save this article

Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems

Author

Listed:
  • C. Russell Philbrick

    (Alstom Esca Corporation, 11120 NE 33rd Place, Bellevue, Washington 98004)

  • Peter K. Kitanidis

    (Department of Civil and Environmental Engineering, Stanford University, M24 Terman Engineering Center, Stanford, California 94305-4020)

Abstract

New dynamic programming methods are developed to solve stochastic control problems with a larger number of state variables than previously possible. These methods apply accurate interpolation to numerical approximation of continuous cost-to-go functions, greatly reducing the number of discrete states that must be evaluated. By efficiently incorporating information on first and second derivatives, the approximation reduces computational effort by several orders of magnitude over traditional methods. Consequently, it is practical to apply dynamic programming to complex stochastic problems with a larger number of state variables than traditionally possible. Results are presented for hypothetical reservoir control problems with up to seven state variables and two random inputs.

Suggested Citation

  • C. Russell Philbrick & Peter K. Kitanidis, 2001. "Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems," Operations Research, INFORMS, vol. 49(3), pages 398-412, June.
    • Handle: RePEc:inm:oropre:v:49:y:2001:i:3:p:398-412
    DOI: 10.1287/opre.49.3.398.11219
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.49.3.398.11219
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.49.3.398.11219?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yongyang Cai & Kenneth Judd, 2015. "Dynamic programming with Hermite approximation," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 81(3), pages 245-267, June.
    2. Borgonovo, E., 2010. "The reliability importance of components and prime implicants in coherent and non-coherent systems including total-order interactions," European Journal of Operational Research, Elsevier, vol. 204(3), pages 485-495, August.
    3. B. Luo & I. Maqsood & G. Huang, 2007. "Planning water resources systems with interval stochastic dynamic programming," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 21(6), pages 997-1014, June.
    4. Luckny Zéphyr & C. Lindsay Anderson, 2018. "Stochastic dynamic programming approach to managing power system uncertainty with distributed storage," Computational Management Science, Springer, vol. 15(1), pages 87-110, January.
    5. Peter Schober & Julian Valentin & Dirk Pflüger, 2022. "Solving High-Dimensional Dynamic Portfolio Choice Models with Hierarchical B-Splines on Sparse Grids," Computational Economics, Springer;Society for Computational Economics, vol. 59(1), pages 185-224, January.
    6. Cervellera, Cristiano & Chen, Victoria C.P. & Wen, Aihong, 2006. "Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization," European Journal of Operational Research, Elsevier, vol. 171(3), pages 1139-1151, June.
    7. Zhenfang Liu & Yang Zhou & Gordon Huang & Bin Luo, 2019. "Risk Aversion Based Inexact Stochastic Dynamic Programming Approach for Water Resources Management Planning under Uncertainty," Sustainability, MDPI, vol. 11(24), pages 1-22, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:49:y:2001:i:3:p:398-412. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.