IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v30y2018i1p106-123.html
   My bibliography  Save this article

Benchmarking a Scalable Approximate Dynamic Programming Algorithm for Stochastic Control of Grid-Level Energy Storage

Author

Listed:
  • Daniel F. Salas

    (Department of Chemical and Biological Engineering, Princeton University, Princeton, New Jersey 08540)

  • Warren B. Powell

    (Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540)

Abstract

We present and benchmark an approximate dynamic programming algorithm that is capable of designing near-optimal control policies for a portfolio of heterogenous storage devices in a time-dependent environment, where wind supply, demand, and electricity prices may evolve stochastically. We found that the algorithm was able to design storage policies that are within 0.08% of optimal on deterministic models, and within 0.86% on stochastic models. We use the algorithm to analyze a dual-storage system with different capacities and losses, and show that the policy properly uses the low-loss device (which is typically much more expensive) for high-frequency variations. We close by demonstrating the algorithm on a five-device system. The algorithm easily scales to handle heterogeneous portfolios of storage devices distributed over the grid and more complex storage networks.

Suggested Citation

  • Daniel F. Salas & Warren B. Powell, 2018. "Benchmarking a Scalable Approximate Dynamic Programming Algorithm for Stochastic Control of Grid-Level Energy Storage," INFORMS Journal on Computing, INFORMS, vol. 30(1), pages 106-123, February.
    • Handle: RePEc:inm:orijoc:v:30:y:2018:i:1:p:106-123
    DOI: 10.1287/ijoc.2017.0768
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2017.0768
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2017.0768?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shapiro, Alexander & Tekaya, Wajdi & da Costa, Joari Paulo & Soares, Murilo Pereira, 2013. "Risk neutral and risk averse Stochastic Dual Dynamic Programming method," European Journal of Operational Research, Elsevier, vol. 224(2), pages 375-391.
    2. Nils Löhndorf & David Wozabal & Stefan Minner, 2013. "Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming," Operations Research, INFORMS, vol. 61(4), pages 810-823, August.
    3. Sioshansi, Ramteen & Hurlbut, David, 2010. "Market protocols in ERCOT and their effect on wind generation," Energy Policy, Elsevier, vol. 38(7), pages 3192-3197, July.
    4. Huseyin Topaloglu & Warren B. Powell, 2005. "A Distributed Decision-Making Structure for Dynamic Resource Allocation Using Nonlinear Functional Approximations," Operations Research, INFORMS, vol. 53(2), pages 281-297, April.
    5. Kuperman, Alon & Aharon, Ilan, 2011. "Battery-ultracapacitor hybrids for pulsed current loads: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(2), pages 981-992, February.
    6. Warren B. Powell & Abraham George & Hugo Simão & Warren Scott & Alan Lamont & Jeffrey Stewart, 2012. "SMART: A Stochastic Multiscale Model for the Analysis of Energy Resources, Technology, and Policy," INFORMS Journal on Computing, INFORMS, vol. 24(4), pages 665-682, November.
    7. Warren Powell & Andrzej Ruszczyński & Huseyin Topaloglu, 2004. "Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems," Mathematics of Operations Research, INFORMS, vol. 29(4), pages 814-836, November.
    8. Philpott, A.B. & de Matos, V.L., 2012. "Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion," European Journal of Operational Research, Elsevier, vol. 218(2), pages 470-483.
    9. Fleten, Stein-Erik & Kristoffersen, Trine Krogh, 2007. "Stochastic programming for optimizing bidding strategies of a Nordic hydropower producer," European Journal of Operational Research, Elsevier, vol. 181(2), pages 916-928, September.
    10. Gregory A. Godfrey & Warren B. Powell, 2001. "An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution," Management Science, INFORMS, vol. 47(8), pages 1101-1112, August.
    11. Huseyin Topaloglu & Warren B. Powell, 2006. "Dynamic-Programming Approximations for Stochastic Time-Staged Integer Multicommodity-Flow Problems," INFORMS Journal on Computing, INFORMS, vol. 18(1), pages 31-42, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Finnah, Benedikt & Gönsch, Jochen & Ziel, Florian, 2022. "Integrated day-ahead and intraday self-schedule bidding for energy storage systems using approximate dynamic programming," European Journal of Operational Research, Elsevier, vol. 301(2), pages 726-746.
    2. Lin, Zhiyi & Song, Chunyue & Zhao, Jun & Yin, Huan, 2022. "Improved approximate dynamic programming for real-time economic dispatch of integrated microgrids," Energy, Elsevier, vol. 255(C).
    3. Wen, Kerui & Li, Weidong & Yu, Samson Shenglong & Li, Ping & Shi, Peng, 2022. "Optimal intra-day operations of behind-the-meter battery storage for primary frequency regulation provision: A hybrid lookahead method," Energy, Elsevier, vol. 247(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    2. Zhou, Shaorui & Zhang, Hui & Shi, Ning & Xu, Zhou & Wang, Fan, 2020. "A new convergent hybrid learning algorithm for two-stage stochastic programs," European Journal of Operational Research, Elsevier, vol. 283(1), pages 33-46.
    3. Luckny Zéphyr & C. Lindsay Anderson, 2018. "Stochastic dynamic programming approach to managing power system uncertainty with distributed storage," Computational Management Science, Springer, vol. 15(1), pages 87-110, January.
    4. Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
    5. D. Ávila & A. Papavasiliou & N. Löhndorf, 2022. "Parallel and distributed computing for stochastic dual dynamic programming," Computational Management Science, Springer, vol. 19(2), pages 199-226, June.
    6. Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
    7. Michelle Bandarra & Vincent Guigues, 2021. "Single cut and multicut stochastic dual dynamic programming with cut selection for multistage stochastic linear programs: convergence proof and numerical experiments," Computational Management Science, Springer, vol. 18(2), pages 125-148, June.
    8. Cong Shi & Weidong Chen & Izak Duenyas, 2016. "Technical Note—Nonparametric Data-Driven Algorithms for Multiproduct Inventory Systems with Censored Demand," Operations Research, INFORMS, vol. 64(2), pages 362-370, April.
    9. Davi Valladão & Thuener Silva & Marcus Poggi, 2019. "Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns," Annals of Operations Research, Springer, vol. 282(1), pages 379-405, November.
    10. Pan, Zhenning & Yu, Tao & Li, Jie & Qu, Kaiping & Yang, Bo, 2020. "Risk-averse real-time dispatch of integrated electricity and heat system using a modified approximate dynamic programming approach," Energy, Elsevier, vol. 198(C).
    11. Xinqi Chen & Xingyu Bai & Zeyu Zheng & Nian Si, 2025. "Bias Analysis of Experiments for Multi-Item Multi-Period Inventory Control Policies," Papers 2501.11996, arXiv.org.
    12. Löhndorf, Nils & Wozabal, David, 2021. "Gas storage valuation in incomplete markets," European Journal of Operational Research, Elsevier, vol. 288(1), pages 318-330.
    13. Escudero, Laureano F. & Monge, Juan F. & Rodríguez-Chía, Antonio M., 2020. "On pricing-based equilibrium for network expansion planning. A multi-period bilevel approach under uncertainty," European Journal of Operational Research, Elsevier, vol. 287(1), pages 262-279.
    14. W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
    15. SteadieSeifi, M. & Dellaert, N.P. & Nuijten, W. & Van Woensel, T. & Raoufi, R., 2014. "Multimodal freight transportation planning: A literature review," European Journal of Operational Research, Elsevier, vol. 233(1), pages 1-15.
    16. Gah-Yi Ban, 2020. "Confidence Intervals for Data-Driven Inventory Policies with Demand Censoring," Operations Research, INFORMS, vol. 68(2), pages 309-326, March.
    17. Pagnoncelli, Bernardo K. & Homem-de-Mello, Tito & Lagos, Guido & Castañeda, Pablo & García, Javier, 2024. "Solving constrained consumption–investment problems by decomposition algorithms," European Journal of Operational Research, Elsevier, vol. 319(1), pages 292-302.
    18. Soares, Murilo Pereira & Street, Alexandre & Valladão, Davi Michel, 2017. "On the solution variability reduction of Stochastic Dual Dynamic Programming applied to energy planning," European Journal of Operational Research, Elsevier, vol. 258(2), pages 743-760.
    19. Belgacem Bouzaiene-Ayari & Clark Cheng & Sourav Das & Ricardo Fiorillo & Warren B. Powell, 2016. "From Single Commodity to Multiattribute Models for Locomotive Optimization: A Comparison of Optimal Integer Programming and Approximate Dynamic Programming," Transportation Science, INFORMS, vol. 50(2), pages 366-389, May.
    20. Andy Philpott & Vitor de Matos & Erlon Finardi, 2013. "On Solving Multistage Stochastic Programs with Coherent Risk Measures," Operations Research, INFORMS, vol. 61(4), pages 957-970, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:30:y:2018:i:1:p:106-123. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.