IDEAS home Printed from https://ideas.repec.org/a/eee/renene/v178y2021icp13-24.html
   My bibliography  Save this article

An advanced real-time dispatching strategy for a distributed energy system based on the reinforcement learning algorithm

Author

Listed:
  • Meng, Fanyi
  • Bai, Yang
  • Jin, Jingliang

Abstract

A desirable dispatching strategy is essentially important for securely and economically operating of wind-thermal hybrid distribution systems. Existing dispatch strategies usually assume that wind power has priority of injection. For real-time control, such strategies are simple and easy to realize, but they lack flexibility and incur higher operation and maintenance (O&M) costs. This study analyzed the power dispatching process as a dynamic sequential control problem and established a Markov decision process model to explore the optimal coordinated dispatch strategy for coping with wind and demand disturbance. As a salient feature, the improved dispatch strategy minimizes the long-run expected operation and maintenance costs. To evaluate the model efficiently, a Monte Carlo method and the Q-learning algorithm were employed to the growing computational cost over the state space. Through a specified numerical case, we demonstrated the properties of the coordinated dispatch strategy and used it to address a 24-h real-time dispatching problem. The proposed algorithm shows high efficiency in solving real-time dispatching problems.

Suggested Citation

  • Meng, Fanyi & Bai, Yang & Jin, Jingliang, 2021. "An advanced real-time dispatching strategy for a distributed energy system based on the reinforcement learning algorithm," Renewable Energy, Elsevier, vol. 178(C), pages 13-24.
    • Handle: RePEc:eee:renene:v:178:y:2021:i:c:p:13-24
    DOI: 10.1016/j.renene.2021.06.032
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960148121008983
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.renene.2021.06.032?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jin, Jingliang & Zhou, Dequn & Zhou, Peng & Qian, Shuqu & Zhang, Mingming, 2016. "Dispatching strategies for coordinating environmental awareness and risk perception in wind power integrated system," Energy, Elsevier, vol. 106(C), pages 453-463.
    2. Liao, Gwo-Ching, 2011. "A novel evolutionary algorithm for dynamic economic dispatch with energy saving and emission reduction in power system integrated wind power," Energy, Elsevier, vol. 36(2), pages 1018-1029.
    3. Tuohy, Aidan & Meibom, Peter & Denny, Eleanor & O'Malley, Mark, 2009. "Unit commitment for systems with significant wind penetration," MPRA Paper 34849, University Library of Munich, Germany.
    4. Sun, Mei & Li, Juan & Gao, Cuixia & Han, Dun, 2017. "Identifying regime shifts in the US electricity market based on price fluctuations," Applied Energy, Elsevier, vol. 194(C), pages 658-666.
    5. Kaldellis, J.K. & Vlachos, G.Th., 2006. "Optimum sizing of an autonomous wind-diesel hybrid system for various representative wind-potential cases," Applied Energy, Elsevier, vol. 83(2), pages 113-132, February.
    6. Anthony Papavasiliou & Shmuel S. Oren, 2013. "Multiarea Stochastic Unit Commitment for High Wind Penetration in a Transmission Constrained Network," Operations Research, INFORMS, vol. 61(3), pages 578-592, June.
    7. Zhang, Ling & Zhou, Peng & Newton, Sidney & Fang, Jian-xin & Zhou, De-qun & Zhang, Lu-ping, 2015. "Evaluating clean energy alternatives for Jiangsu, China: An improved multi-criteria decision making method," Energy, Elsevier, vol. 90(P1), pages 953-964.
    8. Jin, Jingliang & Zhou, Peng & Li, Chenyu & Guo, Xiaojun & Zhang, Mingming, 2019. "Low-carbon power dispatch with wind power based on carbon trading mechanism," Energy, Elsevier, vol. 170(C), pages 250-260.
    9. Younes, Mimoun & Khodja, Fouad & Kherfane, Riad Lakhdar, 2014. "Multi-objective economic emission dispatch solution using hybrid FFA (firefly algorithm) and considering wind power penetration," Energy, Elsevier, vol. 67(C), pages 595-606.
    10. Yuan, Xiaohui & Su, Anjun & Yuan, Yanbin & Nie, Hao & Wang, Liang, 2009. "An improved PSO for dynamic load dispatch of generators with valve-point effects," Energy, Elsevier, vol. 34(1), pages 67-74.
    11. Sun, Qirun & Wu, Zhi & Gu, Wei & Zhu, Tao & Zhong, Lei & Gao, Ting, 2021. "Flexible expansion planning of distribution system integrating multiple renewable energy sources: An approximate dynamic programming approach," Energy, Elsevier, vol. 226(C).
    12. Laura, Castro-Santos & Vicente, Diaz-Casas, 2014. "Life-cycle cost analysis of floating offshore wind farms," Renewable Energy, Elsevier, vol. 66(C), pages 41-48.
    13. Zhang, Mingming & Liu, Liyun & Wang, Qunwei & Zhou, Dequn, 2020. "Valuing investment decisions of renewable energy projects considering changing volatility," Energy Economics, Elsevier, vol. 92(C).
    14. PAPAVASILIOU, Anthony & OREN, Schmuel S., 2013. "Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network," LIDAM Reprints CORE 2500, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    15. Jin, Jingliang & Zhou, Peng & Zhang, Mingming & Yu, Xianyu & Din, Hao, 2018. "Balancing low-carbon power dispatching strategy for wind power integrated system," Energy, Elsevier, vol. 149(C), pages 914-924.
    16. Panapakidis, Ioannis P. & Dagoumas, Athanasios S., 2017. "Day-ahead natural gas demand forecasting based on the combination of wavelet transform and ANFIS/genetic algorithm/neural network model," Energy, Elsevier, vol. 118(C), pages 231-245.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ding, Yihong & Tan, Qinliang & Shan, Zijing & Han, Jian & Zhang, Yimei, 2023. "A two-stage dispatching optimization strategy for hybrid renewable energy system with low-carbon and sustainability in ancillary service market," Renewable Energy, Elsevier, vol. 207(C), pages 647-659.
    2. Dai, Yeming & Sun, Xilian & Qi, Yao & Leng, Mingming, 2021. "A real-time, personalized consumption-based pricing scheme for the consumptions of traditional and renewable energies," Renewable Energy, Elsevier, vol. 180(C), pages 452-466.
    3. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    4. Jin, Jingliang & Wen, Qinglan & Cheng, Siqi & Qiu, Yaru & Zhang, Xianyue & Guo, Xiaojun, 2022. "Optimization of carbon emission reduction paths in the low-carbon power dispatching process," Renewable Energy, Elsevier, vol. 188(C), pages 425-436.
    5. Fang, Jianhao & Hu, Weifei & Liu, Zhenyu & Chen, Weiyi & Tan, Jianrong & Jiang, Zhiyu & Verma, Amrit Shankar, 2022. "Wind turbine rotor speed design optimization considering rain erosion based on deep reinforcement learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Bo & Wang, Shuming & Zhou, Xianzhong & Watada, Junzo, 2016. "Multi-objective unit commitment with wind penetration and emission concerns under stochastic and fuzzy uncertainties," Energy, Elsevier, vol. 111(C), pages 18-31.
    2. Rahmani, Shima & Amjady, Nima, 2017. "A new optimal power flow approach for wind energy integrated power systems," Energy, Elsevier, vol. 134(C), pages 349-359.
    3. Abdul Rauf & Mahmoud Kassas & Muhammad Khalid, 2022. "Data-Driven Optimal Battery Storage Sizing for Grid-Connected Hybrid Distributed Generations Considering Solar and Wind Uncertainty," Sustainability, MDPI, vol. 14(17), pages 1-27, September.
    4. De Vos, K. & Stevens, N. & Devolder, O. & Papavasiliou, A. & Hebb, B. & Matthys-Donnadieu, J., 2019. "Dynamic dimensioning approach for operating reserves: Proof of concept in Belgium," Energy Policy, Elsevier, vol. 124(C), pages 272-285.
    5. Johnson, Samuel C. & Papageorgiou, Dimitri J. & Mallapragada, Dharik S. & Deetjen, Thomas A. & Rhodes, Joshua D. & Webber, Michael E., 2019. "Evaluating rotational inertia as a component of grid reliability with high penetrations of variable renewable energy," Energy, Elsevier, vol. 180(C), pages 258-271.
    6. Aghaei, Jamshid & Nikoobakht, Ahmad & Siano, Pierluigi & Nayeripour, Majid & Heidari, Alireza & Mardaneh, Mohammad, 2016. "Exploring the reliability effects on the short term AC security-constrained unit commitment: A stochastic evaluation," Energy, Elsevier, vol. 114(C), pages 1016-1032.
    7. Heejung Park, 2022. "A Unit Commitment Model Considering Feasibility of Operating Reserves under Stochastic Optimization Framework," Energies, MDPI, vol. 15(17), pages 1-22, August.
    8. Yonghan Feng & Sarah Ryan, 2016. "Solution sensitivity-based scenario reduction for stochastic unit commitment," Computational Management Science, Springer, vol. 13(1), pages 29-62, January.
    9. Ilias G. Marneris & Pandelis N. Biskas & Anastasios G. Bakirtzis, 2017. "Stochastic and Deterministic Unit Commitment Considering Uncertainty and Variability Reserves for High Renewable Integration," Energies, MDPI, vol. 10(1), pages 1-25, January.
    10. Álvaro Lorca & X. Andy Sun & Eugene Litvinov & Tongxin Zheng, 2016. "Multistage Adaptive Robust Optimization for the Unit Commitment Problem," Operations Research, INFORMS, vol. 64(1), pages 32-51, February.
    11. Jan Abrell & Friedrich Kunz, 2015. "Integrating Intermittent Renewable Wind Generation - A Stochastic Multi-Market Electricity Model for the European Electricity Market," Networks and Spatial Economics, Springer, vol. 15(1), pages 117-147, March.
    12. Fattahi, Salar & Ashraphijuo, Morteza & Lavaei, Javad & Atamtürk, Alper, 2017. "Conic relaxations of the unit commitment problem," Energy, Elsevier, vol. 134(C), pages 1079-1095.
    13. David Schönheit & Dominik Möst, 2019. "The Effect of Offshore Wind Capacity Expansion on Uncertainties in Germany’s Day-Ahead Wind Energy Forecasts," Energies, MDPI, vol. 12(13), pages 1-23, July.
    14. Mohammadi-ivatloo, Behnam & Rabiee, Abbas & Soroudi, Alireza & Ehsan, Mehdi, 2012. "Imperialist competitive algorithm for solving non-convex dynamic economic power dispatch," Energy, Elsevier, vol. 44(1), pages 228-240.
    15. Niknam, Taher & Azizipanah-Abarghooee, Rasoul & Roosta, Alireza & Amiri, Babak, 2012. "A new multi-objective reserve constrained combined heat and power dynamic economic emission dispatch," Energy, Elsevier, vol. 42(1), pages 530-545.
    16. Faezeh Akhavizadegan & Lizhi Wang & James McCalley, 2020. "Scenario Selection for Iterative Stochastic Transmission Expansion Planning," Energies, MDPI, vol. 13(5), pages 1-18, March.
    17. Victor M. Zavala & Kibaek Kim & Mihai Anitescu & John Birge, 2017. "A Stochastic Electricity Market Clearing Formulation with Consistent Pricing Properties," Operations Research, INFORMS, vol. 65(3), pages 557-576, June.
    18. Skolfield, J. Kyle & Escobedo, Adolfo R., 2022. "Operations research in optimal power flow: A guide to recent and emerging methodologies and applications," European Journal of Operational Research, Elsevier, vol. 300(2), pages 387-404.
    19. Niknam, Taher & Azizipanah-Abarghooee, Rasoul & Narimani, Mohammad Rasoul, 2012. "Reserve constrained dynamic optimal power flow subject to valve-point effects, prohibited zones and multi-fuel constraints," Energy, Elsevier, vol. 47(1), pages 451-464.
    20. Noori, Ehsan & Khazaei, Ehsan & Tavaro, Mehdi & Bardideh, Farhad, 2019. "Economically Operation of Power Utilities Base on MILP Approach," MPRA Paper 95910, University Library of Munich, Germany.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:renene:v:178:y:2021:i:c:p:13-24. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/renewable-energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.