IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v285y2021ics0306261920317876.html
   My bibliography  Save this article

Stimulus-response control strategy based on autonomous decentralized system theory for exploitation of flexibility by virtual power plant

Author

Listed:
  • Zhou, Huan
  • Fan, Shuai
  • Wu, Qing
  • Dong, Lianxin
  • Li, Zuyi
  • He, Guangyu

Abstract

To adequately utilize flexible resources on demand side, Virtual Power Plant (VPP) is an effective solution through the aggregation and application of distributed energy resources (DER). While centralized control approaches are easy to achieve global optimum for the scheduling of every DER, they have limitations when dealing with massive number of complex and heterogeneous DERs with time-varying states. Existing decentralized control approaches are mainly based on the assumption that all DERs are completely rational, which is quite far from the reality. In this paper, using a bottom-up approach, we propose a stimulus–response control strategy to realize exploitation of flexibility by VPP. In such a strategy, DERs are dynamically aggregated through autonomous decentralized system, and interact with each other via subscription and publication of topics, regardless of the source and recipient of the messages, thus removing the direct coupling relationship between VPP Operator and DERs. Furthermore, each DER makes an independent decision through edge computing at an agent that has a general End-to-End structure and is driven by the stimulus message received from VPP Operator. We develop a simple yet efficient double deep q-network (DDQN) algorithm to optimize the state sequence of DER agents. A simulation study is conducted with over 1000 DERs including photovoltaics, electric vehicles and air conditioners. Results indicate that the proposed approach can dynamically aggregate DERs and exploit their flexibility with each DER agent dynamically adapting to the change of stimulus signals, thus achieving dynamic, automatic and adaptive exploitation of flexibility by VPP.

Suggested Citation

  • Zhou, Huan & Fan, Shuai & Wu, Qing & Dong, Lianxin & Li, Zuyi & He, Guangyu, 2021. "Stimulus-response control strategy based on autonomous decentralized system theory for exploitation of flexibility by virtual power plant," Applied Energy, Elsevier, vol. 285(C).
    • Handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317876
    DOI: 10.1016/j.apenergy.2020.116424
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920317876
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2020.116424?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yu, Mengmeng & Hong, Seung Ho, 2017. "Incentive-based demand response considering hierarchical electricity market: A Stackelberg game approach," Applied Energy, Elsevier, vol. 203(C), pages 267-279.
    2. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    3. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    4. Perera, A.T.D. & Nik, Vahid M. & Wickramasinghe, P.U. & Scartezzini, Jean-Louis, 2019. "Redefining energy system flexibility for distributed energy system design," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    5. Xiao, Jucheng & He, Guangyu & Zhou, Huan & Zhang, Siyuan & Wang, Zhihua, 2019. "Decentralized transfer of contingency reserve: Concept, benefit assessment, impacting factors, and benefit mechanism," Applied Energy, Elsevier, vol. 255(C).
    6. Mahmud, Khizir & Khan, Behram & Ravishankar, Jayashri & Ahmadi, Abdollah & Siano, Pierluigi, 2020. "An internet of energy framework with distributed energy resources, prosumers and small-scale virtual power plants: An overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 127(C).
    7. Wang, Han & Riaz, Shariq & Mancarella, Pierluigi, 2020. "Integrated techno-economic modeling, flexibility analysis, and business case assessment of an urban virtual power plant with multi-market co-optimization," Applied Energy, Elsevier, vol. 259(C).
    8. Wozabal, David & Rameseder, Gunther, 2020. "Optimal bidding of a virtual power plant on the Spanish day-ahead and intraday market for electricity," European Journal of Operational Research, Elsevier, vol. 280(2), pages 639-655.
    9. Tang, Rui & Li, Hangxin & Wang, Shengwei, 2019. "A game theory-based decentralized control strategy for power demand management of building cluster using thermal mass and energy storage," Applied Energy, Elsevier, vol. 242(C), pages 809-820.
    10. Fan, Shuai & Liu, Jiang & Wu, Qing & Cui, Mingjian & Zhou, Huan & He, Guangyu, 2020. "Optimal coordination of virtual power plant with photovoltaics and electric vehicles: A temporally coupled distributed online algorithm," Applied Energy, Elsevier, vol. 277(C).
    11. Bale, Catherine S.E. & Varga, Liz & Foxon, Timothy J., 2015. "Energy and complexity: New ways forward," Applied Energy, Elsevier, vol. 138(C), pages 150-159.
    12. Jin, Xiaolong & Wu, Qiuwei & Jia, Hongjie, 2020. "Local flexibility markets: Literature review on concepts, models and clearing methods," Applied Energy, Elsevier, vol. 261(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Francesco Gulotta & Edoardo Daccò & Alessandro Bosisio & Davide Falabretti, 2023. "Opening of Ancillary Service Markets to Distributed Energy Resources: A Review," Energies, MDPI, vol. 16(6), pages 1-25, March.
    2. Saavedra, Aldo & Negrete-Pincetic, Matias & Rodríguez, Rafael & Salgado, Marcelo & Lorca, Álvaro, 2022. "Flexible load management using flexibility bands," Applied Energy, Elsevier, vol. 317(C).
    3. Ju, Liwei & Yin, Zhe & Zhou, Qingqing & Li, Qiaochu & Wang, Peng & Tian, Wenxu & Li, Peng & Tan, Zhongfu, 2022. "Nearly-zero carbon optimal operation model and benefit allocation strategy for a novel virtual power plant using carbon capture, power-to-gas, and waste incineration power in rural areas," Applied Energy, Elsevier, vol. 310(C).
    4. Xiong, Houbo & Luo, Fengji & Yan, Mingyu & Yan, Lei & Guo, Chuangxin & Ranzi, Gianluca, 2024. "Distributionally robust and transactive energy management scheme for integrated wind-concentrated solar virtual power plants," Applied Energy, Elsevier, vol. 368(C).
    5. Xue, Lin & Zhang, Yao & Wang, Jianxue & Li, Haotian & Li, Fangshi, 2024. "Privacy-preserving multi-level co-regulation of VPPs via hierarchical safe deep reinforcement learning," Applied Energy, Elsevier, vol. 371(C).
    6. Mei, Shufan & Tan, Qinliang & Liu, Yuan & Trivedi, Anupam & Srinivasan, Dipti, 2023. "Optimal bidding strategy for virtual power plant participating in combined electricity and ancillary services market considering dynamic demand response price and integrated consumption satisfaction," Energy, Elsevier, vol. 284(C).
    7. Zhou, Wenbing & Chi, Yuanying & Tang, Songlin & Hu, Yu & Wang, Zhengzao & Zhang, Xufeng, 2024. "The evolution analysis of low-carbon power transition strategies and carbon emission decoupling based on carbon neutrality target," Energy, Elsevier, vol. 311(C).
    8. Gengsheng He & Yu Huang & Guori Huang & Xi Liu & Pei Li & Yan Zhang, 2024. "Assessment of Low-Carbon Flexibility in Self-Organized Virtual Power Plants Using Multi-Agent Reinforcement Learning," Energies, MDPI, vol. 17(15), pages 1-20, July.
    9. Dong, Lianxin & Fan, Shuai & Wang, Zhihua & Xiao, Jucheng & Zhou, Huan & Li, Zuyi & He, Guangyu, 2021. "An adaptive decentralized economic dispatch method for virtual power plant," Applied Energy, Elsevier, vol. 300(C).
    10. Ju, Liwei & Yin, Zhe & Lu, Xiaolong & Yang, Shenbo & Li, Peng & Rao, Rao & Tan, Zhongfu, 2022. "A Tri-dimensional Equilibrium-based stochastic optimal dispatching model for a novel virtual power plant incorporating carbon Capture, Power-to-Gas and electric vehicle aggregator," Applied Energy, Elsevier, vol. 324(C).
    11. Chen, Siqi & Zhang, Kuan & Liu, Nian & Xie, Yawen, 2024. "Unlock the aggregated flexibility of electricity-hydrogen integrated virtual power plant for peak-regulation," Applied Energy, Elsevier, vol. 360(C).
    12. Wafa Nafkha-Tayari & Seifeddine Ben Elghali & Ehsan Heydarian-Forushani & Mohamed Benbouzid, 2022. "Virtual Power Plants Optimization Issue: A Comprehensive Review on Methods, Solutions, and Prospects," Energies, MDPI, vol. 15(10), pages 1-20, May.
    13. Dong, Lianxin & Wu, Qing & Hong, Juhua & Wang, Zhihua & Fan, Shuai & He, Guangyu, 2023. "An adaptive decentralized regulation strategy for the cluster with massive inverter air conditionings," Applied Energy, Elsevier, vol. 330(PA).
    14. Esfahani, Moein & Alizadeh, Ali & Amjady, Nima & Kamwa, Innocent, 2024. "A distributed VPP-integrated co-optimization framework for energy scheduling, frequency regulation, and voltage support using data-driven distributionally robust optimization with Wasserstein metric," Applied Energy, Elsevier, vol. 361(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wafa Nafkha-Tayari & Seifeddine Ben Elghali & Ehsan Heydarian-Forushani & Mohamed Benbouzid, 2022. "Virtual Power Plants Optimization Issue: A Comprehensive Review on Methods, Solutions, and Prospects," Energies, MDPI, vol. 15(10), pages 1-20, May.
    2. Xiong, Chang & Su, Yixin & Wang, Hao & Dong, Zhengcheng & Tian, Meng & Shi, Binghua, 2024. "Consensus-based decentralized scheduling method for collaborative operation in seaport virtual power plant," Applied Energy, Elsevier, vol. 373(C).
    3. Tsaousoglou, Georgios & Giraldo, Juan S. & Paterakis, Nikolaos G., 2022. "Market Mechanisms for Local Electricity Markets: A review of models, solution concepts and algorithmic techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    4. Benedikt Finnah, 2022. "Optimal bidding functions for renewable energies in sequential electricity markets," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 44(1), pages 1-27, March.
    5. Park, Keonwoo & Moon, Ilkyeong, 2022. "Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid," Applied Energy, Elsevier, vol. 328(C).
    6. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    7. Ma, Siyu & Liu, Hui & Wang, Ni & Huang, Lidong & Goh, Hui Hwang, 2023. "Incentive-based demand response under incomplete information based on the deep deterministic policy gradient," Applied Energy, Elsevier, vol. 351(C).
    8. Jiang, Yuzheng & Dong, Jun & Huang, Hexiang, 2024. "Optimal bidding strategy for the price-maker virtual power plant in the day-ahead market based on multi-agent twin delayed deep deterministic policy gradient algorithm," Energy, Elsevier, vol. 306(C).
    9. Paraskevas Koukaras & Paschalis Gkaidatzis & Napoleon Bezas & Tommaso Bragatto & Federico Carere & Francesca Santori & Marcel Antal & Dimosthenis Ioannidis & Christos Tjortjis & Dimitrios Tzovaras, 2021. "A Tri-Layer Optimization Framework for Day-Ahead Energy Scheduling Based on Cost and Discomfort Minimization," Energies, MDPI, vol. 14(12), pages 1-24, June.
    10. Mohseni, Soheil & Brent, Alan C. & Kelly, Scott & Browne, Will N. & Burmester, Daniel, 2021. "Strategic design optimisation of multi-energy-storage-technology micro-grids considering a two-stage game-theoretic market for demand response aggregation," Applied Energy, Elsevier, vol. 287(C).
    11. Kobashi, Takuro & Choi, Younghun & Hirano, Yujiro & Yamagata, Yoshiki & Say, Kelvin, 2022. "Rapid rise of decarbonization potentials of photovoltaics plus electric vehicles in residential houses over commercial districts," Applied Energy, Elsevier, vol. 306(PB).
    12. Younes Zahraoui & Tarmo Korõtko & Argo Rosin & Hannes Agabus, 2023. "Market Mechanisms and Trading in Microgrid Local Electricity Markets: A Comprehensive Review," Energies, MDPI, vol. 16(5), pages 1-52, February.
    13. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    14. Catra Indra Cahyadi & Suwarno Suwarno & Aminah Asmara Dewi & Musri Kona & Muhammad Arif & Muhammad Caesar Akbar, 2023. "Solar Prediction Strategy for Managing Virtual Power Stations," International Journal of Energy Economics and Policy, Econjournals, vol. 13(4), pages 503-512, July.
    15. Zeng, Yu & Wei, Xuan & Yao, Yuan & Xu, Yinliang & Sun, Hongbin & Kin Victor Chan, Wai & Feng, Wei, 2023. "Determining the pricing and deployment strategy for virtual power plants of peer-to-peer prosumers: A game-theoretic approach," Applied Energy, Elsevier, vol. 345(C).
    16. Qi, Chunyang & Song, Chuanxue & Xiao, Feng & Song, Shixin, 2022. "Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method," Energy, Elsevier, vol. 250(C).
    17. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    18. Naval, Natalia & Yusta, Jose M., 2021. "Virtual power plant models and electricity markets - A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    19. Zheng, Shunlin & Sun, Yi & Li, Bin & Qi, Bing & Zhang, Xudong & Li, Fei, 2021. "Incentive-based integrated demand response for multiple energy carriers under complex uncertainties and double coupling effects," Applied Energy, Elsevier, vol. 283(C).
    20. Xie, Haonan & Ahmad, Tanveer & Zhang, Dongdong & Goh, Hui Hwang & Wu, Thomas, 2024. "Community-based virtual power plants’ technology and circular economy models in the energy sector: A Techno-economy study," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317876. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.