IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i21p7866-d951269.html
   My bibliography  Save this article

System Frequency Control Method Driven by Deep Reinforcement Learning and Customer Satisfaction for Thermostatically Controlled Load

Author

Listed:
  • Rusi Chen

    (State Grid Hubei Electric Power Research Institute, Wuhan 430077, China)

  • Haiguang Liu

    (State Grid Hubei Electric Power Research Institute, Wuhan 430077, China)

  • Chengquan Liu

    (Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China)

  • Guangzheng Yu

    (Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China)

  • Xuan Yang

    (State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China)

  • Yue Zhou

    (State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China)

Abstract

The intermittence and fluctuation of renewable energy aggravate the power fluctuation of the power grid and pose a severe challenge to the frequency stability of the power system. Thermostatically controlled loads can participate in the frequency regulation of the power grid due to their flexibility. Aiming to solve the problem of the traditional control methods, which have limited adjustment ability, and to have a positive influence on customers, a deep reinforcement learning control strategy based on the framework of soft actor–critic is proposed, considering customer satisfaction. Firstly, the energy storage index and the discomfort index of different users are defined. Secondly, the fuzzy comprehensive evaluation method is applied to evaluate customer satisfaction. Then, the multi-agent models of thermostatically controlled loads are established based on the soft actor–critic algorithm. The models are trained by using the local information of thermostatically controlled loads, and the comprehensive evaluation index fed back by users and the frequency deviation. After training, each agent can realize the cooperative response of thermostatically controlled loads to the system frequency only by relying on the local information. The simulation results show that the proposed strategy can not only reduce the frequency fluctuation, but also improve customer satisfaction.

Suggested Citation

  • Rusi Chen & Haiguang Liu & Chengquan Liu & Guangzheng Yu & Xuan Yang & Yue Zhou, 2022. "System Frequency Control Method Driven by Deep Reinforcement Learning and Customer Satisfaction for Thermostatically Controlled Load," Energies, MDPI, vol. 15(21), pages 1-19, October.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:21:p:7866-:d:951269
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/21/7866/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/21/7866/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zexu Chen & Jing Shi & Zhaofang Song & Wangwang Yang & Zitong Zhang, 2022. "Genetic Algorithm Based Temperature-Queuing Method for Aggregated IAC Load Control," Energies, MDPI, vol. 15(2), pages 1-16, January.
    2. Pallonetto, Fabiano & De Rosa, Mattia & Milano, Federico & Finn, Donal P., 2019. "Demand response algorithms for smart-grid ready residential buildings using machine learning models," Applied Energy, Elsevier, vol. 239(C), pages 1265-1282.
    3. Rongxiang Zhang & Xiaodong Chu & Wen Zhang & Yutian Liu, 2015. "Active Participation of Air Conditioners in Power System Frequency Control Considering Users’ Thermal Comfort," Energies, MDPI, vol. 8(10), pages 1-24, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nallapaneni Manoj Kumar & Aneesh A. Chand & Maria Malvoni & Kushal A. Prasad & Kabir A. Mamun & F.R. Islam & Shauhrat S. Chopra, 2020. "Distributed Energy Resources and the Application of AI, IoT, and Blockchain in Smart Grids," Energies, MDPI, vol. 13(21), pages 1-42, November.
    2. Yang, Shiyu & Oliver Gao, H. & You, Fengqi, 2022. "Model predictive control for Demand- and Market-Responsive building energy management by leveraging active latent heat storage," Applied Energy, Elsevier, vol. 327(C).
    3. Yang, Shiyu & Oliver Gao, H. & You, Fengqi, 2022. "Model predictive control in phase-change-material-wallboard-enhanced building energy management considering electricity price dynamics," Applied Energy, Elsevier, vol. 326(C).
    4. Oleh Lukianykhin & Tetiana Bogodorova, 2021. "Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning," Energies, MDPI, vol. 14(8), pages 1-22, April.
    5. Osaru Agbonaye & Patrick Keatley & Ye Huang & Motasem Bani Mustafa & Neil Hewitt, 2020. "Design, Valuation and Comparison of Demand Response Strategies for Congestion Management," Energies, MDPI, vol. 13(22), pages 1-29, November.
    6. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    7. Tetsushi Ono & Aya Hagishima & Jun Tanimoto, 2022. "Non-Intrusive Detection of Occupants’ On/Off Behaviours of Residential Air Conditioning," Sustainability, MDPI, vol. 14(22), pages 1-20, November.
    8. Yuchun Li & Yinghua Han & Jinkuan Wang & Qiang Zhao, 2018. "A MBCRF Algorithm Based on Ensemble Learning for Building Demand Response Considering the Thermal Comfort," Energies, MDPI, vol. 11(12), pages 1-20, December.
    9. Sabarathinam Srinivasan & Suresh Kumarasamy & Zacharias E. Andreadakis & Pedro G. Lind, 2023. "Artificial Intelligence and Mathematical Models of Power Grids Driven by Renewable Energy Sources: A Survey," Energies, MDPI, vol. 16(14), pages 1-56, July.
    10. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    11. Boza, Pal & Evgeniou, Theodoros, 2021. "Artificial intelligence to support the integration of variable renewable energy sources to the power system," Applied Energy, Elsevier, vol. 290(C).
    12. Ran, Fengming & Gao, Dian-ce & Zhang, Xu & Chen, Shuyue, 2020. "A virtual sensor based self-adjusting control for HVAC fast demand response in commercial buildings towards smart grid applications," Applied Energy, Elsevier, vol. 269(C).
    13. Máximo A. Domínguez-Garabitos & Víctor S. Ocaña-Guevara & Félix Santos-García & Adriana Arango-Manrique & Miguel Aybar-Mejía, 2022. "A Methodological Proposal for Implementing Demand-Shifting Strategies in the Wholesale Electricity Market," Energies, MDPI, vol. 15(4), pages 1-28, February.
    14. Madia Safdar & Ghulam Amjad Hussain & Matti Lehtonen, 2019. "Costs of Demand Response from Residential Customers’ Perspective," Energies, MDPI, vol. 12(9), pages 1-16, April.
    15. Lu, Qing & Zhang, Yufeng, 2022. "A multi-objective optimization model considering users' satisfaction and multi-type demand response in dynamic electricity price," Energy, Elsevier, vol. 240(C).
    16. Das, Laya & Garg, Dinesh & Srinivasan, Babji, 2020. "NeuralCompression: A machine learning approach to compress high frequency measurements in smart grid," Applied Energy, Elsevier, vol. 257(C).
    17. Siyue Lu & Baoqun Zhang & Longfei Ma & Hui Xu & Yuantong Li & Shaobing Yang, 2023. "Economic Load-Reduction Strategy of Central Air Conditioning Based on Convolutional Neural Network and Pre-Cooling," Energies, MDPI, vol. 16(13), pages 1-22, June.
    18. Lilia Tightiz & Joon Yoo, 2022. "A Review on a Data-Driven Microgrid Management System Integrating an Active Distribution Network: Challenges, Issues, and New Trends," Energies, MDPI, vol. 15(22), pages 1-24, November.
    19. Kanakadhurga, Dharmaraj & Prabaharan, Natarajan, 2022. "Demand side management in microgrid: A critical review of key issues and recent trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    20. Wang, Zeyu & Liu, Jian & Zhang, Yuanxin & Yuan, Hongping & Zhang, Ruixue & Srinivasan, Ravi S., 2021. "Practical issues in implementing machine-learning models for building energy efficiency: Moving beyond obstacles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 143(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:21:p:7866-:d:951269. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.