IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i3p1466-d1055062.html
   My bibliography  Save this article

Reinforcement Learning-Based Pricing and Incentive Strategy for Demand Response in Smart Grids

Author

Listed:
  • Eduardo J. Salazar

    (Doctoral Program in Electrical Engineering, Institute of Electrical Energy (IEE), National University of San Juan (UNSJ), National Scientific and Technical Research Council (CONICET), Libertador General San Martin Avenue 1109, San Juan 5400, Argentina)

  • Mauro Jurado

    (Doctoral Program in Electrical Engineering, Institute of Electrical Energy (IEE), National University of San Juan (UNSJ), National Scientific and Technical Research Council (CONICET), Libertador General San Martin Avenue 1109, San Juan 5400, Argentina)

  • Mauricio E. Samper

    (Doctoral Program in Electrical Engineering, Institute of Electrical Energy (IEE), National University of San Juan (UNSJ), National Scientific and Technical Research Council (CONICET), Libertador General San Martin Avenue 1109, San Juan 5400, Argentina)

Abstract

International agreements support the modernization of electricity networks and renewable energy resources (RES). However, these RES affect market prices due to resource variability (e.g., solar). Among the alternatives, Demand Response (DR) is presented as a tool to improve the balance between electricity supply and demand by adapting consumption to available production. In this sense, this work focuses on developing a DR model that combines price and incentive-based demand response models (P-B and I-B) to efficiently manage consumer demand with data from a real San Juan—Argentina distribution network. In addition, a price scheme is proposed in real time and by the time of use in relation to the consumers’ influence in the peak demand of the system. The proposed schemes increase load factor and improve demand displacement compared to a demand response reference model. In addition, the proposed reinforcement learning model improves short-term and long-term price search. Finally, a description and formulation of the market where the work was implemented is presented.

Suggested Citation

  • Eduardo J. Salazar & Mauro Jurado & Mauricio E. Samper, 2023. "Reinforcement Learning-Based Pricing and Incentive Strategy for Demand Response in Smart Grids," Energies, MDPI, vol. 16(3), pages 1-33, February.
    • Handle: RePEc:gam:jeners:v:16:y:2023:i:3:p:1466-:d:1055062
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/3/1466/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/3/1466/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zeng, Lanting & Qiu, Dawei & Sun, Mingyang, 2022. "Resilience enhancement of multi-agent reinforcement learning-based demand response against adversarial attacks," Applied Energy, Elsevier, vol. 324(C).
    2. Zhong, Shengyuan & Wang, Xiaoyuan & Zhao, Jun & Li, Wenjia & Li, Hao & Wang, Yongzhen & Deng, Shuai & Zhu, Jiebei, 2021. "Deep reinforcement learning framework for dynamic pricing demand response of regenerative electric heating," Applied Energy, Elsevier, vol. 288(C).
    3. Sonsaard, Parinya & Ketjoy, Nipon & Mensin, Yodthong, 2023. "Market strategy options to implement Thailand demand response program policy," Energy Policy, Elsevier, vol. 173(C).
    4. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    5. Jerzy Andruszkiewicz & Józef Lorenc & Agnieszka Weychan, 2019. "Demand Price Elasticity of Residential Electricity Consumers with Zonal Tariff Settlement Based on Their Load Profiles," Energies, MDPI, vol. 12(22), pages 1-22, November.
    6. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    7. Anjo, João & Neves, Diana & Silva, Carlos & Shivakumar, Abhishek & Howells, Mark, 2018. "Modeling the long-term impact of demand response in energy planning: The Portuguese electric system case study," Energy, Elsevier, vol. 165(PA), pages 456-468.
    8. Davide Deltetto & Davide Coraci & Giuseppe Pinto & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Exploring the Potentialities of Deep Reinforcement Learning for Incentive-Based Demand Response in a Cluster of Small Commercial Buildings," Energies, MDPI, vol. 14(10), pages 1-25, May.
    9. Liu, Yingqi, 2017. "Demand response and energy efficiency in the capacity resource procurement: Case studies of forward capacity markets in ISO New England, PJM and Great Britain," Energy Policy, Elsevier, vol. 100(C), pages 271-282.
    10. Parrish, Bryony & Heptonstall, Phil & Gross, Rob & Sovacool, Benjamin K., 2020. "A systematic review of motivations, enablers and barriers for consumer engagement with residential demand response," Energy Policy, Elsevier, vol. 138(C).
    11. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    12. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    13. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    14. Yu, Mengmeng & Hong, Seung Ho, 2016. "Supply–demand balancing for power management in smart grid: A Stackelberg game approach," Applied Energy, Elsevier, vol. 164(C), pages 702-710.
    15. Daiva Stanelyte & Neringa Radziukyniene & Virginijus Radziukynas, 2022. "Overview of Demand-Response Services: A Review," Energies, MDPI, vol. 15(5), pages 1-31, February.
    16. Fleschutz, Markus & Bohlayer, Markus & Braun, Marco & Henze, Gregor & Murphy, Michael D., 2021. "The effect of price-based demand response on carbon emissions in European electricity markets: The importance of adequate carbon prices," Applied Energy, Elsevier, vol. 295(C).
    17. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    18. Kong, Xiangyu & Kong, Deqian & Yao, Jingtao & Bai, Linquan & Xiao, Jie, 2020. "Online pricing of demand response based on long short-term memory and reinforcement learning," Applied Energy, Elsevier, vol. 271(C).
    19. Guo, Peiyang & Li, Victor O.K. & Lam, Jacqueline C.K., 2017. "Smart demand response in China: Challenges and drivers," Energy Policy, Elsevier, vol. 107(C), pages 1-10.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. António Gomes Martins & Luís Pires Neves & José Luís Sousa, 2023. "Electricity Demand Side Management," Energies, MDPI, vol. 16(16), pages 1-3, August.
    2. Mauro Jurado & Eduardo Salazar & Mauricio Samper & Rodolfo Rosés & Diego Ojeda Esteybar, 2023. "Day-Ahead Operational Planning for DisCos Based on Demand Response Flexibility and Volt/Var Control," Energies, MDPI, vol. 16(20), pages 1-20, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Xie, Jiahan & Ajagekar, Akshay & You, Fengqi, 2023. "Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings," Applied Energy, Elsevier, vol. 342(C).
    3. Zeng, Huibin & Shao, Bilin & Dai, Hongbin & Tian, Ning & Zhao, Wei, 2023. "Incentive-based demand response strategies for natural gas considering carbon emissions and load volatility," Applied Energy, Elsevier, vol. 348(C).
    4. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    5. Sabarathinam Srinivasan & Suresh Kumarasamy & Zacharias E. Andreadakis & Pedro G. Lind, 2023. "Artificial Intelligence and Mathematical Models of Power Grids Driven by Renewable Energy Sources: A Survey," Energies, MDPI, vol. 16(14), pages 1-56, July.
    6. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    7. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    8. Pinto, Giuseppe & Kathirgamanathan, Anjukan & Mangina, Eleni & Finn, Donal P. & Capozzoli, Alfonso, 2022. "Enhancing energy management in grid-interactive buildings: A comparison among cooperative and coordinated architectures," Applied Energy, Elsevier, vol. 310(C).
    9. Ma, Siyu & Liu, Hui & Wang, Ni & Huang, Lidong & Goh, Hui Hwang, 2023. "Incentive-based demand response under incomplete information based on the deep deterministic policy gradient," Applied Energy, Elsevier, vol. 351(C).
    10. Zhou, Kaile & Peng, Ning & Yin, Hui & Hu, Rong, 2023. "Urban virtual power plant operation optimization with incentive-based demand response," Energy, Elsevier, vol. 282(C).
    11. Zhang, Xiongfeng & Lu, Renzhi & Jiang, Junhui & Hong, Seung Ho & Song, Won Seok, 2021. "Testbed implementation of reinforcement learning-based demand response energy management system," Applied Energy, Elsevier, vol. 297(C).
    12. Barja-Martinez, Sara & Aragüés-Peñalba, Mònica & Munné-Collado, Íngrid & Lloret-Gallego, Pau & Bullich-Massagué, Eduard & Villafafila-Robles, Roberto, 2021. "Artificial intelligence techniques for enabling Big Data services in distribution networks: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    13. Mahdi Khodayar & Jacob Regan, 2023. "Deep Neural Networks in Power Systems: A Review," Energies, MDPI, vol. 16(12), pages 1-38, June.
    14. Hui Wang & Yao Xu, 2024. "Optimized Decision-Making for Multi-Market Green Power Transactions of Electricity Retailers under Demand-Side Response: The Chinese Market Case Study," Energies, MDPI, vol. 17(11), pages 1-16, May.
    15. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    16. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    17. Lin, Jin & Dong, Jun & Liu, Dongran & Zhang, Yaoyu & Ma, Tongtao, 2022. "From peak shedding to low-carbon transitions: Customer psychological factors in demand response," Energy, Elsevier, vol. 238(PA).
    18. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    19. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    20. Ajagekar, Akshay & Decardi-Nelson, Benjamin & You, Fengqi, 2024. "Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 355(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:3:p:1466-:d:1055062. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.