IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i4p1440-d750803.html
   My bibliography  Save this article

Distributed Reinforcement Learning for the Management of a Smart Grid Interconnecting Independent Prosumers

Author

Listed:
  • Dominique Barth

    (DAVID Laboratory, UVSQ/Université Paris-Saclay, 45 Avenue des Etats Unis, 78035 Versailles, France
    These authors contributed equally to this work.)

  • Benjamin Cohen-Boulakia

    (LINEACT, CESI, 92000 Nanterre, France
    These authors contributed equally to this work.)

  • Wilfried Ehounou

    (LINEACT, CESI, 92000 Nanterre, France
    Laboratoire de Mathématiques Informatique, Université Nangui Abrogoua, Abidjan 02 BP V 102, Côte d’Ivoire
    These authors contributed equally to this work.)

Abstract

In the context of an eco-responsible production and distribution of electrical energy at the local scale of an urban territory, we consider a smart grid as a system interconnecting different prosumers, which all retain their decision-making autonomy and defend their own interests in a comprehensive system where the rules, accepted by all, encourage virtuous behavior. In this paper, we present and analyze a model and a management method for smart grids that is shared between different kinds of independent actors, who respect their own interests, and that encourages each actor to behavior that allows, as much as possible, an energy independence of the smart grid from external energy suppliers. We consider here a game theory model, in which each actor of the smart grid is a player, and we investigate distributed machine-learning algorithms to allow decision-making, thus, leading the game to converge to stable situations, in particular to a Nash equilibrium. We propose a Linear Reward Inaction algorithm that achieves Nash equilibria most of the time, both for a single time slot and across time, allowing the smart grid to maximize its energy independence from external energy suppliers.

Suggested Citation

  • Dominique Barth & Benjamin Cohen-Boulakia & Wilfried Ehounou, 2022. "Distributed Reinforcement Learning for the Management of a Smart Grid Interconnecting Independent Prosumers," Energies, MDPI, vol. 15(4), pages 1-19, February.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:4:p:1440-:d:750803
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/4/1440/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/4/1440/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chien, Steve & Sinclair, Alistair, 2011. "Convergence to approximate Nash equilibria in congestion games," Games and Economic Behavior, Elsevier, vol. 71(2), pages 315-327, March.
    2. Milchtaich, Igal, 1996. "Congestion Games with Player-Specific Payoff Functions," Games and Economic Behavior, Elsevier, vol. 13(1), pages 111-124, March.
    3. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    4. Ri Piao & Deok-Joo Lee & Taegu Kim, 2020. "Real-Time Pricing Scheme in Smart Grid Considering Time Preference: Game Theoretic Approach," Energies, MDPI, vol. 13(22), pages 1-19, November.
    5. Iver Bakken Sperstad & Magnus Korpås, 2019. "Energy Storage Scheduling in Distribution Systems Considering Wind and Photovoltaic Generation Uncertainties," Energies, MDPI, vol. 12(7), pages 1-24, March.
    6. Mohamadou Nassourou & Joaquim Blesa & Vicenç Puig, 2020. "Robust Economic Model Predictive Control Based on a Zonotope and Local Feedback Controller for Energy Dispatch in Smart-Grids Considering Demand Uncertainty," Energies, MDPI, vol. 13(3), pages 1-19, February.
    7. Sylvain Cros & Jordi Badosa & André Szantaï & Martial Haeffelin, 2020. "Reliability Predictors for Solar Irradiance Satellite-Based Forecast," Energies, MDPI, vol. 13(21), pages 1-21, October.
    8. Calvillo, C.F. & Sánchez-Miralles, A. & Villar, J., 2016. "Energy management and planning in smart cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 273-287.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Muhammad Majid Hussain & Rizwan Akram & Zulfiqar Ali Memon & Mian Hammad Nazir & Waqas Javed & Muhammad Siddique, 2021. "Demand Side Management Techniques for Home Energy Management Systems for Smart Cities," Sustainability, MDPI, vol. 13(21), pages 1-20, October.
    2. Matthias Feldotto & Lennart Leder & Alexander Skopalik, 2018. "Congestion games with mixed objectives," Journal of Combinatorial Optimization, Springer, vol. 36(4), pages 1145-1167, November.
    3. Maximilian Drees & Matthias Feldotto & Sören Riechers & Alexander Skopalik, 2019. "Pure Nash equilibria in restricted budget games," Journal of Combinatorial Optimization, Springer, vol. 37(2), pages 620-638, February.
    4. Daskalakis, Constantinos & Papadimitriou, Christos H., 2015. "Approximate Nash equilibria in anonymous games," Journal of Economic Theory, Elsevier, vol. 156(C), pages 207-245.
    5. Christian Ewerhart, 2020. "Ordinal potentials in smooth games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 70(4), pages 1069-1100, November.
    6. Biancardi, Marta & Di Bari, Antonio & Villani, Giovanni, 2021. "R&D investment decision on smart cities: Energy sustainability and opportunity," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    7. Tami Tamir, 2023. "Cost-sharing games in real-time scheduling systems," International Journal of Game Theory, Springer;Game Theory Society, vol. 52(1), pages 273-301, March.
    8. Tsoumalis, Georgios I. & Bampos, Zafeirios N. & Biskas, Pandelis N. & Keranidis, Stratos D. & Symeonidis, Polychronis A. & Voulgarakis, Dimitrios K., 2022. "A novel system for providing explicit demand response from domestic natural gas boilers," Applied Energy, Elsevier, vol. 317(C).
    9. Le Breton, Michel & Weber, Shlomo, 2009. "Existence of Pure Strategies Nash Equilibria in Social Interaction Games with Dyadic Externalities," CEPR Discussion Papers 7279, C.E.P.R. Discussion Papers.
    10. Zhang, Yang & Yang, Qingyu & Li, Donghe & An, Dou, 2022. "A reinforcement and imitation learning method for pricing strategy of electricity retailer with customers’ flexibility," Applied Energy, Elsevier, vol. 323(C).
    11. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    12. Bi, Huibo & Shang, Wen-Long & Chen, Yanyan & Wang, Kezhi & Yu, Qing & Sui, Yi, 2021. "GIS aided sustainable urban road management with a unifying queueing and neural network model," Applied Energy, Elsevier, vol. 291(C).
    13. Arnold, Tone & Wooders, Myrna, 2002. "Dynamic Club Formation with Coordination," Economic Research Papers 269414, University of Warwick - Department of Economics.
    14. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    15. Zhenghao Wang & Yonghui Liu & Zihao Yang & Wanhao Yang, 2021. "Load Frequency Control of Multi-Region Interconnected Power Systems with Wind Power and Electric Vehicles Based on Sliding Mode Control," Energies, MDPI, vol. 14(8), pages 1-15, April.
    16. Ryo Kawasaki & Hideo Konishi & Junki Yukawa, 2023. "Equilibria in bottleneck games," International Journal of Game Theory, Springer;Game Theory Society, vol. 52(3), pages 649-685, September.
    17. Hideo Konishi, 2004. "Uniqueness of User Equilibrium in Transportation Networks with Heterogeneous Commuters," Transportation Science, INFORMS, vol. 38(3), pages 315-330, August.
    18. Igal Milchtaich, 2000. "Generic Uniqueness of Equilibrium in Large Crowding Games," Mathematics of Operations Research, INFORMS, vol. 25(3), pages 349-364, August.
    19. Sellak, Hamza & Ouhbi, Brahim & Frikh, Bouchra & Palomares, Iván, 2017. "Towards next-generation energy planning decision-making: An expert-based framework for intelligent decision support," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 1544-1577.
    20. Yang, Zhen & Gao, Weijun & Han, Qing & Qi, Liyan, 2024. "Aggravating or alleviating? Smart city construction and urban inequality in China," Technology in Society, Elsevier, vol. 77(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:4:p:1440-:d:750803. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.