IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i14p4131-d591038.html
   My bibliography  Save this article

Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market

Author

Listed:
  • Hannie Zang

    (School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Korea)

  • JongWon Kim

    (AI Graduate School, Gwangju Institute of Science and Technology (GIST), Gwangju 61005, Korea)

Abstract

Many studies have proposed a peer-to-peer energy market where the prosumers’ actions, including energy consumption, charge and discharge schedule of energy storage systems, and transactions in local energy markets, are controlled by a central operator. In this paper, prosumers’ actions are not controlled by an operator, and the prosumers freely participate in the local energy market to trade energy with other prosumers. We designed and modeled a local energy market with a management algorithm that uses community energy storage for prosumers who competitively participate in trade in the real-time energy market. We propose an energy-trade management algorithm that manages the trades of prosumers in two phases based on bids and offers submitted by prosumers. The first phase is to manage the trade of prosumers who have submitted fair prices to trade with other prosumers in the real-time energy market. The second phase is managing the trade of prosumers that could not trade in the first phase. Community energy storage is employed in the second phase and controlled by a reinforcement learning-based trading algorithm to decide whether to buy, sell, or do nothing with the prosumers. The action of buying and selling means charging and discharging the community energy storage, respectively. Numerical results show that the proposed trading algorithm gains a near-maximum profit. Besides, we verified that community energy storage yields more profit than the battery wear-out cost.

Suggested Citation

  • Hannie Zang & JongWon Kim, 2021. "Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market," Energies, MDPI, vol. 14(14), pages 1-18, July.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:14:p:4131-:d:591038
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/14/4131/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/14/4131/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Akter, M.N. & Mahmud, M.A. & Haque, M.E. & Oo, Amanullah M.T., 2020. "An optimal distributed energy management scheme for solving transactive energy sharing problems in residential microgrids," Applied Energy, Elsevier, vol. 270(C).
    2. Umer, Khalid & Huang, Qi & Khorasany, Mohsen & Afzal, Muhammad & Amin, Waqas, 2021. "A novel communication efficient peer-to-peer energy trading scheme for enhanced privacy in microgrids," Applied Energy, Elsevier, vol. 296(C).
    3. Jin-Gyeom Kim & Bowon Lee, 2020. "Automatic P2P Energy Trading Model Based on Reinforcement Learning Using Long Short-Term Delayed Reward," Energies, MDPI, vol. 13(20), pages 1-27, October.
    4. Bucciarelli, Martina & Paoletti, Simone & Vicino, Antonio, 2018. "Optimal sizing of energy storage systems under uncertain demand and generation," Applied Energy, Elsevier, vol. 225(C), pages 611-621.
    5. Roberts, Mike B. & Bruce, Anna & MacGill, Iain, 2019. "Impact of shared battery energy storage systems on photovoltaic self-consumption and electricity bills in apartment buildings," Applied Energy, Elsevier, vol. 245(C), pages 78-95.
    6. Koirala, Binod Prasad & van Oost, Ellen & van der Windt, Henny, 2018. "Community energy storage: A responsible innovation towards a sustainable energy system?," Applied Energy, Elsevier, vol. 231(C), pages 570-585.
    7. Yael Parag & Benjamin K. Sovacool, 2016. "Electricity market design for the prosumer era," Nature Energy, Nature, vol. 1(4), pages 1-6, April.
    8. Nikolaos Koltsaklis & Ioannis P. Panapakidis & David Pozo & Georgios C. Christoforidis, 2021. "A Prosumer Model Based on Smart Home Energy Management and Forecasting Techniques," Energies, MDPI, vol. 14(6), pages 1-32, March.
    9. Anvari-Moghaddam, Amjad & Rahimi-Kian, Ashkan & Mirian, Maryam S. & Guerrero, Josep M., 2017. "A multi-agent based energy management solution for integrated buildings and microgrid system," Applied Energy, Elsevier, vol. 203(C), pages 41-56.
    10. Jing, Rui & Xie, Mei Na & Wang, Feng Xiang & Chen, Long Xiang, 2020. "Fair P2P energy trading between residential and commercial multi-energy systems enabling integrated demand-side management," Applied Energy, Elsevier, vol. 262(C).
    11. Han, Sekyung & Han, Soohee & Aki, Hirohisa, 2014. "A practical battery wear model for electric vehicle charging applications," Applied Energy, Elsevier, vol. 113(C), pages 1100-1108.
    12. Lüth, Alexandra & Zepter, Jan Martin & Crespo del Granado, Pedro & Egging, Ruud, 2018. "Local electricity market designs for peer-to-peer trading: The role of battery flexibility," Applied Energy, Elsevier, vol. 229(C), pages 1233-1243.
    13. Zhang, Chenghua & Wu, Jianzhong & Zhou, Yue & Cheng, Meng & Long, Chao, 2018. "Peer-to-Peer energy trading in a Microgrid," Applied Energy, Elsevier, vol. 220(C), pages 1-12.
    14. Björn Nykvist & Måns Nilsson, 2015. "Rapidly falling costs of battery packs for electric vehicles," Nature Climate Change, Nature, vol. 5(4), pages 329-332, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shida Zhang & Petr Musilek, 2023. "The Impact of Battery Storage on Power Flow and Economy in an Automated Transactive Energy Market," Energies, MDPI, vol. 16(5), pages 1-13, February.
    2. Yuzhe Xie & Yan Yao & Yawu Wang & Weiqiang Cha & Sheng Zhou & Yue Wu & Chunyi Huang, 2022. "A Cooperative Game-Based Sizing and Configuration of Community-Shared Energy Storage," Energies, MDPI, vol. 15(22), pages 1-17, November.
    3. Mohammad Alipour & Rodney A. Stewart & Oz Sahin, 2021. "Beyond the Diffusion of Residential Solar Photovoltaic Systems at Scale: Allegorising the Battery Energy Storage Adoption Behaviour," Energies, MDPI, vol. 14(16), pages 1-12, August.
    4. Liangyi Pu & Song Wang & Xiaodong Huang & Xing Liu & Yawei Shi & Huiwei Wang, 2022. "Peer-to-Peer Trading for Energy-Saving Based on Reinforcement Learning," Energies, MDPI, vol. 15(24), pages 1-16, December.
    5. Peter Klement & Tobias Brandt & Lucas Schmeling & Antonieta Alcorta de Bronstein & Steffen Wehkamp & Fernando Andres Penaherrera Vaca & Mathias Lanezki & Patrik Schönfeldt & Alexander Hill & Nemanja K, 2022. "Local Energy Markets in Action: Smart Integration of National Markets, Distributed Energy Resources and Incentivisation to Promote Citizen Participation," Energies, MDPI, vol. 15(8), pages 1-24, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francesca Andreolli & Chiara D'Alpaos & Peter Kort, 2023. "Does P2P Trading Favor Investments in PV-Battery Systems?," Working Papers 2023.02, Fondazione Eni Enrico Mattei.
    2. Azim, M. Imran & Tushar, Wayes & Saha, Tapan K. & Yuen, Chau & Smith, David, 2022. "Peer-to-peer kilowatt and negawatt trading: A review of challenges and recent advances in distribution networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 169(C).
    3. López, Iraide & Goitia-Zabaleta, Nerea & Milo, Aitor & Gómez-Cornejo, Julen & Aranzabal, Itxaso & Gaztañaga, Haizea & Fernandez, Elvira, 2024. "European energy communities: Characteristics, trends, business models and legal framework," Renewable and Sustainable Energy Reviews, Elsevier, vol. 197(C).
    4. Meritxell Domènech Monfort & César De Jesús & Natapon Wanapinit & Niklas Hartmann, 2022. "A Review of Peer-to-Peer Energy Trading with Standard Terminology Proposal and a Techno-Economic Characterisation Matrix," Energies, MDPI, vol. 15(23), pages 1-29, November.
    5. Tushar, Wayes & Yuen, Chau & Saha, Tapan K. & Morstyn, Thomas & Chapman, Archie C. & Alam, M. Jan E. & Hanif, Sarmad & Poor, H. Vincent, 2021. "Peer-to-peer energy systems for connected communities: A review of recent advances and emerging challenges," Applied Energy, Elsevier, vol. 282(PA).
    6. Ableitner, Liliane & Tiefenbeck, Verena & Meeuw, Arne & Wörner, Anselma & Fleisch, Elgar & Wortmann, Felix, 2020. "User behavior in a real-world peer-to-peer electricity market," Applied Energy, Elsevier, vol. 270(C).
    7. Zhou, Yuekuan & Lund, Peter D., 2023. "Peer-to-peer energy sharing and trading of renewable energy in smart communities ─ trading pricing models, decision-making and agent-based collaboration," Renewable Energy, Elsevier, vol. 207(C), pages 177-193.
    8. Zare, Amir & Mehdinejad, Mehdi & Abedi, Mehrdad, 2024. "Designing a decentralized peer-to-peer energy market for an active distribution network considering loss and transaction fee allocation, and fairness," Applied Energy, Elsevier, vol. 358(C).
    9. García-Muñoz, Fernando & Dávila, Sebastián & Quezada, Franco, 2023. "A Benders decomposition approach for solving a two-stage local energy market problem under uncertainty," Applied Energy, Elsevier, vol. 329(C).
    10. Capper, Timothy & Gorbatcheva, Anna & Mustafa, Mustafa A. & Bahloul, Mohamed & Schwidtal, Jan Marc & Chitchyan, Ruzanna & Andoni, Merlinda & Robu, Valentin & Montakhabi, Mehdi & Scott, Ian J. & Franci, 2022. "Peer-to-peer, community self-consumption, and transactive energy: A systematic literature review of local energy market models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 162(C).
    11. Siripha Junlakarn & Phimsupha Kokchang & Kulyos Audomvongseree, 2022. "Drivers and Challenges of Peer-to-Peer Energy Trading Development in Thailand," Energies, MDPI, vol. 15(3), pages 1-25, February.
    12. Filipe Bandeiras & Álvaro Gomes & Mário Gomes & Paulo Coelho, 2023. "Exploring Energy Trading Markets in Smart Grid and Microgrid Systems and Their Implications for Sustainability in Smart Cities," Energies, MDPI, vol. 16(2), pages 1-41, January.
    13. Nizami, Sohrab & Tushar, Wayes & Hossain, M.J. & Yuen, Chau & Saha, Tapan & Poor, H. Vincent, 2022. "Transactive energy for low voltage residential networks: A review," Applied Energy, Elsevier, vol. 323(C).
    14. Karami, Mahdi & Madlener, Reinhard, 2022. "Business models for peer-to-peer energy trading in Germany based on households’ beliefs and preferences," Applied Energy, Elsevier, vol. 306(PB).
    15. Qiu, Dawei & Ye, Yujian & Papadaskalopoulos, Dimitrios & Strbac, Goran, 2021. "Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach," Applied Energy, Elsevier, vol. 292(C).
    16. Lüth, Alexandra & Zepter, Jan Martin & Crespo del Granado, Pedro & Egging, Ruud, 2018. "Local electricity market designs for peer-to-peer trading: The role of battery flexibility," Applied Energy, Elsevier, vol. 229(C), pages 1233-1243.
    17. Javier Parra-Domínguez & Esteban Sánchez & Ángel Ordóñez, 2023. "The Prosumer: A Systematic Review of the New Paradigm in Energy and Sustainable Development," Sustainability, MDPI, vol. 15(13), pages 1-44, July.
    18. Schwidtal, J.M. & Piccini, P. & Troncia, M. & Chitchyan, R. & Montakhabi, M. & Francis, C. & Gorbatcheva, A. & Capper, T. & Mustafa, M.A. & Andoni, M. & Robu, V. & Bahloul, M. & Scott, I.J. & Mbavarir, 2023. "Emerging business models in local energy markets: A systematic review of peer-to-peer, community self-consumption, and transactive energy models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 179(C).
    19. Faia, Ricardo & Lezama, Fernando & Soares, João & Pinto, Tiago & Vale, Zita, 2024. "Local electricity markets: A review on benefits, barriers, current trends and future perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 190(PA).
    20. Rodrigues, Stefane Dias & Garcia, Vinicius Jacques, 2023. "Transactive energy in microgrid communities: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 171(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:14:p:4131-:d:591038. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.