IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i5p1906-d764646.html
   My bibliography  Save this article

Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities

Author

Listed:
  • Pannee Suanpang

    (Faculty of Science and Technology, Suan Dusit University, Bangkok 10300, Thailand)

  • Pitchaya Jamjuntr

    (Computer Engineering Department, King Mongkut’s University of Technology Thonburi, Bangkok 10140, Thailand)

  • Kittisak Jermsittiparsert

    (Faculty of Education, University of City Island, 9945 Famagusta, Cyprus
    Faculty of Social and Political Sciences, Universitas Muhammadiyah Sinjai, Kabupaten Sinjai 92615, Sulawesi Selatan, Indonesia
    Faculty of Social and Political Sciences, Universitas Muhammadiyah Makassar, Kota Makassar 90221, Sulawesi Selatan, Indonesia
    Publication Research Institute and Community Service, Universitas Muhammadiyah Sidenreng Rappang, Sidenreng Rappang Regency 91651, South Sulawesi, Indonesia)

  • Phuripoj Kaewyong

    (Faculty of Science and Technology, Suan Dusit University, Bangkok 10300, Thailand)

Abstract

Autonomous energy management is becoming a significant mechanism for attaining sustainability in energy management. This resulted in this research paper, which aimed to apply deep reinforcement learning algorithms for an autonomous energy management system of a microgrid. This paper proposed a novel microgrid model that consisted of a combustion set of a household load, renewable energy, an energy storage system, and a generator, which were connected to the main grid. The proposed autonomous energy management system was designed to cooperate with the various flexible sources and loads by defining the priority resources, loads, and electricity prices. The system was implemented by using deep reinforcement learning algorithms that worked effectively in order to control the power storage, solar panels, generator, and main grid. The system model could achieve the optimal performance with near-optimal policies. As a result, this method could save 13.19% in the cost compared to conducting manual control of energy management. In this study, there was a focus on applying Q-learning for the microgrid in the tourism industry in remote areas which can produce and store energy. Therefore, we proposed an autonomous energy management system for effective energy management. In future work, the system could be improved by applying deep learning to use energy price data to predict the future energy price, when the system could produce more energy than the demand and store it for selling at the most appropriate price; this would make the autonomous energy management system smarter and provide better benefits for the tourism industry. This proposed autonomous energy management could be applied to other industries, for example businesses or factories which need effective energy management to maintain microgrid stability and also save energy.

Suggested Citation

  • Pannee Suanpang & Pitchaya Jamjuntr & Kittisak Jermsittiparsert & Phuripoj Kaewyong, 2022. "Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities," Energies, MDPI, vol. 15(5), pages 1-13, March.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1906-:d:764646
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/5/1906/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/5/1906/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Elyas Rakhshani & Kumars Rouzbehi & Adolfo J. Sánchez & Ana Cabrera Tobar & Edris Pouresmaeil, 2019. "Integration of Large Scale PV-Based Generation into Power Systems: A Survey," Energies, MDPI, vol. 12(8), pages 1-19, April.
    2. Hirsch, Adam & Parag, Yael & Guerrero, Josep, 2018. "Microgrids: A review of technologies, key drivers, and outstanding issues," Renewable and Sustainable Energy Reviews, Elsevier, vol. 90(C), pages 402-411.
    3. Kittisak Jermsittiparsert & Thitinan Chankoson, 2019. "Behavior of Tourism Industry under the Situation of Environmental Threats and Carbon Emission: Time Series Analysis from Thailand," International Journal of Energy Economics and Policy, Econjournals, vol. 9(6), pages 366-372.
    4. Kamlesh Kumar, 2020. "Social, Economic, and Environmental Impacts of Renewable Energy Resources," Chapters, in: Kenneth Eloghene Okedu & Ahmed Tahour & Abdel Ghani Aissaoui (ed.), Wind Solar Hybrid Renewable Energy System, IntechOpen.
    5. Huseyin Salvarli & Mustafa Seckin Salvarli, 2020. "For Sustainable Development: Future Trends in Renewable Energy and Enabling Technologies," Chapters, in: Mansour Al Qubeissi & Ahmad El-Kharouf & Hakan Serhad Soyhan (ed.), Renewable Energy - Resources, Challenges and Applications, IntechOpen.
    6. Alexander Lavrik & Yuri Zhukovskiy & Pavel Tcvetkov, 2021. "Optimizing the Size of Autonomous Hybrid Microgrids with Regard to Load Shifting," Energies, MDPI, vol. 14(16), pages 1-19, August.
    7. Piotr Maśloch & Grzegorz Maśloch & Łukasz Kuźmiński & Henryk Wojtaszek & Ireneusz Miciuła, 2020. "Autonomous Energy Regions as a Proposed Choice of Selecting Selected EU Regions—Aspects of Their Creation and Management," Energies, MDPI, vol. 13(23), pages 1-27, December.
    8. Li, Yunpeng & Hu, Clark & Huang, Chao & Duan, Liqiong, 2017. "The concept of smart tourism in the context of tourism information services," Tourism Management, Elsevier, vol. 58(C), pages 293-300.
    9. Calvillo, C.F. & Sánchez-Miralles, A. & Villar, J., 2016. "Energy management and planning in smart cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 273-287.
    10. Hongyan Ma, 2020. "The Construction Path and Mode of Public Tourism Information Service System Based on the Perspective of Smart City," Complexity, Hindawi, vol. 2020, pages 1-11, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.
    2. Pannee Suanpang & Pattanaphong Pothipassa & Kittisak Jermsittiparsert & Titiya Netwong, 2022. "Integration of Kouprey-Inspired Optimization Algorithms with Smart Energy Nodes for Sustainable Energy Management of Agricultural Orchards," Energies, MDPI, vol. 15(8), pages 1-18, April.
    3. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pannee Suanpang & Pattanaphong Pothipassa & Kittisak Jermsittiparsert & Titiya Netwong, 2022. "Integration of Kouprey-Inspired Optimization Algorithms with Smart Energy Nodes for Sustainable Energy Management of Agricultural Orchards," Energies, MDPI, vol. 15(8), pages 1-18, April.
    2. Ahl, Amanda & Yarime, Masaru & Tanaka, Kenji & Sagawa, Daishi, 2019. "Review of blockchain-based distributed energy: Implications for institutional development," Renewable and Sustainable Energy Reviews, Elsevier, vol. 107(C), pages 200-211.
    3. Yildizbasi, Abdullah, 2021. "Blockchain and renewable energy: Integration challenges in circular economy era," Renewable Energy, Elsevier, vol. 176(C), pages 183-197.
    4. Yoro, Kelvin O. & Daramola, Michael O. & Sekoai, Patrick T. & Wilson, Uwemedimo N. & Eterigho-Ikelegbe, Orevaoghene, 2021. "Update on current approaches, challenges, and prospects of modeling and simulation in renewable and sustainable energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    5. Farhat Afzah Samoon & Ikhlaq Hussain & Sheikh Javed Iqbal, 2023. "ILA Optimisation Based Control for Enhancing DC Link Voltage with Seamless and Adaptive VSC Control in a PV-BES Based AC Microgrid," Energies, MDPI, vol. 16(21), pages 1-23, October.
    6. Emrani-Rahaghi, Pouria & Hashemi-Dezaki, Hamed & Ketabi, Abbas, 2023. "Efficient voltage control of low voltage distribution networks using integrated optimized energy management of networked residential multi-energy microgrids," Applied Energy, Elsevier, vol. 349(C).
    7. Francesco Lo Franco & Mattia Ricco & Riccardo Mandrioli & Gabriele Grandi, 2020. "Electric Vehicle Aggregate Power Flow Prediction and Smart Charging System for Distributed Renewable Energy Self-Consumption Optimization," Energies, MDPI, vol. 13(19), pages 1-25, September.
    8. Dimitrios Trigkas & Chrysovalantou Ziogou & Spyros Voutetakis & Simira Papadopoulou, 2021. "Virtual Energy Storage in RES-Powered Smart Grids with Nonlinear Model Predictive Control," Energies, MDPI, vol. 14(4), pages 1-22, February.
    9. Jihed Hmad & Azeddine Houari & Allal El Moubarek Bouzid & Abdelhakim Saim & Hafedh Trabelsi, 2023. "A Review on Mode Transition Strategies between Grid-Connected and Standalone Operation of Voltage Source Inverters-Based Microgrids," Energies, MDPI, vol. 16(13), pages 1-41, June.
    10. Biancardi, Marta & Di Bari, Antonio & Villani, Giovanni, 2021. "R&D investment decision on smart cities: Energy sustainability and opportunity," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    11. Matija Kostelac & Lin Herenčić & Tomislav Capuder, 2022. "Planning and Operational Aspects of Individual and Clustered Multi-Energy Microgrid Options," Energies, MDPI, vol. 15(4), pages 1-17, February.
    12. Terlouw, Tom & AlSkaif, Tarek & Bauer, Christian & van Sark, Wilfried, 2019. "Optimal energy management in all-electric residential energy systems with heat and electricity storage," Applied Energy, Elsevier, vol. 254(C).
    13. Soheil Mohseni & Alan C. Brent & Daniel Burmester, 2020. "Community Resilience-Oriented Optimal Micro-Grid Capacity Expansion Planning: The Case of Totarabank Eco-Village, New Zealand," Energies, MDPI, vol. 13(15), pages 1-29, August.
    14. Roth, Jonathan & Martin, Amory & Miller, Clayton & Jain, Rishee K., 2020. "SynCity: Using open data to create a synthetic city of hourly building energy estimates by integrating data-driven and physics-based methods," Applied Energy, Elsevier, vol. 280(C).
    15. Hammad Alnuman & Kuo-Hsien Hsia & Mohammadreza Askari Sepestanaki & Emad M. Ahmed & Saleh Mobayen & Ammar Armghan, 2023. "Design of Continuous Finite-Time Controller Based on Adaptive Tuning Approach for Disturbed Boost Converters," Mathematics, MDPI, vol. 11(7), pages 1-23, April.
    16. Giulietti, Monica & Le Coq, Chloé & Willems, Bert & Anaya, Karim, 2019. "Smart Consumers in the Internet of Energy : Flexibility Markets & Services from Distributed Energy Resources," Other publications TiSEM 2edb43b5-bbd6-487d-abdf-7, Tilburg University, School of Economics and Management.
    17. Iwona Bąk & Anna Spoz & Magdalena Zioło & Marek Dylewski, 2021. "Dynamic Analysis of the Similarity of Objects in Research on the Use of Renewable Energy Resources in European Union Countries," Energies, MDPI, vol. 14(13), pages 1-24, July.
    18. Polleux, Louis & Guerassimoff, Gilles & Marmorat, Jean-Paul & Sandoval-Moreno, John & Schuhler, Thierry, 2022. "An overview of the challenges of solar power integration in isolated industrial microgrids with reliability constraints," Renewable and Sustainable Energy Reviews, Elsevier, vol. 155(C).
    19. Yang, Zhen & Gao, Weijun & Han, Qing & Qi, Liyan, 2024. "Aggravating or alleviating? Smart city construction and urban inequality in China," Technology in Society, Elsevier, vol. 77(C).
    20. Guodong Liu & Maximiliano F. Ferrari & Thomas B. Ollis & Kevin Tomsovic, 2022. "An MILP-Based Distributed Energy Management for Coordination of Networked Microgrids," Energies, MDPI, vol. 15(19), pages 1-20, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1906-:d:764646. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.