IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i5p1906-d764646.html
   My bibliography  Save this article

Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities

Author

Listed:
  • Pannee Suanpang

    (Faculty of Science and Technology, Suan Dusit University, Bangkok 10300, Thailand)

  • Pitchaya Jamjuntr

    (Computer Engineering Department, King Mongkut’s University of Technology Thonburi, Bangkok 10140, Thailand)

  • Kittisak Jermsittiparsert

    (Faculty of Education, University of City Island, 9945 Famagusta, Cyprus
    Faculty of Social and Political Sciences, Universitas Muhammadiyah Sinjai, Kabupaten Sinjai 92615, Sulawesi Selatan, Indonesia
    Faculty of Social and Political Sciences, Universitas Muhammadiyah Makassar, Kota Makassar 90221, Sulawesi Selatan, Indonesia
    Publication Research Institute and Community Service, Universitas Muhammadiyah Sidenreng Rappang, Sidenreng Rappang Regency 91651, South Sulawesi, Indonesia)

  • Phuripoj Kaewyong

    (Faculty of Science and Technology, Suan Dusit University, Bangkok 10300, Thailand)

Abstract

Autonomous energy management is becoming a significant mechanism for attaining sustainability in energy management. This resulted in this research paper, which aimed to apply deep reinforcement learning algorithms for an autonomous energy management system of a microgrid. This paper proposed a novel microgrid model that consisted of a combustion set of a household load, renewable energy, an energy storage system, and a generator, which were connected to the main grid. The proposed autonomous energy management system was designed to cooperate with the various flexible sources and loads by defining the priority resources, loads, and electricity prices. The system was implemented by using deep reinforcement learning algorithms that worked effectively in order to control the power storage, solar panels, generator, and main grid. The system model could achieve the optimal performance with near-optimal policies. As a result, this method could save 13.19% in the cost compared to conducting manual control of energy management. In this study, there was a focus on applying Q-learning for the microgrid in the tourism industry in remote areas which can produce and store energy. Therefore, we proposed an autonomous energy management system for effective energy management. In future work, the system could be improved by applying deep learning to use energy price data to predict the future energy price, when the system could produce more energy than the demand and store it for selling at the most appropriate price; this would make the autonomous energy management system smarter and provide better benefits for the tourism industry. This proposed autonomous energy management could be applied to other industries, for example businesses or factories which need effective energy management to maintain microgrid stability and also save energy.

Suggested Citation

  • Pannee Suanpang & Pitchaya Jamjuntr & Kittisak Jermsittiparsert & Phuripoj Kaewyong, 2022. "Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities," Energies, MDPI, vol. 15(5), pages 1-13, March.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1906-:d:764646
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/5/1906/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/5/1906/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Elyas Rakhshani & Kumars Rouzbehi & Adolfo J. Sánchez & Ana Cabrera Tobar & Edris Pouresmaeil, 2019. "Integration of Large Scale PV-Based Generation into Power Systems: A Survey," Energies, MDPI, vol. 12(8), pages 1-19, April.
    2. Hirsch, Adam & Parag, Yael & Guerrero, Josep, 2018. "Microgrids: A review of technologies, key drivers, and outstanding issues," Renewable and Sustainable Energy Reviews, Elsevier, vol. 90(C), pages 402-411.
    3. Kittisak Jermsittiparsert & Thitinan Chankoson, 2019. "Behavior of Tourism Industry under the Situation of Environmental Threats and Carbon Emission: Time Series Analysis from Thailand," International Journal of Energy Economics and Policy, Econjournals, vol. 9(6), pages 366-372.
    4. Kamlesh Kumar, 2020. "Social, Economic, and Environmental Impacts of Renewable Energy Resources," Chapters, in: Kenneth Eloghene Okedu & Ahmed Tahour & Abdel Ghani Aissaoui (ed.), Wind Solar Hybrid Renewable Energy System, IntechOpen.
    5. Huseyin Salvarli & Mustafa Seckin Salvarli, 2020. "For Sustainable Development: Future Trends in Renewable Energy and Enabling Technologies," Chapters, in: Mansour Al Qubeissi & Ahmad El-Kharouf & Hakan Serhad Soyhan (ed.), Renewable Energy - Resources, Challenges and Applications, IntechOpen.
    6. Alexander Lavrik & Yuri Zhukovskiy & Pavel Tcvetkov, 2021. "Optimizing the Size of Autonomous Hybrid Microgrids with Regard to Load Shifting," Energies, MDPI, vol. 14(16), pages 1-19, August.
    7. Piotr Maśloch & Grzegorz Maśloch & Łukasz Kuźmiński & Henryk Wojtaszek & Ireneusz Miciuła, 2020. "Autonomous Energy Regions as a Proposed Choice of Selecting Selected EU Regions—Aspects of Their Creation and Management," Energies, MDPI, vol. 13(23), pages 1-27, December.
    8. Li, Yunpeng & Hu, Clark & Huang, Chao & Duan, Liqiong, 2017. "The concept of smart tourism in the context of tourism information services," Tourism Management, Elsevier, vol. 58(C), pages 293-300.
    9. Calvillo, C.F. & Sánchez-Miralles, A. & Villar, J., 2016. "Energy management and planning in smart cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 273-287.
    10. Hongyan Ma, 2020. "The Construction Path and Mode of Public Tourism Information Service System Based on the Perspective of Smart City," Complexity, Hindawi, vol. 2020, pages 1-11, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mudhafar Al-Saadi & Maher Al-Greer & Michael Short, 2023. "Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey," Energies, MDPI, vol. 16(4), pages 1-38, February.
    2. Pannee Suanpang & Pattanaphong Pothipassa & Kittisak Jermsittiparsert & Titiya Netwong, 2022. "Integration of Kouprey-Inspired Optimization Algorithms with Smart Energy Nodes for Sustainable Energy Management of Agricultural Orchards," Energies, MDPI, vol. 15(8), pages 1-18, April.
    3. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pannee Suanpang & Pattanaphong Pothipassa & Kittisak Jermsittiparsert & Titiya Netwong, 2022. "Integration of Kouprey-Inspired Optimization Algorithms with Smart Energy Nodes for Sustainable Energy Management of Agricultural Orchards," Energies, MDPI, vol. 15(8), pages 1-18, April.
    2. Ahl, Amanda & Yarime, Masaru & Tanaka, Kenji & Sagawa, Daishi, 2019. "Review of blockchain-based distributed energy: Implications for institutional development," Renewable and Sustainable Energy Reviews, Elsevier, vol. 107(C), pages 200-211.
    3. Yildizbasi, Abdullah, 2021. "Blockchain and renewable energy: Integration challenges in circular economy era," Renewable Energy, Elsevier, vol. 176(C), pages 183-197.
    4. Yoro, Kelvin O. & Daramola, Michael O. & Sekoai, Patrick T. & Wilson, Uwemedimo N. & Eterigho-Ikelegbe, Orevaoghene, 2021. "Update on current approaches, challenges, and prospects of modeling and simulation in renewable and sustainable energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    5. Farhat Afzah Samoon & Ikhlaq Hussain & Sheikh Javed Iqbal, 2023. "ILA Optimisation Based Control for Enhancing DC Link Voltage with Seamless and Adaptive VSC Control in a PV-BES Based AC Microgrid," Energies, MDPI, vol. 16(21), pages 1-23, October.
    6. Emrani-Rahaghi, Pouria & Hashemi-Dezaki, Hamed & Ketabi, Abbas, 2023. "Efficient voltage control of low voltage distribution networks using integrated optimized energy management of networked residential multi-energy microgrids," Applied Energy, Elsevier, vol. 349(C).
    7. Hussain Abdalla Sajwani & Bassel Soudan & Abdul Ghani Olabi, 2024. "Empowering Sustainability: Understanding Determinants of Consumer Investment in Microgrid Technology in the UAE," Energies, MDPI, vol. 17(9), pages 1-28, May.
    8. Francesco Lo Franco & Mattia Ricco & Riccardo Mandrioli & Gabriele Grandi, 2020. "Electric Vehicle Aggregate Power Flow Prediction and Smart Charging System for Distributed Renewable Energy Self-Consumption Optimization," Energies, MDPI, vol. 13(19), pages 1-25, September.
    9. Ray, Manojit & Chakraborty, Basab, 2022. "Impact of demand flexibility and tiered resilience on solar photovoltaic adoption in humanitarian settlements," Renewable Energy, Elsevier, vol. 193(C), pages 895-912.
    10. Grzegorz Ślusarz & Barbara Gołębiewska & Marek Cierpiał-Wolan & Jarosław Gołębiewski & Dariusz Twaróg & Sebastian Wójcik, 2021. "Regional Diversification of Potential, Production and Efficiency of Use of Biogas and Biomass in Poland," Energies, MDPI, vol. 14(3), pages 1-20, January.
    11. Dimitrios Trigkas & Chrysovalantou Ziogou & Spyros Voutetakis & Simira Papadopoulou, 2021. "Virtual Energy Storage in RES-Powered Smart Grids with Nonlinear Model Predictive Control," Energies, MDPI, vol. 14(4), pages 1-22, February.
    12. Jihed Hmad & Azeddine Houari & Allal El Moubarek Bouzid & Abdelhakim Saim & Hafedh Trabelsi, 2023. "A Review on Mode Transition Strategies between Grid-Connected and Standalone Operation of Voltage Source Inverters-Based Microgrids," Energies, MDPI, vol. 16(13), pages 1-41, June.
    13. Biancardi, Marta & Di Bari, Antonio & Villani, Giovanni, 2021. "R&D investment decision on smart cities: Energy sustainability and opportunity," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    14. Matija Kostelac & Lin Herenčić & Tomislav Capuder, 2022. "Planning and Operational Aspects of Individual and Clustered Multi-Energy Microgrid Options," Energies, MDPI, vol. 15(4), pages 1-17, February.
    15. Ahmed Y. Hatata & Mohamed A. Essa & Bishoy E. Sedhom, 2022. "Implementation and Design of FREEDM System Differential Protection Method Based on Internet of Things," Energies, MDPI, vol. 15(15), pages 1-24, August.
    16. Antoine Boche & Clément Foucher & Luiz Fernando Lavado Villa, 2022. "Understanding Microgrid Sustainability: A Systemic and Comprehensive Review," Energies, MDPI, vol. 15(8), pages 1-29, April.
    17. Dominique Barth & Benjamin Cohen-Boulakia & Wilfried Ehounou, 2022. "Distributed Reinforcement Learning for the Management of a Smart Grid Interconnecting Independent Prosumers," Energies, MDPI, vol. 15(4), pages 1-19, February.
    18. Terlouw, Tom & AlSkaif, Tarek & Bauer, Christian & van Sark, Wilfried, 2019. "Optimal energy management in all-electric residential energy systems with heat and electricity storage," Applied Energy, Elsevier, vol. 254(C).
    19. Soheil Mohseni & Alan C. Brent & Daniel Burmester, 2020. "Community Resilience-Oriented Optimal Micro-Grid Capacity Expansion Planning: The Case of Totarabank Eco-Village, New Zealand," Energies, MDPI, vol. 13(15), pages 1-29, August.
    20. Islam Elbayoumi Salem & Ahmed Mohamed Elbaz & Alamir Al-Alawi & Nasser Alhamar Alkathiri & Kareem A. Rashwan, 2022. "Investigating the Role of Green Hotel Sustainable Strategies to Improve Customer Cognitive and Affective Image: Evidence from PLS-SEM and fsQCA," Sustainability, MDPI, vol. 14(6), pages 1-17, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1906-:d:764646. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.