IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i14p5469-d1197254.html
   My bibliography  Save this article

Dynamic Pricing Based on Demand Response Using Actor–Critic Agent Reinforcement Learning

Author

Listed:
  • Ahmed Ismail

    (Faculty of Electrical and Electronics Engineering, Yildiz Technical University, Davutpasa Campus, Esenler, 34220 Istanbul, Turkey)

  • Mustafa Baysal

    (Faculty of Electrical and Electronics Engineering, Yildiz Technical University, Davutpasa Campus, Esenler, 34220 Istanbul, Turkey)

Abstract

Eco-friendly technologies for sustainable energy development require the efficient utilization of energy resources. Real-time pricing (RTP), also known as dynamic pricing, offers advantages over other pricing systems by enabling demand response (DR) actions. However, existing methods for determining and controlling DR have limitations in managing an increasing demand and predicting future pricing. This paper presents a novel approach to address the limitations of existing methods for determining and controlling demand response (DR) in the context of dynamic pricing systems for sustainable energy development. By leveraging actor–critic agent reinforcement learning (RL) techniques, a dynamic pricing DR model is proposed for efficient energy management. The model’s learning framework was trained using DR and real-time pricing data extracted from the Australian Energy Market Operator (AEMO) spanning a period of 17 years. The efficacy of the RL-based dynamic pricing approach was evaluated through two predicting cases: actual-predicted demand and actual-predicted price. Initially, long short-term memory (LSTM) models were employed to predict price and demand, and the results were subsequently enhanced using the deep RL model. Remarkably, the proposed approach achieved an impressive accuracy of 99% for every 30 min future price prediction. The results demonstrated the efficiency of the proposed RL-based model in accurately predicting both demand and price for effective energy management.

Suggested Citation

  • Ahmed Ismail & Mustafa Baysal, 2023. "Dynamic Pricing Based on Demand Response Using Actor–Critic Agent Reinforcement Learning," Energies, MDPI, vol. 16(14), pages 1-19, July.
    • Handle: RePEc:gam:jeners:v:16:y:2023:i:14:p:5469-:d:1197254
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/14/5469/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/14/5469/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ihab Taleb & Guillaume Guerard & Frédéric Fauberteau & Nga Nguyen, 2022. "A Flexible Deep Learning Method for Energy Forecasting," Energies, MDPI, vol. 15(11), pages 1-16, May.
    2. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    3. Miller, Mark & Alberini, Anna, 2016. "Sensitivity of price elasticity of demand to aggregation, unobserved heterogeneity, price trends, and price endogeneity: Evidence from U.S. Data," Energy Policy, Elsevier, vol. 97(C), pages 235-249.
    4. Yu, Mengmeng & Lu, Renzhi & Hong, Seung Ho, 2016. "A real-time decision model for industrial load management in a smart grid," Applied Energy, Elsevier, vol. 183(C), pages 1488-1497.
    5. Wang, Jianzhou & Zhu, Wenjin & Zhang, Wenyu & Sun, Donghuai, 2009. "A trend fixed on firstly and seasonal adjustment model combined with the [epsilon]-SVR for short-term forecasting of electricity demand," Energy Policy, Elsevier, vol. 37(11), pages 4901-4909, November.
    6. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    7. Ying Ji & Jianhui Wang & Jiacan Xu & Xiaoke Fang & Huaguang Zhang, 2019. "Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning," Energies, MDPI, vol. 12(12), pages 1-21, June.
    8. Zhou, P. & Ang, B.W. & Poh, K.L., 2006. "A trigonometric grey prediction approach to forecasting electricity demand," Energy, Elsevier, vol. 31(14), pages 2839-2847.
    9. Frank A. Wolak, 2011. "Do Residential Customers Respond to Hourly Prices? Evidence from a Dynamic Pricing Experiment," American Economic Review, American Economic Association, vol. 101(3), pages 83-87, May.
    10. Akay, Diyar & Atak, Mehmet, 2007. "Grey prediction with rolling mechanism for electricity demand forecasting of Turkey," Energy, Elsevier, vol. 32(9), pages 1670-1675.
    11. Li, Xiao Hui & Hong, Seung Ho, 2014. "User-expected price-based demand response algorithm for a home-to-grid system," Energy, Elsevier, vol. 64(C), pages 437-449.
    12. Torriti, Jacopo, 2012. "Price-based demand side management: Assessing the impacts of time-of-use tariffs on residential electricity demand and peak shifting in Northern Italy," Energy, Elsevier, vol. 44(1), pages 576-583.
    13. Mirasgedis, S. & Sarafidis, Y. & Georgopoulou, E. & Lalas, D.P. & Moschovits, M. & Karagiannis, F. & Papakonstantinou, D., 2006. "Models for mid-term electricity demand forecasting incorporating weather influences," Energy, Elsevier, vol. 31(2), pages 208-227.
    14. Taylor, James W., 2010. "Triple seasonal methods for short-term electricity demand forecasting," European Journal of Operational Research, Elsevier, vol. 204(1), pages 139-152, July.
    15. Nolan, Sheila & O’Malley, Mark, 2015. "Challenges and barriers to demand response deployment and evaluation," Applied Energy, Elsevier, vol. 152(C), pages 1-10.
    16. Gao, Dian-ce & Sun, Yongjun & Lu, Yuehong, 2015. "A robust demand response control of commercial buildings for smart grid under load prediction uncertainty," Energy, Elsevier, vol. 93(P1), pages 275-283.
    17. Aghaei, Jamshid & Alizadeh, Mohammad-Iman & Siano, Pierluigi & Heidari, Alireza, 2016. "Contribution of emergency demand response programs in power system reliability," Energy, Elsevier, vol. 103(C), pages 688-696.
    18. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    19. Faria, P. & Vale, Z., 2011. "Demand response in electrical energy supply: An optimal real time pricing approach," Energy, Elsevier, vol. 36(8), pages 5374-5384.
    20. J W Taylor, 2003. "Short-term electricity demand forecasting using double seasonal exponential smoothing," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(8), pages 799-805, August.
    21. Katrina Jessoe & David Rapson, 2015. "Commercial and Industrial Demand Response Under Mandatory Time-of-Use Electricity Pricing," Journal of Industrial Economics, Wiley Blackwell, vol. 63(3), pages 397-421, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    2. Aftab Ahmed Almani & Xueshan Han, 2023. "Real-Time Pricing-Enabled Demand Response Using Long Short-Time Memory Deep Learning," Energies, MDPI, vol. 16(5), pages 1-13, March.
    3. Dutta, Goutam & Mitra, Krishnendranath, 2015. "Dynamic Pricing of Electricity: A Survey of Related Research," IIMA Working Papers WP2015-08-03, Indian Institute of Management Ahmedabad, Research and Publication Department.
    4. Goutam Dutta & Krishnendranath Mitra, 2017. "A literature review on dynamic pricing of electricity," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(10), pages 1131-1145, October.
    5. Weide Li & Demeng Kong & Jinran Wu, 2017. "A Novel Hybrid Model Based on Extreme Learning Machine, k-Nearest Neighbor Regression and Wavelet Denoising Applied to Short-Term Electric Load Forecasting," Energies, MDPI, vol. 10(5), pages 1-16, May.
    6. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    7. Wang, Shuai & Yu, Lean & Tang, Ling & Wang, Shouyang, 2011. "A novel seasonal decomposition based least squares support vector regression ensemble learning approach for hydropower consumption forecasting in China," Energy, Elsevier, vol. 36(11), pages 6542-6554.
    8. Zhu, Suling & Wang, Jianzhou & Zhao, Weigang & Wang, Jujie, 2011. "A seasonal hybrid procedure for electricity demand forecasting in China," Applied Energy, Elsevier, vol. 88(11), pages 3807-3815.
    9. Zhang, Xiongfeng & Lu, Renzhi & Jiang, Junhui & Hong, Seung Ho & Song, Won Seok, 2021. "Testbed implementation of reinforcement learning-based demand response energy management system," Applied Energy, Elsevier, vol. 297(C).
    10. AL-Musaylh, Mohanad S. & Deo, Ravinesh C. & Adamowski, Jan F. & Li, Yan, 2019. "Short-term electricity demand forecasting using machine learning methods enriched with ground-based climate and ECMWF Reanalysis atmospheric predictors in southeast Queensland, Australia," Renewable and Sustainable Energy Reviews, Elsevier, vol. 113(C), pages 1-1.
    11. Debnath, Kumar Biswajit & Mourshed, Monjur, 2018. "Forecasting methods in energy planning models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 88(C), pages 297-325.
    12. Meyabadi, A. Fattahi & Deihimi, M.H., 2017. "A review of demand-side management: Reconsidering theoretical framework," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 367-379.
    13. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    14. Gholami, M. & Barbaresi, A. & Torreggiani, D. & Tassinari, P., 2020. "Upscaling of spatial energy planning, phases, methods, and techniques: A systematic review through meta-analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 132(C).
    15. Janusz Sowinski, 2021. "The Impact of the Selection of Exogenous Variables in the ANFIS Model on the Results of the Daily Load Forecast in the Power Company," Energies, MDPI, vol. 14(2), pages 1-18, January.
    16. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    17. Christopher Bennett & Rodney A. Stewart & Junwei Lu, 2014. "Autoregressive with Exogenous Variables and Neural Network Short-Term Load Forecast Models for Residential Low Voltage Distribution Networks," Energies, MDPI, vol. 7(5), pages 1-23, April.
    18. Wang, Yuanyuan & Wang, Jianzhou & Zhao, Ge & Dong, Yao, 2012. "Application of residual modification approach in seasonal ARIMA for electricity demand forecasting: A case study of China," Energy Policy, Elsevier, vol. 48(C), pages 284-294.
    19. Haider, Haider Tarish & See, Ong Hang & Elmenreich, Wilfried, 2016. "Residential demand response scheme based on adaptive consumption level pricing," Energy, Elsevier, vol. 113(C), pages 301-308.
    20. Bianco, Vincenzo & Manca, Oronzio & Nardini, Sergio & Minea, Alina A., 2010. "Analysis and forecasting of nonresidential electricity consumption in Romania," Applied Energy, Elsevier, vol. 87(11), pages 3584-3590, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:14:p:5469-:d:1197254. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.