IDEAS home Printed from https://ideas.repec.org/a/eee/rensus/v56y2016icp1172-1181.html
   My bibliography  Save this article

Application of fuzzy Q-learning for electricity market modeling by considering renewable power penetration

Author

Listed:
  • Salehizadeh, Mohammad Reza
  • Soltaniyan, Salman

Abstract

By increasing renewable resource penetration, the need for developing fast and reliable market modeling approaches in the presence of these resources has gained greater attention. In this paper, fuzzy Q-learning approach is proposed for hour-ahead electricity market modeling in presence of renewable resources. The proposed approach is implemented on IEEE 30-bus test system. The effectiveness of the proposed approach is evaluated and compared with Q-learning approach for both normal and stressful cases. Simulation results indicate that the proposed approach is able to model electricity market for a range of continuous multidimensional renewable power penetration in considerably less iterations compared with Q-learning approach. Moreover, the probability of finding Nash equilibrium is becoming higher by using fuzzy Q-learning approach, while the other indices such as average social welfare, average of locational marginal prices (LMPs), and average standard of deviation of LMPs do not change considerably.

Suggested Citation

  • Salehizadeh, Mohammad Reza & Soltaniyan, Salman, 2016. "Application of fuzzy Q-learning for electricity market modeling by considering renewable power penetration," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 1172-1181.
    • Handle: RePEc:eee:rensus:v:56:y:2016:i:c:p:1172-1181
    DOI: 10.1016/j.rser.2015.12.020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1364032115014033
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.rser.2015.12.020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Junjie Sun & Leigh Tesfatsion, 2007. "Dynamic Testing of Wholesale Power Market Designs: An Open-Source Agent-Based Framework," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 291-327, October.
    2. Veena, P. & Indragandhi, V. & Jeyabharath, R. & Subramaniyaswamy, V., 2014. "Review of grid integration schemes for renewable power generation system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 34(C), pages 628-641.
    3. Gürkan, Gül & Langestraat, Romeo, 2014. "Modeling and analysis of renewable energy obligations and technology bandings in the UK electricity market," Energy Policy, Elsevier, vol. 70(C), pages 85-95.
    4. Pode, Ramchandra, 2015. "Battery charging stations for home lighting in Mekong region countries," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 543-560.
    5. Green, Richard & Vasilakos, Nicholas, 2010. "Market behaviour with large amounts of intermittent generation," Energy Policy, Elsevier, vol. 38(7), pages 3211-3220, July.
    6. Hota, Ashish Ranjan & Juvvanapudi, Mahesh & Bajpai, Prabodh, 2014. "Issues and solution approaches in PHEV integration to smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 30(C), pages 217-229.
    7. Tampakis, Stilianos & Τsantopoulos, Georgios & Arabatzis, Garyfallos & Rerras, Ioannis, 2013. "Citizens’ views on various forms of energy and their contribution to the environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 20(C), pages 473-482.
    8. Thure Traber & Claudia Kemfert, 2009. "Impacts of the German Support for Renewable Energy on Electricity Prices, Emissions, and Firms," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3), pages 155-178.
    9. Zafeiriou, Eleni & Arabatzis, Garyfallos & Koutroumanidis, Theodoros, 2011. "The fuelwood market in Greece: An empirical approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(6), pages 3008-3018, August.
    10. Reddy, K.S. & Kumar, Madhusudan & Mallick, T.K. & Sharon, H. & Lokeswaran, S., 2014. "A review of Integration, Control, Communication and Metering (ICCM) of renewable energy based smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 38(C), pages 180-192.
    11. Yekini Suberu, Mohammed & Wazir Mustafa, Mohd & Bashir, Nouruddeen, 2014. "Energy storage systems for renewable energy power sector integration and mitigation of intermittency," Renewable and Sustainable Energy Reviews, Elsevier, vol. 35(C), pages 499-514.
    12. Jan Abrell & Friedrich Kunz, 2015. "Integrating Intermittent Renewable Wind Generation - A Stochastic Multi-Market Electricity Model for the European Electricity Market," Networks and Spatial Economics, Springer, vol. 15(1), pages 117-147, March.
    13. Saguan, Marcelo & Meeus, Leonardo, 2014. "Impact of the regulatory framework for transmission investments on the cost of renewable energy in the EU," Energy Economics, Elsevier, vol. 43(C), pages 185-194.
    14. Wiser, Ryan & Bachrach, Devra & Bolinger, Mark & Golove, William, 2004. "Comparing the risk profiles of renewable and natural gas-fired electricity contracts," Renewable and Sustainable Energy Reviews, Elsevier, vol. 8(4), pages 335-363, August.
    15. Oró, Eduard & Depoorter, Victor & Garcia, Albert & Salom, Jaume, 2015. "Energy efficiency and renewable energy integration in data centres. Strategies and modelling review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 42(C), pages 429-445.
    16. Ventosa, Mariano & Baillo, Alvaro & Ramos, Andres & Rivier, Michel, 2005. "Electricity market modeling trends," Energy Policy, Elsevier, vol. 33(7), pages 897-913, May.
    17. Richard Green & Nicholas Vasilakos, 2011. "The Long-term Impact of Wind Power on Electricity Prices and Generating Capacity," Working Paper series, University of East Anglia, Centre for Competition Policy (CCP) 2011-04, Centre for Competition Policy, University of East Anglia, Norwich, UK..
    18. Traber, Thure & Kemfert, Claudia, 2011. "Gone with the wind? -- Electricity market prices and incentives to invest in thermal power plants under increasing wind energy supply," Energy Economics, Elsevier, vol. 33(2), pages 249-256, March.
    19. Dallinger, David & Wietschel, Martin, 2012. "Grid integration of intermittent renewable energy sources using price-responsive plug-in electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 16(5), pages 3370-3382.
    20. Weidlich, Anke & Veit, Daniel, 2008. "A critical survey of agent-based wholesale electricity market models," Energy Economics, Elsevier, vol. 30(4), pages 1728-1759, July.
    21. Richstein, Jörn C. & Chappin, Émile J.L. & de Vries, Laurens J., 2015. "Adjusting the CO2 cap to subsidised RES generation: Can CO2 prices be decoupled from renewable policy?," Applied Energy, Elsevier, vol. 156(C), pages 693-702.
    22. Ayodele, T.R. & Ogunjuyigbe, A.S.O., 2015. "Mitigation of wind power intermittency: Storage technology approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 447-456.
    23. Widén, Joakim & Carpman, Nicole & Castellucci, Valeria & Lingfors, David & Olauson, Jon & Remouit, Flore & Bergkvist, Mikael & Grabbe, Mårten & Waters, Rafael, 2015. "Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 356-375.
    24. Sichao, Kan & Yamamoto, Hiromi & Yamaji, Kenji, 2010. "Evaluation of CO2 free electricity trading market in Japan by multi-agent simulations," Energy Policy, Elsevier, vol. 38(7), pages 3309-3319, July.
    25. Aghaei, Jamshid & Alizadeh, Mohammad-Iman, 2013. "Demand response in smart electricity grids equipped with renewable energy sources: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 18(C), pages 64-72.
    26. Dominguez, R. & Baringo, L. & Conejo, A.J., 2012. "Optimal offering strategy for a concentrating solar power plant," Applied Energy, Elsevier, vol. 98(C), pages 316-325.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Koppelaar, Rembrandt H.E.M. & Keirstead, James & Shah, Nilay & Woods, Jeremy, 2016. "A review of policy analysis purpose and capabilities of electricity system models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 59(C), pages 1531-1544.
    2. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    3. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    4. Mohammad Reza Salehizadeh & Mahdi Amidi Koohbijari & Hassan Nouri & Akın Taşcıkaraoğlu & Ozan Erdinç & João P. S. Catalão, 2019. "Bi-Objective Optimization Model for Optimal Placement of Thyristor-Controlled Series Compensator Devices," Energies, MDPI, vol. 12(13), pages 1-16, July.
    5. Huiru Zhao & Yuwei Wang & Sen Guo & Mingrui Zhao & Chao Zhang, 2016. "Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling," Energies, MDPI, vol. 9(9), pages 1-20, September.
    6. Ning Wang & Weisheng Xu & Weihui Shao & Zhiyu Xu, 2019. "A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids," Energies, MDPI, vol. 12(15), pages 1-26, July.
    7. Huiru Zhao & Yuwei Wang & Mingrui Zhao & Qingkun Tan & Sen Guo, 2017. "Day-Ahead Market Modeling for Strategic Wind Power Producers under Robust Market Clearing," Energies, MDPI, vol. 10(7), pages 1-27, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Würzburg, Klaas & Labandeira, Xavier & Linares, Pedro, 2013. "Renewable generation and electricity prices: Taking stock and new evidence for Germany and Austria," Energy Economics, Elsevier, vol. 40(S1), pages 159-171.
    2. Browne, Oliver & Poletti, Stephen & Young, David, 2015. "How does market power affect the impact of large scale wind investment in 'energy only' wholesale electricity markets?," Energy Policy, Elsevier, vol. 87(C), pages 17-27.
    3. Ringler, Philipp & Keles, Dogan & Fichtner, Wolf, 2016. "Agent-based modelling and simulation of smart electricity grids and markets – A literature review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 205-215.
    4. Susanne Koschker & Dominik Möst, 2016. "Perfect competition vs. strategic behaviour models to derive electricity prices and the influence of renewables on market power," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(3), pages 661-686, July.
    5. Balint, T. & Lamperti, F. & Mandel, A. & Napoletano, M. & Roventini, A. & Sapio, A., 2017. "Complexity and the Economics of Climate Change: A Survey and a Look Forward," Ecological Economics, Elsevier, vol. 138(C), pages 252-265.
    6. repec:spo:wpmain:info:hdl:2441/1nlv566svi86iqtetenms15tc4 is not listed on IDEAS
    7. Katsanevakis, Markos & Stewart, Rodney A. & Lu, Junwei, 2017. "Aggregated applications and benefits of energy storage systems with application-specific control methods: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 75(C), pages 719-741.
    8. repec:hal:spmain:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    9. Bhagwat, Pradyumna C. & Marcheselli, Anna & Richstein, Jörn C. & Chappin, Emile J.L. & De Vries, Laurens J., 2017. "An analysis of a forward capacity market with long-term contracts," Energy Policy, Elsevier, vol. 111(C), pages 255-267.
    10. Ren, Guorui & Liu, Jinfu & Wan, Jie & Guo, Yufeng & Yu, Daren, 2017. "Overview of wind power intermittency: Impacts, measurements, and mitigation solutions," Applied Energy, Elsevier, vol. 204(C), pages 47-65.
    11. Farid, Amro M. & Jiang, Bo & Muzhikyan, Aramazd & Youcef-Toumi, Kamal, 2016. "The need for holistic enterprise control assessment methods for the future electricity grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 669-685.
    12. repec:spo:wpecon:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    13. Jean-Luc Gaffard & Mauro Napoletano, 2012. "Agent-based models and economic policy," Post-Print hal-03461120, HAL.
    14. Bublitz, Andreas & Keles, Dogan & Fichtner, Wolf, 2017. "An analysis of the decline of electricity spot prices in Europe: Who is to blame?," Energy Policy, Elsevier, vol. 107(C), pages 323-336.
    15. Wolf-Peter Schill & Claudia Kemfert, 2011. "Modeling Strategic Electricity Storage: The Case of Pumped Hydro Storage in Germany," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3), pages 59-88.
    16. repec:spo:wpmain:info:hdl:2441/5qr7f0k4sk8rbq4do5u6v70rm0 is not listed on IDEAS
    17. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    18. Zamani-Dehkordi, Payam & Rakai, Logan & Zareipour, Hamidreza, 2016. "Deciding on the support schemes for upcoming wind farms in competitive electricity markets," Energy, Elsevier, vol. 116(P1), pages 8-19.
    19. repec:hal:spmain:info:hdl:2441/1nlv566svi86iqtetenms15tc4 is not listed on IDEAS
    20. repec:hal:spmain:info:hdl:2441/5qr7f0k4sk8rbq4do5u6v70rm0 is not listed on IDEAS
    21. repec:hal:wpspec:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    22. Poghosyan, Anush & Greetham, Danica Vukadinović & Haben, Stephen & Lee, Tamsin, 2015. "Long term individual load forecast under different electrical vehicles uptake scenarios," Applied Energy, Elsevier, vol. 157(C), pages 699-709.
    23. Sébastien Phan & Fabien Roques, 2015. "Is the depressive effect of renewables on power prices contagious? A cross border econometric analysis," Cambridge Working Papers in Economics 1527, Faculty of Economics, University of Cambridge.
    24. repec:spo:wpmain:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    25. Foteinis, S. & Tsoutsos, T., 2017. "Strategies to improve sustainability and offset the initial high capital expenditure of wave energy converters (WECs)," Renewable and Sustainable Energy Reviews, Elsevier, vol. 70(C), pages 775-785.
    26. Eid, Cherrelle & Codani, Paul & Perez, Yannick & Reneses, Javier & Hakvoort, Rudi, 2016. "Managing electric flexibility from Distributed Energy Resources: A review of incentives for market design," Renewable and Sustainable Energy Reviews, Elsevier, vol. 64(C), pages 237-247.
    27. Rubin, Ofir D. & Babcock, Bruce A., 2013. "The impact of expansion of wind power capacity and pricing methods on the efficiency of deregulated electricity markets," Energy, Elsevier, vol. 59(C), pages 676-688.
    28. Cristian Zambrano & Yris Olaya, 2017. "An agent-based simulation approach to congestion management for the Colombian electricity market," Annals of Operations Research, Springer, vol. 258(2), pages 217-236, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:rensus:v:56:y:2016:i:c:p:1172-1181. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600126/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.