IDEAS home Printed from https://ideas.repec.org/a/eee/proeco/v255y2023ics0925527322002936.html
   My bibliography  Save this article

Deviations from commitments: Markov decision process formulations for the role of energy storage

Author

Listed:
  • Karakoyun, Ece Cigdem
  • Avci, Harun
  • Kocaman, Ayse Selin
  • Nadar, Emre

Abstract

We study the energy commitment, generation, and storage problem for a wind power producer who can own and operate a battery for different purposes. We consider two main problem settings: In the first setting, the producer may choose to deviate from her commitments based on the latest available information, using the battery to support such deviations. In the second setting, the producer is required to fulfill her commitments, using the battery as a back-up source. We also consider the special cases of these settings with no battery. In these settings, the producer decides how much energy to commit to purchasing or selling, how much energy to generate in the wind power plant, and how much energy to charge into or discharge from the battery. We formulate the producer’s decision-making process as a Markov decision process (MDP) by taking into account uncertainties in the electricity price and wind speed in a market setting where the price can be negative. We analytically compare the total profits of the two main settings. We then conduct data-calibrated numerical experiments to examine the effects of system components, imbalance pricing parameters, negative prices, and wind availability on the system operations and profits. Using the battery to intentionally deviate from commitments rather than to minimize such deviations improves the total profit by 16.5%, while this change in the role of the battery increases the total imbalance by 16 times, on average, in our experiments.

Suggested Citation

  • Karakoyun, Ece Cigdem & Avci, Harun & Kocaman, Ayse Selin & Nadar, Emre, 2023. "Deviations from commitments: Markov decision process formulations for the role of energy storage," International Journal of Production Economics, Elsevier, vol. 255(C).
    • Handle: RePEc:eee:proeco:v:255:y:2023:i:c:s0925527322002936
    DOI: 10.1016/j.ijpe.2022.108711
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0925527322002936
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijpe.2022.108711?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Laslett, Dean & Carter, Craig & Creagh, Chris & Jennings, Philip, 2017. "A large-scale renewable electricity supply system by 2030: Solar, wind, energy efficiency, storage and inertia for the South West Interconnected System (SWIS) in Western Australia," Renewable Energy, Elsevier, vol. 113(C), pages 713-731.
    2. Ederer, Nikolaus, 2015. "The market value and impact of offshore wind on the electricity spot market: Evidence from Germany," Applied Energy, Elsevier, vol. 154(C), pages 805-814.
    3. Xu, Xiao & Hu, Weihao & Cao, Di & Huang, Qi & Liu, Zhou & Liu, Wen & Chen, Zhe & Blaabjerg, Frede, 2020. "Scheduling of wind-battery hybrid system in the electricity market using distributionally robust optimization," Renewable Energy, Elsevier, vol. 156(C), pages 47-56.
    4. Geoffrey G. Parker & Burcu Tan & Osman Kazan, 2019. "Electric Power Industry: Operational and Public Policy Challenges and Opportunities," Production and Operations Management, Production and Operations Management Society, vol. 28(11), pages 2738-2777, November.
    5. Eicke, Anselm & Ruhnau, Oliver & Hirth, Lion, 2021. "Electricity balancing as a market equilibrium: An instrument-based estimation of supply and demand for imbalance energy," Energy Economics, Elsevier, vol. 102(C).
    6. Gomes, I.L.R. & Pousinho, H.M.I. & Melício, R. & Mendes, V.M.F., 2017. "Stochastic coordination of joint wind and photovoltaic systems with energy storage in day-ahead market," Energy, Elsevier, vol. 124(C), pages 310-320.
    7. Tauchen, George, 1986. "Finite state markov-chain approximations to univariate and vector autoregressions," Economics Letters, Elsevier, vol. 20(2), pages 177-181.
    8. Nils Löhndorf & David Wozabal & Stefan Minner, 2013. "Optimizing Trading Decisions for Hydro Storage Systems Using Approximate Dual Dynamic Programming," Operations Research, INFORMS, vol. 61(4), pages 810-823, August.
    9. Eicke, Anselm & Ruhnau, Oliver & Hirth, Lion, 2021. "Electricity balancing as a market equilibrium," EconStor Preprints 233852, ZBW - Leibniz Information Centre for Economics.
    10. Weitemeyer, Stefan & Kleinhans, David & Vogt, Thomas & Agert, Carsten, 2015. "Integration of Renewable Energy Sources in future power systems: The role of storage," Renewable Energy, Elsevier, vol. 75(C), pages 14-20.
    11. Zhang, Qi & Wang, Ge & Li, Yan & Li, Hailong & McLellan, Benjamin & Chen, Siyuan, 2018. "Substitution effect of renewable portfolio standards and renewable energy certificate trading for feed-in tariff," Applied Energy, Elsevier, vol. 227(C), pages 426-435.
    12. Finnah, Benedikt & Gönsch, Jochen, 2021. "Optimizing trading decisions of wind power plants with hybrid energy storage systems using backwards approximate dynamic programming," International Journal of Production Economics, Elsevier, vol. 238(C).
    13. Wang, Qi & Zhang, Chunyu & Ding, Yi & Xydis, George & Wang, Jianhui & Østergaard, Jacob, 2015. "Review of real-time electricity markets for integrating Distributed Energy Resources and Demand Response," Applied Energy, Elsevier, vol. 138(C), pages 695-706.
    14. Vilim, Michael & Botterud, Audun, 2014. "Wind power bidding in electricity markets with high wind penetration," Applied Energy, Elsevier, vol. 118(C), pages 141-155.
    15. Al-Swaiti, Mustafa S. & Al-Awami, Ali T. & Khalid, Mohammad Waqas, 2017. "Co-optimized trading of wind-thermal-pumped storage system in energy and regulation markets," Energy, Elsevier, vol. 138(C), pages 991-1005.
    16. Liu, Wenfeng & Zhang, Xingping & Feng, Sida, 2019. "Does renewable energy policy work? Evidence from a panel data analysis," Renewable Energy, Elsevier, vol. 135(C), pages 635-642.
    17. Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
    18. Boomsma, Trine Krogh & Juul, Nina & Fleten, Stein-Erik, 2014. "Bidding in sequential electricity markets: The Nordic case," European Journal of Operational Research, Elsevier, vol. 238(3), pages 797-809.
    19. Amro Elshurafa, 2020. "The Value of Storage in Electricity Generation: A Qualitative and Quantitative Review," Discussion Papers ks--2020-dp23, King Abdullah Petroleum Studies and Research Center.
    20. De Jonghe, Cedric & Delarue, Erik & Belmans, Ronnie & D'haeseleer, William, 2011. "Determining optimal electricity technology mix with high level of wind power penetration," Applied Energy, Elsevier, vol. 88(6), pages 2231-2238, June.
    21. Pina, André & Silva, Carlos A. & Ferrão, Paulo, 2013. "High-resolution modeling framework for planning electricity systems with high penetration of renewables," Applied Energy, Elsevier, vol. 112(C), pages 215-223.
    22. Mathiesen, Brian Vad & Lund, Henrik & Karlsson, Kenneth, 2011. "100% Renewable energy systems, climate mitigation and economic growth," Applied Energy, Elsevier, vol. 88(2), pages 488-501, February.
    23. Finnah, Benedikt & Gönsch, Jochen & Ziel, Florian, 2022. "Integrated day-ahead and intraday self-schedule bidding for energy storage systems using approximate dynamic programming," European Journal of Operational Research, Elsevier, vol. 301(2), pages 726-746.
    24. Jochen Gönsch & Michael Hassler, 2016. "Sell or store? An ADP approach to marketing renewable energy," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(3), pages 633-660, July.
    25. Theodore J. Sheskin, 1985. "Technical Note—A Markov Chain Partitioning Algorithm for Computing Steady State Probabilities," Operations Research, INFORMS, vol. 33(1), pages 228-235, February.
    26. Díaz, Guzmán & Coto, José & Gómez-Aleixandre, Javier, 2019. "Optimal operation value of combined wind power and energy storage in multi-stage electricity markets," Applied Energy, Elsevier, vol. 235(C), pages 1153-1168.
    27. Francesco Lisi and Enrico Edoli, 2018. "Analyzing and Forecasting Zonal Imbalance Signs in the Italian Electricity Market," The Energy Journal, International Association for Energy Economics, vol. 0(Number 5).
    28. Nur Sunar & John R. Birge, 2019. "Strategic Commitment to a Production Schedule with Uncertain Supply and Demand: Renewable Energy in Day-Ahead Electricity Markets," Management Science, INFORMS, vol. 65(2), pages 714-734, February.
    29. Patrick Jaillet & Ehud I. Ronn & Stathis Tompaidis, 2004. "Valuation of Commodity-Based Swing Options," Management Science, INFORMS, vol. 50(7), pages 909-921, July.
    30. Connolly, D. & Lund, H. & Mathiesen, B.V. & Leahy, M., 2011. "The first step towards a 100% renewable energy-system for Ireland," Applied Energy, Elsevier, vol. 88(2), pages 502-507, February.
    31. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    32. van der Veen, Reinier A.C. & Abbasy, Alireza & Hakvoort, Rudi A., 2012. "Agent-based analysis of the impact of the imbalance pricing mechanism on market behavior in electricity balancing markets," Energy Economics, Elsevier, vol. 34(4), pages 874-881.
    33. Yangfang (Helen) Zhou & Alan Scheller‐Wolf & Nicola Secomandi & Stephen Smith, 2019. "Managing Wind‐Based Electricity Generation in the Presence of Storage and Transmission Capacity," Production and Operations Management, Production and Operations Management Society, vol. 28(4), pages 970-989, April.
    34. Jae Ho Kim & Warren B. Powell, 2011. "Optimal Energy Commitments with Storage and Intermittent Supply," Operations Research, INFORMS, vol. 59(6), pages 1347-1360, December.
    35. Chaves-Ávila, J.P. & Hakvoort, R.A. & Ramos, A., 2014. "The impact of European balancing rules on wind power economics and on short-term bidding strategies," Energy Policy, Elsevier, vol. 68(C), pages 383-393.
    36. Gomes, I.L.R. & Melicio, R. & Mendes, V.M.F. & Pousinho, H.M.I., 2019. "Decision making for sustainable aggregation of clean energy in day-ahead market: Uncertainty and risk," Renewable Energy, Elsevier, vol. 133(C), pages 692-702.
    37. Peker, Meltem & Kocaman, Ayse Selin & Kara, Bahar Y., 2018. "Benefits of transmission switching and energy storage in power systems with high renewable energy penetration," Applied Energy, Elsevier, vol. 228(C), pages 1182-1197.
    38. Zhong, Jin & Bollen, Math & Rönnberg, Sarah, 2021. "Towards a 100% renewable energy electricity generation system in Sweden," Renewable Energy, Elsevier, vol. 171(C), pages 812-824.
    39. Chen, Jun & Rabiti, Cristian, 2017. "Synthetic wind speed scenarios generation for probabilistic analysis of hybrid energy systems," Energy, Elsevier, vol. 120(C), pages 507-517.
    40. Bird, Lori & Lew, Debra & Milligan, Michael & Carlini, E. Maria & Estanqueiro, Ana & Flynn, Damian & Gomez-Lazaro, Emilio & Holttinen, Hannele & Menemenlis, Nickie & Orths, Antje & Eriksen, Peter Børr, 2016. "Wind and solar energy curtailment: A review of international experience," Renewable and Sustainable Energy Reviews, Elsevier, vol. 65(C), pages 577-586.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sheybanivaziri, Samaneh & Le Dréau, Jérôme & Kazmi, Hussain, 2024. "Forecasting price spikes in day-ahead electricity markets: techniques, challenges, and the road ahead," Discussion Papers 2024/1, Norwegian School of Economics, Department of Business and Management Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benedikt Finnah, 2022. "Optimal bidding functions for renewable energies in sequential electricity markets," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 44(1), pages 1-27, March.
    2. Finnah, Benedikt & Gönsch, Jochen & Ziel, Florian, 2022. "Integrated day-ahead and intraday self-schedule bidding for energy storage systems using approximate dynamic programming," European Journal of Operational Research, Elsevier, vol. 301(2), pages 726-746.
    3. Díaz, Guzmán & Coto, José & Gómez-Aleixandre, Javier, 2019. "Optimal operation value of combined wind power and energy storage in multi-stage electricity markets," Applied Energy, Elsevier, vol. 235(C), pages 1153-1168.
    4. Nadarajah, Selvaprabu & Secomandi, Nicola, 2023. "A review of the operations literature on real options in energy," European Journal of Operational Research, Elsevier, vol. 309(2), pages 469-487.
    5. Weitzel, Timm & Glock, Christoph H., 2018. "Energy management for stationary electric energy storage systems: A systematic literature review," European Journal of Operational Research, Elsevier, vol. 264(2), pages 582-606.
    6. Kim, Seokwoo & Choi, Dong Gu, 2024. "A sample robust optimal bidding model for a virtual power plant," European Journal of Operational Research, Elsevier, vol. 316(3), pages 1101-1113.
    7. Anna Maria Gambaro & Nicola Secomandi, 2021. "A Discussion of Non‐Gaussian Price Processes for Energy and Commodity Operations," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 47-67, January.
    8. Østergaard, P.A. & Lund, H. & Thellufsen, J.Z. & Sorknæs, P. & Mathiesen, B.V., 2022. "Review and validation of EnergyPLAN," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    9. Laslett, Dean & Carter, Craig & Creagh, Chris & Jennings, Philip, 2017. "A large-scale renewable electricity supply system by 2030: Solar, wind, energy efficiency, storage and inertia for the South West Interconnected System (SWIS) in Western Australia," Renewable Energy, Elsevier, vol. 113(C), pages 713-731.
    10. Yangfang (Helen) Zhou & Alan Scheller‐Wolf & Nicola Secomandi & Stephen Smith, 2019. "Managing Wind‐Based Electricity Generation in the Presence of Storage and Transmission Capacity," Production and Operations Management, Production and Operations Management Society, vol. 28(4), pages 970-989, April.
    11. Vidal-Amaro, Juan José & Østergaard, Poul Alberg & Sheinbaum-Pardo, Claudia, 2015. "Optimal energy mix for transitioning from fossil fuels to renewable energy sources – The case of the Mexican electricity system," Applied Energy, Elsevier, vol. 150(C), pages 80-96.
    12. Schwidtal, Jan Marc & Agostini, Marco & Coppo, Massimiliano & Bignucolo, Fabio & Lorenzoni, Arturo, 2023. "Optimized operation of distributed energy resources: The opportunities of value stacking for Power-to-Gas aggregated with PV," Applied Energy, Elsevier, vol. 334(C).
    13. van Kooten, G. Cornelis & Withey, Patrick & Duan, Jon, 2020. "How big a battery?," Renewable Energy, Elsevier, vol. 146(C), pages 196-204.
    14. Koltsaklis, Nikolaos E. & Georgiadis, Michael C., 2015. "A multi-period, multi-regional generation expansion planning model incorporating unit commitment constraints," Applied Energy, Elsevier, vol. 158(C), pages 310-331.
    15. Klyve, Øyvind Sommer & Klæboe, Gro & Nygård, Magnus Moe & Marstein, Erik Stensrud, 2023. "Limiting imbalance settlement costs from variable renewable energy sources in the Nordics: Internal balancing vs. balancing market participation," Applied Energy, Elsevier, vol. 350(C).
    16. Gro Klæboe & Jørgen Braathen & Anders Lund Eriksrud & Stein-Erik Fleten, 2022. "Day-ahead market bidding taking the balancing power market into account," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 683-703, October.
    17. Haas, J. & Cebulla, F. & Cao, K. & Nowak, W. & Palma-Behnke, R. & Rahmann, C. & Mancarella, P., 2017. "Challenges and trends of energy storage expansion planning for flexibility provision in low-carbon power systems – a review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 603-619.
    18. Schrotenboer, Albert H. & Veenstra, Arjen A.T. & uit het Broek, Michiel A.J. & Ursavas, Evrim, 2022. "A Green Hydrogen Energy System: Optimal control strategies for integrated hydrogen storage and power generation with wind energy," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    19. Hansen, Kenneth & Breyer, Christian & Lund, Henrik, 2019. "Status and perspectives on 100% renewable energy systems," Energy, Elsevier, vol. 175(C), pages 471-480.
    20. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:proeco:v:255:y:2023:i:c:s0925527322002936. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijpe .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.