IDEAS home Printed from https://ideas.repec.org/a/bla/wireae/v13y2024i1ne504.html
   My bibliography  Save this article

Principal component analysis of day‐ahead electricity price forecasting in CAISO and its implications for highly integrated renewable energy markets

Author

Listed:
  • Joseph Nyangon
  • Ruth Akintunde

Abstract

Electricity price forecasting is crucial for grid management, renewable energy integration, power system planning, and price volatility management. However, poor accuracy due to complex generation mix data and heteroskedasticity poses a challenge for utilities and grid operators. This paper evaluates advanced analytics methods that utilize principal component analysis (PCA) to improve forecasting accuracy amidst heteroskedastic noise. Drawing on the experience of the California Independent System Operator (CAISO), a leading producer of renewable electricity, the study analyzes hourly electricity prices and demand data from 2016 to 2021 to assess the impact of day‐ahead forecasting on California's evolving generation mix. To enhance data quality, traditional outlier analysis using the interquartile range (IQR) method is first applied, followed by a novel supervised PCA technique called robust PCA (RPCA) for more effective outlier detection and elimination. The combined approach significantly improves data symmetry and reduces skewness. Multiple linear regression models are then constructed to forecast electricity prices using both raw and transformed features obtained through PCA. Results demonstrate that the model utilizing transformed features, after outlier removal using the traditional method and SAS Sparse Matrix method, achieves the highest forecasting performance. Notably, the SAS Sparse Matrix outlier removal method, implemented via proc RPCA, greatly contributes to improved model accuracy. This study highlights that PCA methods enhance electricity price forecasting accuracy, facilitating the integration of renewables like solar and wind, thereby aiding grid management and promoting renewable growth in day‐ahead markets. This article is categorized under: Energy and Power Systems > Energy Management Energy and Power Systems > Distributed Generation Emerging Technologies > Digitalization

Suggested Citation

  • Joseph Nyangon & Ruth Akintunde, 2024. "Principal component analysis of day‐ahead electricity price forecasting in CAISO and its implications for highly integrated renewable energy markets," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 13(1), January.
    • Handle: RePEc:bla:wireae:v:13:y:2024:i:1:n:e504
    DOI: 10.1002/wene.504
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/wene.504
    Download Restriction: no
    File URL: https://libkey.io/10.1002/wene.504?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ying-Yi Hong & Ching-Ping Wu, 2012. "Day-Ahead Electricity Price Forecasting Using a Hybrid Principal Component Analysis Network," Energies, MDPI, vol. 5(11), pages 1-15, November.
    2. Daoping Wang & Dabo Guan & Shupeng Zhu & Michael Mac Kinnon & Guannan Geng & Qiang Zhang & Heran Zheng & Tianyang Lei & Shuai Shao & Peng Gong & Steven J. Davis, 2021. "Economic footprint of California wildfires in 2018," Nature Sustainability, Nature, vol. 4(3), pages 252-260, March.
    3. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    4. Naderi, Mobin & Khayat, Yousef & Shafiee, Qobad & Blaabjerg, Frede & Bevrani, Hassan, 2023. "Dynamic modeling, stability analysis and control of interconnected microgrids: A review," Applied Energy, Elsevier, vol. 334(C).
    5. Michael G. Pollitt & Karim L. Anaya, 2021. "Competition in Markets for Ancillary Services? The Implications of Rising Distributed Generation," The Energy Journal, , vol. 42(1_suppl), pages 1-28, June.
    6. Azadeh Shomali & Jonatan Pinkse, 2016. "The consequences of smart grids for the business model of electricity firms," Post-Print hal-02022695, HAL.
    7. Daron Acemoglu & Ali Kakhbod & Asuman Ozdaglar, 2017. "Competition in Electricity Markets with Renewable Energy Sources," The Energy Journal, , vol. 38(1_suppl), pages 137-156, June.
    8. Weron, Rafal & Misiorek, Adam, 2008. "Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models," International Journal of Forecasting, Elsevier, vol. 24(4), pages 744-763.
    9. Florian van Triel & Timothy E. Lipman, 2020. "Modeling the Future California Electricity Grid and Renewable Energy Integration with Electric Vehicles," Energies, MDPI, vol. 13(20), pages 1-20, October.
    10. Nyangon, Joseph & Byrne, John, 2018. "Diversifying Electricity Customer Choice: REVing Up the New York Energy Vision for Polycentric Innovation," MPRA Paper 91486, University Library of Munich, Germany.
    11. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    12. Mohammad Mahdi Forootan & Iman Larki & Rahim Zahedi & Abolfazl Ahmadi, 2022. "Machine Learning and Deep Learning in Energy Systems: A Review," Sustainability, MDPI, vol. 14(8), pages 1-49, April.
    13. Sreedharan, P. & Farbes, J. & Cutter, E. & Woo, C.K. & Wang, J., 2016. "Microgrid and renewable generation integration: University of California, San Diego," Applied Energy, Elsevier, vol. 169(C), pages 709-720.
    14. Voisin, Nathalie & Dyreson, Ana & Fu, Tao & O'Connell, Matt & Turner, Sean W.D. & Zhou, Tian & Macknick, Jordan, 2020. "Impact of climate change on water availability and its propagation through the Western U.S. power grid," Applied Energy, Elsevier, vol. 276(C).
    15. Abiodun, Kehinde & Hood, Karoline & Cox, John L. & Newman, Alexandra M. & Zolan, Alex J., 2023. "The value of concentrating solar power in ancillary services markets," Applied Energy, Elsevier, vol. 334(C).
    16. Joseph Nyangon & John Byrne, 2023. "Estimating the impacts of natural gas power generation growth on solar electricity development: PJM's evolving resource mix and ramping capability," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 12(1), January.
    17. Kun Li & Joseph D. Cursio & Yunchuan Sun, 2018. "Principal Component Analysis of Price Fluctuation in the Smart Grid Electricity Market," Sustainability, MDPI, vol. 10(11), pages 1-16, November.
    18. Hou, Qingchun & Zhang, Ning & Du, Ershun & Miao, Miao & Peng, Fei & Kang, Chongqing, 2019. "Probabilistic duck curve in high PV penetration power system: Concept, modeling, and empirical analysis in China," Applied Energy, Elsevier, vol. 242(C), pages 205-215.
    19. Lago, Jesus & De Ridder, Fjo & De Schutter, Bart, 2018. "Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms," Applied Energy, Elsevier, vol. 221(C), pages 386-405.
    20. Talaat, M. & Farahat, M.A. & Elkholy, M.H., 2019. "Renewable power integration: Experimental and simulation study to investigate the ability of integrating wave, solar and wind energies," Energy, Elsevier, vol. 170(C), pages 668-682.
    21. Yang, Zhang & Ce, Li & Lian, Li, 2017. "Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods," Applied Energy, Elsevier, vol. 190(C), pages 291-305.
    22. Woo, C.K. & Chen, Y. & Olson, A. & Moore, J. & Schlag, N. & Ong, A. & Ho, T., 2017. "Electricity price behavior and carbon trading: New evidence from California," Applied Energy, Elsevier, vol. 204(C), pages 531-543.
    23. Yuan, Jiahai & Li, Xinying & Xu, Chuanbo & Zhao, Changhong & Liu, Yuanxin, 2019. "Investment risk assessment of coal-fired power plants in countries along the Belt and Road initiative based on ANP-Entropy-TODIM method," Energy, Elsevier, vol. 176(C), pages 623-640.
    24. Debora Maia-Silva & Rohini Kumar & Roshanak Nateghi, 2020. "The critical role of humidity in modeling summer electricity demand across the United States," Nature Communications, Nature, vol. 11(1), pages 1-8, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joseph Nyangon & Ruth Akintunde, 2024. "Anomaly Detection in California Electricity Price Forecasting: Enhancing Accuracy and Reliability Using Principal Component Analysis," Papers 2412.07787, arXiv.org.
    2. Ehsani, Behdad & Pineau, Pierre-Olivier & Charlin, Laurent, 2024. "Price forecasting in the Ontario electricity market via TriConvGRU hybrid model: Univariate vs. multivariate frameworks," Applied Energy, Elsevier, vol. 359(C).
    3. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    4. Galarneau-Vincent, Rémi & Gauthier, Geneviève & Godin, Frédéric, 2023. "Foreseeing the worst: Forecasting electricity DART spikes," Energy Economics, Elsevier, vol. 119(C).
    5. Vladimir Franki & Darin Majnarić & Alfredo Višković, 2023. "A Comprehensive Review of Artificial Intelligence (AI) Companies in the Power Sector," Energies, MDPI, vol. 16(3), pages 1-35, January.
    6. Krishna Prakash N. & Jai Govind Singh, 2023. "Electricity price forecasting using hybrid deep learned networks," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1750-1771, November.
    7. Halužan, Marko & Verbič, Miroslav & Zorić, Jelena, 2020. "Performance of alternative electricity price forecasting methods: Findings from the Greek and Hungarian power exchanges," Applied Energy, Elsevier, vol. 277(C).
    8. Grzegorz Marcjasz, 2020. "Forecasting Electricity Prices Using Deep Neural Networks: A Robust Hyper-Parameter Selection Scheme," Energies, MDPI, vol. 13(18), pages 1-18, September.
    9. Lehna, Malte & Scheller, Fabian & Herwartz, Helmut, 2022. "Forecasting day-ahead electricity prices: A comparison of time series and neural network models taking external regressors into account," Energy Economics, Elsevier, vol. 106(C).
    10. Jiang, Ping & Nie, Ying & Wang, Jianzhou & Huang, Xiaojia, 2023. "Multivariable short-term electricity price forecasting using artificial intelligence and multi-input multi-output scheme," Energy Economics, Elsevier, vol. 117(C).
    11. Yin, Linfei & Qiu, Yao, 2022. "Neural network dynamic differential control for long-term price guidance mechanism of flexible energy service providers," Energy, Elsevier, vol. 255(C).
    12. Lago, Jesus & De Ridder, Fjo & Vrancx, Peter & De Schutter, Bart, 2018. "Forecasting day-ahead electricity prices in Europe: The importance of considering market integration," Applied Energy, Elsevier, vol. 211(C), pages 890-903.
    13. He Jiang & Yao Dong & Jianzhou Wang, 2024. "Electricity price forecasting using quantile regression averaging with nonconvex regularization," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(6), pages 1859-1879, September.
    14. Demir, Sumeyra & Mincev, Krystof & Kok, Koen & Paterakis, Nikolaos G., 2021. "Data augmentation for time series regression: Applying transformations, autoencoders and adversarial networks to electricity price forecasting," Applied Energy, Elsevier, vol. 304(C).
    15. Palacio, Sebastián M., 2020. "Predicting collusive patterns in a liberalized electricity market with mandatory auctions of forward contracts," Energy Policy, Elsevier, vol. 139(C).
    16. Ciarreta, Aitor & Martinez, Blanca & Nasirov, Shahriyar, 2023. "Forecasting electricity prices using bid data," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1253-1271.
    17. Chai, Shanglei & Li, Qiang & Abedin, Mohammad Zoynul & Lucey, Brian M., 2024. "Forecasting electricity prices from the state-of-the-art modeling technology and the price determinant perspectives," Research in International Business and Finance, Elsevier, vol. 67(PA).
    18. Florian Ziel & Rick Steinert & Sven Husmann, 2015. "Forecasting day ahead electricity spot prices: The impact of the EXAA to other European electricity markets," Papers 1501.00818, arXiv.org, revised Dec 2015.
    19. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
    20. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:wireae:v:13:y:2024:i:1:n:e504. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=2041-8396 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.