IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2412.07787.html
   My bibliography  Save this paper

Anomaly Detection in California Electricity Price Forecasting: Enhancing Accuracy and Reliability Using Principal Component Analysis

Author

Listed:
  • Joseph Nyangon
  • Ruth Akintunde

Abstract

Accurate and reliable electricity price forecasting has significant practical implications for grid management, renewable energy integration, power system planning, and price volatility management. This study focuses on enhancing electricity price forecasting in California's grid, addressing challenges from complex generation data and heteroskedasticity. Utilizing principal component analysis (PCA), we analyze CAISO's hourly electricity prices and demand from 2016-2021 to improve day-ahead forecasting accuracy. Initially, we apply traditional outlier analysis with the interquartile range method, followed by robust PCA (RPCA) for more effective outlier elimination. This approach improves data symmetry and reduces skewness. We then construct multiple linear regression models using both raw and PCA-transformed features. The model with transformed features, refined through traditional and SAS Sparse Matrix outlier removal methods, shows superior forecasting performance. The SAS Sparse Matrix method, in particular, significantly enhances model accuracy. Our findings demonstrate that PCA-based methods are key in advancing electricity price forecasting, supporting renewable integration and grid management in day-ahead markets. Keywords: Electricity price forecasting, principal component analysis (PCA), power system planning, heteroskedasticity, renewable energy integration.

Suggested Citation

  • Joseph Nyangon & Ruth Akintunde, 2024. "Anomaly Detection in California Electricity Price Forecasting: Enhancing Accuracy and Reliability Using Principal Component Analysis," Papers 2412.07787, arXiv.org.
    • Handle: RePEc:arx:papers:2412.07787
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2412.07787
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Woo, C.K. & Moore, J. & Schneiderman, B. & Ho, T. & Olson, A. & Alagappan, L. & Chawla, K. & Toyama, N. & Zarnikau, J., 2016. "Merit-order effects of renewable energy and price divergence in California’s day-ahead and real-time electricity markets," Energy Policy, Elsevier, vol. 92(C), pages 299-312.
    2. Yuan, Mengyao & Tong, Fan & Duan, Lei & Dowling, Jacqueline A. & Davis, Steven J. & Lewis, Nathan S. & Caldeira, Ken, 2020. "Would firm generators facilitate or deter variable renewable energy in a carbon-free electricity system?," Applied Energy, Elsevier, vol. 279(C).
    3. Sreedharan, P. & Farbes, J. & Cutter, E. & Woo, C.K. & Wang, J., 2016. "Microgrid and renewable generation integration: University of California, San Diego," Applied Energy, Elsevier, vol. 169(C), pages 709-720.
    4. Joseph Nyangon & John Byrne & Job Taminiau, 2017. "An assessment of price convergence between natural gas and solar photovoltaic in the U.S. electricity market," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 6(3), May.
    5. Joseph Nyangon & John Byrne, 2023. "Estimating the impacts of natural gas power generation growth on solar electricity development: PJM's evolving resource mix and ramping capability," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 12(1), January.
    6. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    7. Lago, Jesus & De Ridder, Fjo & De Schutter, Bart, 2018. "Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms," Applied Energy, Elsevier, vol. 221(C), pages 386-405.
    8. Fan, Qingju, 2016. "Asymmetric multiscale detrended fluctuation analysis of California electricity spot price," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 442(C), pages 252-260.
    9. Talaat, M. & Farahat, M.A. & Elkholy, M.H., 2019. "Renewable power integration: Experimental and simulation study to investigate the ability of integrating wave, solar and wind energies," Energy, Elsevier, vol. 170(C), pages 668-682.
    10. Yang, Zhang & Ce, Li & Lian, Li, 2017. "Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods," Applied Energy, Elsevier, vol. 190(C), pages 291-305.
    11. Vishnupriyan, J. & Manoharan, P.S., 2018. "Multi-criteria decision analysis for renewable energy integration: A southern India focus," Renewable Energy, Elsevier, vol. 121(C), pages 474-488.
    12. Woo, C.K. & Chen, Y. & Olson, A. & Moore, J. & Schlag, N. & Ong, A. & Ho, T., 2017. "Electricity price behavior and carbon trading: New evidence from California," Applied Energy, Elsevier, vol. 204(C), pages 531-543.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    2. Hakan Acaroğlu & Fausto Pedro García Márquez, 2021. "Comprehensive Review on Electricity Market Price and Load Forecasting Based on Wind Energy," Energies, MDPI, vol. 14(22), pages 1-23, November.
    3. Shao, Zhen & Zheng, Qingru & Yang, Shanlin & Gao, Fei & Cheng, Manli & Zhang, Qiang & Liu, Chen, 2020. "Modeling and forecasting the electricity clearing price: A novel BELM based pattern classification framework and a comparative analytic study on multi-layer BELM and LSTM," Energy Economics, Elsevier, vol. 86(C).
    4. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
    5. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    6. Díaz, Guzmán & Coto, José & Gómez-Aleixandre, Javier, 2019. "Prediction and explanation of the formation of the Spanish day-ahead electricity price through machine learning regression," Applied Energy, Elsevier, vol. 239(C), pages 610-625.
    7. Cao, K.H. & Qi, H.S. & Tsai, C.H. & Woo, C.K. & Zarnikau, J., 2021. "Energy trading efficiency in the US Midcontinent electricity markets," Applied Energy, Elsevier, vol. 302(C).
    8. Lu, Xin & Qiu, Jing & Lei, Gang & Zhu, Jianguo, 2022. "Scenarios modelling for forecasting day-ahead electricity prices: Case studies in Australia," Applied Energy, Elsevier, vol. 308(C).
    9. Woo, C.K. & Chen, Y. & Olson, A. & Moore, J. & Schlag, N. & Ong, A. & Ho, T., 2017. "Electricity price behavior and carbon trading: New evidence from California," Applied Energy, Elsevier, vol. 204(C), pages 531-543.
    10. Yang, Haolin & Schell, Kristen R., 2022. "GHTnet: Tri-Branch deep learning network for real-time electricity price forecasting," Energy, Elsevier, vol. 238(PC).
    11. Meng, Anbo & Wang, Peng & Zhai, Guangsong & Zeng, Cong & Chen, Shun & Yang, Xiaoyi & Yin, Hao, 2022. "Electricity price forecasting with high penetration of renewable energy using attention-based LSTM network trained by crisscross optimization," Energy, Elsevier, vol. 254(PA).
    12. Zarnikau, J. & Tsai, C.H. & Woo, C.K., 2020. "Determinants of the wholesale prices of energy and ancillary services in the U.S. Midcontinent electricity market," Energy, Elsevier, vol. 195(C).
    13. Ehsani, Behdad & Pineau, Pierre-Olivier & Charlin, Laurent, 2024. "Price forecasting in the Ontario electricity market via TriConvGRU hybrid model: Univariate vs. multivariate frameworks," Applied Energy, Elsevier, vol. 359(C).
    14. Derek W. Bunn & Angelica Gianfreda & Stefan Kermer, 2018. "A Trading-Based Evaluation of Density Forecasts in a Real-Time Electricity Market," Energies, MDPI, vol. 11(10), pages 1-13, October.
    15. Ghimire, Sujan & Deo, Ravinesh C. & Casillas-Pérez, David & Salcedo-Sanz, Sancho, 2024. "Two-step deep learning framework with error compensation technique for short-term, half-hourly electricity price forecasting," Applied Energy, Elsevier, vol. 353(PA).
    16. Nyangon, Joseph & Darekar, Ayesha, 2024. "Advancements in hydrogen energy systems: A review of levelized costs, financial incentives and technological innovations," Innovation and Green Development, Elsevier, vol. 3(3).
    17. Qiao, Weibiao & Yang, Zhe, 2020. "Forecast the electricity price of U.S. using a wavelet transform-based hybrid model," Energy, Elsevier, vol. 193(C).
    18. Usman Zafar & Neil Kellard & Dmitri Vinogradov, 2022. "Multistage optimization filter for trend‐based short‐term forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 345-360, March.
    19. Jens Baetens & Jeroen D. M. De Kooning & Greet Van Eetvelde & Lieven Vandevelde, 2020. "A Two-Stage Stochastic Optimisation Methodology for the Operation of a Chlor-Alkali Electrolyser under Variable DAM and FCR Market Prices," Energies, MDPI, vol. 13(21), pages 1-19, October.
    20. Bigerna, Simona, 2018. "Estimating temperature effects on the Italian electricity market," Energy Policy, Elsevier, vol. 118(C), pages 257-269.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2412.07787. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.