IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i19p10963-d648867.html
   My bibliography  Save this article

Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption

Author

Listed:
  • Simona-Vasilica Oprea

    (Department of Economic Informatics and Cybernetics, Bucharest University of Economic Studies, Romana Square 6, 010374 Bucharest, Romania)

  • Adela Bâra

    (Department of Economic Informatics and Cybernetics, Bucharest University of Economic Studies, Romana Square 6, 010374 Bucharest, Romania)

  • Florina Camelia Puican

    (Department of Economic Informatics and Cybernetics, Bucharest University of Economic Studies, Romana Square 6, 010374 Bucharest, Romania)

  • Ioan Cosmin Radu

    (Departament of Engineering in Foreign Languages, University Politehnica of Bucharest, Splaiul Independenței, No. 313, 060042 Bucharest, Romania)

Abstract

When analyzing smart metering data, both reading errors and frauds can be identified. The purpose of this analysis is to alert the utility companies to suspicious consumption behavior that could be further investigated with on-site inspections or other methods. The use of Machine Learning (ML) algorithms to analyze consumption readings can lead to the identification of malfunctions, cyberattacks interrupting measurements, or physical tampering with smart meters. Fraud detection is one of the classical anomaly detection examples, as it is not easy to label consumption or transactional data. Furthermore, frauds differ in nature, and learning is not always possible. In this paper, we analyze large datasets of readings provided by smart meters installed in a trial study in Ireland by applying a hybrid approach. More precisely, we propose an unsupervised ML technique to detect anomalous values in the time series, establish a threshold for the percentage of anomalous readings from the total readings, and then label that time series as suspicious or not. Initially, we propose two types of algorithms for anomaly detection for unlabeled data: Spectral Residual-Convolutional Neural Network (SR-CNN) and an anomaly trained model based on martingales for determining variations in time-series data streams. Then, the Two-Class Boosted Decision Tree and Fisher Linear Discriminant analysis are applied on the previously processed dataset. By training the model, we obtain the required capabilities of detecting suspicious consumers proved by an accuracy of 90%, precision score of 0.875, and F1 score of 0.894.

Suggested Citation

  • Simona-Vasilica Oprea & Adela Bâra & Florina Camelia Puican & Ioan Cosmin Radu, 2021. "Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption," Sustainability, MDPI, vol. 13(19), pages 1-20, October.
  • Handle: RePEc:gam:jsusta:v:13:y:2021:i:19:p:10963-:d:648867
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/19/10963/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/13/19/10963/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fan, Cheng & Xiao, Fu & Zhao, Yang & Wang, Jiayuan, 2018. "Analytical investigation of autoencoder-based methods for unsupervised anomaly detection in building energy data," Applied Energy, Elsevier, vol. 211(C), pages 1123-1135.
    2. Capozzoli, Alfonso & Piscitelli, Marco Savino & Brandi, Silvio & Grassi, Daniele & Chicco, Gianfranco, 2018. "Automated load pattern learning and anomaly detection for enhancing energy management in smart buildings," Energy, Elsevier, vol. 157(C), pages 336-352.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sepideh Radhoush & Maryam Bahramipanah & Hashem Nehrir & Zagros Shahooei, 2022. "A Review on State Estimation Techniques in Active Distribution Networks: Existing Practices and Their Challenges," Sustainability, MDPI, vol. 14(5), pages 1-16, February.
    2. Mobarak Abumohsen & Amani Yousef Owda & Majdi Owda, 2023. "Electrical Load Forecasting Using LSTM, GRU, and RNN Algorithms," Energies, MDPI, vol. 16(5), pages 1-31, February.
    3. Alexandra-Nicoleta Ciucu-Durnoi & Margareta Stela Florescu & Camelia Delcea, 2023. "Envisioning Romania’s Path to Sustainable Development: A Prognostic Approach," Sustainability, MDPI, vol. 15(17), pages 1-18, August.
    4. Yiran Wang & Shuowei Jin & Ming Cheng, 2023. "A Convolution–Non-Convolution Parallel Deep Network for Electricity Theft Detection," Sustainability, MDPI, vol. 15(13), pages 1-22, June.
    5. Yuping Zou & Rui Wu & Xuesong Tian & Hua Li, 2023. "Realizing the Improvement of the Reliability and Efficiency of Intelligent Electricity Inspection: IAOA-BP Algorithm for Anomaly Detection," Energies, MDPI, vol. 16(7), pages 1-15, March.
    6. Marta Moure-Garrido & Celeste Campo & Carlos Garcia-Rubio, 2022. "Entropy-Based Anomaly Detection in Household Electricity Consumption," Energies, MDPI, vol. 15(5), pages 1-21, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Zhelun & O’Neill, Zheng & Wen, Jin & Pradhan, Ojas & Yang, Tao & Lu, Xing & Lin, Guanjing & Miyata, Shohei & Lee, Seungjae & Shen, Chou & Chiosa, Roberto & Piscitelli, Marco Savino & Capozzoli, , 2023. "A review of data-driven fault detection and diagnostics for building HVAC systems," Applied Energy, Elsevier, vol. 339(C).
    2. Xavier Serrano-Guerrero & Guillermo Escrivá-Escrivá & Santiago Luna-Romero & Jean-Michel Clairand, 2020. "A Time-Series Treatment Method to Obtain Electrical Consumption Patterns for Anomalies Detection Improvement in Electrical Consumption Profiles," Energies, MDPI, vol. 13(5), pages 1-23, February.
    3. Himeur, Yassine & Ghanem, Khalida & Alsalemi, Abdullah & Bensaali, Faycal & Amira, Abbes, 2021. "Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives," Applied Energy, Elsevier, vol. 287(C).
    4. Roberto Chiosa & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "A Data Analytics-Based Energy Information System (EIS) Tool to Perform Meter-Level Anomaly Detection and Diagnosis in Buildings," Energies, MDPI, vol. 14(1), pages 1-28, January.
    5. Fan, Cheng & Sun, Yongjun & Zhao, Yang & Song, Mengjie & Wang, Jiayuan, 2019. "Deep learning-based feature engineering methods for improved building energy prediction," Applied Energy, Elsevier, vol. 240(C), pages 35-45.
    6. Netzah Calamaro & Yuval Beck & Ran Ben Melech & Doron Shmilovitz, 2021. "An Energy-Fraud Detection-System Capable of Distinguishing Frauds from Other Energy Flow Anomalies in an Urban Environment," Sustainability, MDPI, vol. 13(19), pages 1-38, September.
    7. Zhao, Guanjia & Cui, Zhipeng & Xu, Jing & Liu, Wenhao & Ma, Suxia, 2022. "Hybrid modeling-based digital twin for performance optimization with flexible operation in the direct air-cooling power unit," Energy, Elsevier, vol. 254(PC).
    8. Wang, Xinlin & Yao, Zhihao & Papaefthymiou, Marios, 2023. "A real-time electrical load forecasting and unsupervised anomaly detection framework," Applied Energy, Elsevier, vol. 330(PA).
    9. Ciprian Cristea & Maria Cristea & Dan Doru Micu & Andrei Ceclan & Radu-Adrian Tîrnovan & Florica Mioara Șerban, 2022. "Tridimensional Sustainability and Feasibility Assessment of Grid-Connected Solar Photovoltaic Systems Applied for the Technical University of Cluj-Napoca," Sustainability, MDPI, vol. 14(17), pages 1-23, August.
    10. Choi, Sebin & Yoon, Sungmin, 2024. "Change-point model-based clustering for urban building energy analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 199(C).
    11. Gao, Bixuan & Kong, Xiangyu & Li, Shangze & Chen, Yi & Zhang, Xiyuan & Liu, Ziyu & Lv, Weijia, 2024. "Enhancing anomaly detection accuracy and interpretability in low-quality and class imbalanced data: A comprehensive approach," Applied Energy, Elsevier, vol. 353(PB).
    12. Liang, Xinbin & Zhu, Xu & Chen, Kang & Chen, Siliang & Jin, Xinqiao & Du, Zhimin, 2023. "Endowing data-driven models with rejection ability: Out-of-distribution detection and confidence estimation for black-box models of building energy systems," Energy, Elsevier, vol. 263(PC).
    13. Hong, Yejin & Yoon, Sungmin & Choi, Sebin, 2023. "Operational signature-based symbolic hierarchical clustering for building energy, operation, and efficiency towards carbon neutrality," Energy, Elsevier, vol. 265(C).
    14. Gao, Yuan & Miyata, Shohei & Akashi, Yasunori, 2023. "How to improve the application potential of deep learning model in HVAC fault diagnosis: Based on pruning and interpretable deep learning method," Applied Energy, Elsevier, vol. 348(C).
    15. Fan, Cheng & Chen, Ruikun & Mo, Jinhan & Liao, Longhui, 2024. "Personalized federated learning for cross-building energy knowledge sharing: Cost-effective strategies and model architectures," Applied Energy, Elsevier, vol. 362(C).
    16. Grillone, Benedetto & Mor, Gerard & Danov, Stoyan & Cipriano, Jordi & Sumper, Andreas, 2021. "A data-driven methodology for enhanced measurement and verification of energy efficiency savings in commercial buildings," Applied Energy, Elsevier, vol. 301(C).
    17. Markovič, Rene & Gosak, Marko & Grubelnik, Vladimir & Marhl, Marko & Virtič, Peter, 2019. "Data-driven classification of residential energy consumption patterns by means of functional connectivity networks," Applied Energy, Elsevier, vol. 242(C), pages 506-515.
    18. Li, Kehua & Ma, Zhenjun & Robinson, Duane & Ma, Jun, 2018. "Identification of typical building daily electricity usage profiles using Gaussian mixture model-based clustering and hierarchical clustering," Applied Energy, Elsevier, vol. 231(C), pages 331-342.
    19. Cristina Nichiforov & Antonio Martinez-Molina & Miltiadis Alamaniotis, 2021. "An Intelligent Approach for Performing Energy-Driven Classification of Buildings Utilizing Joint Electricity–Gas Patterns," Energies, MDPI, vol. 14(22), pages 1-11, November.
    20. Zhan, Sicheng & Liu, Zhaoru & Chong, Adrian & Yan, Da, 2020. "Building categorization revisited: A clustering-based approach to using smart meter data for building energy benchmarking," Applied Energy, Elsevier, vol. 269(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:19:p:10963-:d:648867. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.