IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v115y2016ip3p1710-1722.html
   My bibliography  Save this article

A hybrid ICT-solution for smart meter data analytics

Author

Listed:
  • Liu, Xiufeng
  • Nielsen, Per Sieverts

Abstract

Smart meters are increasingly used worldwide. Smart meters are the advanced meters capable of measuring energy consumption at a fine-grained time interval, e.g., every 15 min. Smart meter data are typically bundled with social economic data in analytics, such as meter geographic locations, weather conditions and user information, which makes the data sets very sizable and the analytics complex. Data mining and emerging cloud computing technologies make collecting, processing, and analyzing the so-called big data possible. This paper proposes an innovative ICT-solution to streamline smart meter data analytics. The proposed solution offers an information integration pipeline for ingesting data from smart meters, a scalable platform for processing and mining big data sets, and a web portal for visualizing analytics results. The implemented system has a hybrid architecture of using Spark or Hive for big data processing, and using the machine learning toolkit, MADlib, for doing in-database data analytics in PostgreSQL database. This paper evaluates the key technologies of the proposed ICT-solution, and the results show the effectiveness and efficiency of using the system for both batch and online analytics.

Suggested Citation

  • Liu, Xiufeng & Nielsen, Per Sieverts, 2016. "A hybrid ICT-solution for smart meter data analytics," Energy, Elsevier, vol. 115(P3), pages 1710-1722.
    • Handle: RePEc:eee:energy:v:115:y:2016:i:p3:p:1710-1722
    DOI: 10.1016/j.energy.2016.05.068
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544216306855
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2016.05.068?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arghira, Nicoleta & Hawarah, Lamis & Ploix, Stéphane & Jacomino, Mireille, 2012. "Prediction of appliances energy use in smart homes," Energy, Elsevier, vol. 48(1), pages 128-134.
    2. Bennett, Christopher J. & Stewart, Rodney A. & Lu, Jun Wei, 2015. "Development of a three-phase battery energy storage scheduling and operation system for low voltage distribution networks," Applied Energy, Elsevier, vol. 146(C), pages 122-134.
    3. Bennett, Christopher J. & Stewart, Rodney A. & Lu, Jun Wei, 2014. "Forecasting low voltage distribution network demand profiles using a pattern recognition based expert system," Energy, Elsevier, vol. 67(C), pages 200-212.
    4. Personal, Enrique & Guerrero, Juan Ignacio & Garcia, Antonio & Peña, Manuel & Leon, Carlos, 2014. "Key performance indicators: A useful tool to assess Smart Grid goals," Energy, Elsevier, vol. 76(C), pages 976-988.
    5. Räsänen, Teemu & Voukantsis, Dimitrios & Niska, Harri & Karatzas, Kostas & Kolehmainen, Mikko, 2010. "Data-based method for creating electricity use load profiles using large amount of customer-specific hourly measured electricity use data," Applied Energy, Elsevier, vol. 87(11), pages 3538-3545, November.
    6. Zeng, Chunlei & Wu, Changchun & Zuo, Lili & Zhang, Bin & Hu, Xingqiao, 2014. "Predicting energy consumption of multiproduct pipeline using artificial neural networks," Energy, Elsevier, vol. 66(C), pages 791-798.
    7. Christopher Bennett & Rodney A. Stewart & Junwei Lu, 2014. "Autoregressive with Exogenous Variables and Neural Network Short-Term Load Forecast Models for Residential Low Voltage Distribution Networks," Energies, MDPI, vol. 7(5), pages 1-23, April.
    8. Beckel, Christian & Sadamori, Leyna & Staake, Thorsten & Santini, Silvia, 2014. "Revealing household characteristics from smart meter data," Energy, Elsevier, vol. 78(C), pages 397-410.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gökhan Demirdöğen & Zeynep Işık & Yusuf Arayici, 2020. "Lean Management Framework for Healthcare Facilities Integrating BIM, BEPS and Big Data Analytics," Sustainability, MDPI, vol. 12(17), pages 1-33, August.
    2. Villar-Rodriguez, Esther & Del Ser, Javier & Oregi, Izaskun & Bilbao, Miren Nekane & Gil-Lopez, Sergio, 2017. "Detection of non-technical losses in smart meter data based on load curve profiling and time series analysis," Energy, Elsevier, vol. 137(C), pages 118-128.
    3. Li, Jianbin & Chen, Zhiqiang & Cheng, Long & Liu, Xiufeng, 2022. "Energy data generation with Wasserstein Deep Convolutional Generative Adversarial Networks," Energy, Elsevier, vol. 257(C).
    4. Marek Stawowy & Adam Rosiński & Jacek Paś & Stanisław Duer & Marta Harničárová & Krzysztof Perlicki, 2023. "The Reliability and Exploitation Analysis Method of the ICT System Power Supply with the Use of Modelling Based on Rough Sets," Energies, MDPI, vol. 16(12), pages 1-18, June.
    5. Zhong, Wei & Huang, Wei & Lin, Xiaojie & Li, Zhongbo & Zhou, Yi, 2020. "Research on data-driven identification and prediction of heat response time of urban centralized heating system," Energy, Elsevier, vol. 212(C).
    6. Markovska, Natasa & Duić, Neven & Mathiesen, Brian Vad & Guzović, Zvonimir & Piacentino, Antonio & Schlör, Holger & Lund, Henrik, 2016. "Addressing the main challenges of energy security in the twenty-first century – Contributions of the conferences on Sustainable Development of Energy, Water and Environment Systems," Energy, Elsevier, vol. 115(P3), pages 1504-1512.
    7. Wen, Lulu & Zhou, Kaile & Yang, Shanlin & Li, Lanlan, 2018. "Compression of smart meter big data: A survey," Renewable and Sustainable Energy Reviews, Elsevier, vol. 91(C), pages 59-69.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jieyi Kang & David Reiner, 2021. "Machine Learning on residential electricity consumption: Which households are more responsive to weather?," Working Papers EPRG2113, Energy Policy Research Group, Cambridge Judge Business School, University of Cambridge.
    2. Gonçalves, Rui & Ribeiro, Vitor Miguel & Pereira, Fernando Lobo, 2023. "Variable Split Convolutional Attention: A novel Deep Learning model applied to the household electric power consumption," Energy, Elsevier, vol. 274(C).
    3. Lesley Thomson & David Jenkins, 2023. "The Use of Real Energy Consumption Data in Characterising Residential Energy Demand with an Inventory of UK Datasets," Energies, MDPI, vol. 16(16), pages 1-29, August.
    4. Satre-Meloy, Aven & Diakonova, Marina & Grünewald, Philipp, 2020. "Cluster analysis and prediction of residential peak demand profiles using occupant activity data," Applied Energy, Elsevier, vol. 260(C).
    5. Feras Alasali & Husam Foudeh & Esraa Mousa Ali & Khaled Nusair & William Holderbaum, 2021. "Forecasting and Modelling the Uncertainty of Low Voltage Network Demand and the Effect of Renewable Energy Sources," Energies, MDPI, vol. 14(8), pages 1-31, April.
    6. Guo, Zhifeng & O'Hanley, Jesse R. & Gibson, Stuart, 2022. "Predicting residential electricity consumption patterns based on smart meter and household data: A case study from the Republic of Ireland," Utilities Policy, Elsevier, vol. 79(C).
    7. Viegas, Joaquim L. & Vieira, Susana M. & Melício, R. & Mendes, V.M.F. & Sousa, João M.C., 2016. "Classification of new electricity customers based on surveys and smart metering data," Energy, Elsevier, vol. 107(C), pages 804-817.
    8. Bennett, Christopher J. & Stewart, Rodney A. & Lu, Jun Wei, 2015. "Development of a three-phase battery energy storage scheduling and operation system for low voltage distribution networks," Applied Energy, Elsevier, vol. 146(C), pages 122-134.
    9. Haben, Stephen & Giasemidis, Georgios & Ziel, Florian & Arora, Siddharth, 2019. "Short term load forecasting and the effect of temperature at the low voltage level," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1469-1484.
    10. Haben, Stephen & Arora, Siddharth & Giasemidis, Georgios & Voss, Marcus & Vukadinović Greetham, Danica, 2021. "Review of low voltage load forecasting: Methods, applications, and recommendations," Applied Energy, Elsevier, vol. 304(C).
    11. Trotta, Gianluca, 2020. "An empirical analysis of domestic electricity load profiles: Who consumes how much and when?," Applied Energy, Elsevier, vol. 275(C).
    12. Y, Kiguchi & Y, Heo & M, Weeks & R, Choudhary, 2019. "Predicting intra-day load profiles under time-of-use tariffs using smart meter data," Energy, Elsevier, vol. 173(C), pages 959-970.
    13. Bossink, Bart A.G., 2017. "Demonstrating sustainable energy: A review based model of sustainable energy demonstration projects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 77(C), pages 1349-1362.
    14. Shahid Nawaz Khan & Syed Ali Abbas Kazmi & Abdullah Altamimi & Zafar A. Khan & Mohammed A. Alghassab, 2022. "Smart Distribution Mechanisms—Part I: From the Perspectives of Planning," Sustainability, MDPI, vol. 14(23), pages 1-109, December.
    15. Zhou, Kaile & Yang, Changhui & Shen, Jianxin, 2017. "Discovering residential electricity consumption patterns through smart-meter data mining: A case study from China," Utilities Policy, Elsevier, vol. 44(C), pages 73-84.
    16. Wen, Hanguan & Liu, Xiufeng & Yang, Ming & Lei, Bo & Xu, Cheng & Chen, Zhe, 2024. "A novel approach for identifying customer groups for personalized demand-side management services using household socio-demographic data," Energy, Elsevier, vol. 286(C).
    17. Rongheng Lin & Budan Wu & Yun Su, 2018. "An Adaptive Weighted Pearson Similarity Measurement Method for Load Curve Clustering," Energies, MDPI, vol. 11(9), pages 1-17, September.
    18. Muratori, Matteo & Roberts, Matthew C. & Sioshansi, Ramteen & Marano, Vincenzo & Rizzoni, Giorgio, 2013. "A highly resolved modeling technique to simulate residential power demand," Applied Energy, Elsevier, vol. 107(C), pages 465-473.
    19. Hartmann, Bálint & Divényi, Dániel & Vokony, István, 2018. "Evaluation of business possibilities of energy storage at commercial and industrial consumers – A case study," Applied Energy, Elsevier, vol. 222(C), pages 59-66.
    20. Hau, Lee Cheun & Lim, Yun Seng & Liew, Serena Miao San, 2020. "A novel spontaneous self-adjusting controller of energy storage system for maximum demand reductions under penetration of photovoltaic system," Applied Energy, Elsevier, vol. 260(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:115:y:2016:i:p3:p:1710-1722. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.