IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i21p5478-d1512214.html
   My bibliography  Save this article

Using Time-Series Databases for Energy Data Infrastructures

Author

Listed:
  • Christos Hadjichristofi

    (Software Engineering Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece)

  • Spyridon Diochnos

    (Software Engineering Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece)

  • Kyriakos Andresakis

    (Electric Energy Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece)

  • Vassilios Vescoukis

    (Software Engineering Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece
    School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15773 Athens, Greece)

Abstract

The management of energy market data, such as load, production, forecasts, and prices, is critical for energy market participants, who develop in-house energy data infrastructure services to aggregate data from many sources to support their business operations. Energy data management frequently involves time sensitive operations, including rapid data ingestion, real-time querying, filling in gaps from missing or delayed data, and updating large volumes of timestamped and loosely structured data, all of which demand high processing power. Traditional relational database management systems (RDBMSs) often struggle with these operations, whereas time series databases (TSDBs) appear to be a more efficient solution, providing enhanced scalability, reliability, real-time data availability and superior performance. This paper examines the advantages of TSDBs over RDBMS for energy data management, demonstrating that TSDBs can either replace or complement RDBMSs. We present quantitative improvements in digestion, integration, architecture, and performance, demonstrating that operations such as importing and querying time-series energy data, along with the overall system’s efficiency, can be significantly improved, achieving up to 100 times faster operations compared to relational databases, all without requiring extensive modifications to the existing information system’s architecture.

Suggested Citation

  • Christos Hadjichristofi & Spyridon Diochnos & Kyriakos Andresakis & Vassilios Vescoukis, 2024. "Using Time-Series Databases for Energy Data Infrastructures," Energies, MDPI, vol. 17(21), pages 1-23, November.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5478-:d:1512214
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/21/5478/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/21/5478/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mateusz Andrychowicz, 2021. "RES and ES Integration in Combination with Distribution Grid Development Using MILP," Energies, MDPI, vol. 14(2), pages 1-19, January.
    2. Bidong Liu & Jakub Nowotarski & Tao Hong & Rafal Weron, 2015. "Probabilistic load forecasting via Quantile Regression Averaging on sister forecasts," HSC Research Reports HSC/15/01, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    3. Luo, Shuman & Weng, Yang, 2019. "A two-stage supervised learning approach for electricity price forecasting by leveraging different data sources," Applied Energy, Elsevier, vol. 242(C), pages 1497-1512.
    4. Uniejewski, Bartosz & Weron, Rafał, 2021. "Regularized quantile regression averaging for probabilistic electricity price forecasting," Energy Economics, Elsevier, vol. 95(C).
    5. Agrawal, Rahul Kumar & Muchahary, Frankle & Tripathi, Madan Mohan, 2019. "Ensemble of relevance vector machines and boosted trees for electricity price forecasting," Applied Energy, Elsevier, vol. 250(C), pages 540-548.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    2. Heydari, Azim & Majidi Nezhad, Meysam & Pirshayan, Elmira & Astiaso Garcia, Davide & Keynia, Farshid & De Santoli, Livio, 2020. "Short-term electricity price and load forecasting in isolated power grids based on composite neural network and gravitational search optimization algorithm," Applied Energy, Elsevier, vol. 277(C).
    3. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    4. Janczura, Joanna & Wójcik, Edyta, 2022. "Dynamic short-term risk management strategies for the choice of electricity market based on probabilistic forecasts of profit and risk measures. The German and the Polish market case study," Energy Economics, Elsevier, vol. 110(C).
    5. Haider Ali & Faheem Aslam & Paulo Ferreira, 2021. "Modeling Dynamic Multifractal Efficiency of US Electricity Market," Energies, MDPI, vol. 14(19), pages 1-16, September.
    6. Olivares, Kin G. & Challu, Cristian & Marcjasz, Grzegorz & Weron, Rafał & Dubrawski, Artur, 2023. "Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx," International Journal of Forecasting, Elsevier, vol. 39(2), pages 884-900.
    7. Loizidis, Stylianos & Kyprianou, Andreas & Georghiou, George E., 2024. "Electricity market price forecasting using ELM and Bootstrap analysis: A case study of the German and Finnish Day-Ahead markets," Applied Energy, Elsevier, vol. 363(C).
    8. Lu, Renzhi & Bai, Ruichang & Huang, Yuan & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2021. "Data-driven real-time price-based demand response for industrial facilities energy management," Applied Energy, Elsevier, vol. 283(C).
    9. Elmore, Clay T. & Dowling, Alexander W., 2021. "Learning spatiotemporal dynamics in wholesale energy markets with dynamic mode decomposition," Energy, Elsevier, vol. 232(C).
    10. Jasiński, Tomasz, 2020. "Use of new variables based on air temperature for forecasting day-ahead spot electricity prices using deep neural networks: A new approach," Energy, Elsevier, vol. 213(C).
    11. He Jiang & Sheng Pan & Yao Dong & Jianzhou Wang, 2024. "Probabilistic electricity price forecasting based on penalized temporal fusion transformer," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(5), pages 1465-1491, August.
    12. Barja-Martinez, Sara & Aragüés-Peñalba, Mònica & Munné-Collado, Íngrid & Lloret-Gallego, Pau & Bullich-Massagué, Eduard & Villafafila-Robles, Roberto, 2021. "Artificial intelligence techniques for enabling Big Data services in distribution networks: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    13. Yang, Haolin & Schell, Kristen R., 2021. "Real-time electricity price forecasting of wind farms with deep neural network transfer learning and hybrid datasets," Applied Energy, Elsevier, vol. 299(C).
    14. Niematallah Elamin & Mototsugu Fukushige, 2016. "A Quantile Regression Model for Electricity Peak Demand Forecasting: An Approach to Avoiding Power Blackouts," Discussion Papers in Economics and Business 16-22, Osaka University, Graduate School of Economics.
    15. Faramarz Saghi & Mustafa Jahangoshai Rezaee, 2023. "Integrating Wavelet Decomposition and Fuzzy Transformation for Improving the Accuracy of Forecasting Crude Oil Price," Computational Economics, Springer;Society for Computational Economics, vol. 61(2), pages 559-591, February.
    16. Uniejewski, Bartosz & Maciejowska, Katarzyna, 2023. "LASSO principal component averaging: A fully automated approach for point forecast pooling," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1839-1852.
    17. Grzegorz Marcjasz & Tomasz Serafin & Rafał Weron, 2018. "Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 11(9), pages 1-20, September.
    18. Bartosz Uniejewski, 2024. "Regularization for electricity price forecasting," Papers 2404.03968, arXiv.org.
    19. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2022. "Forecasting natural gas consumption using Bagging and modified regularization techniques," Energy Economics, Elsevier, vol. 106(C).
    20. Sepehr Moalem & Roya M. Ahari & Ghazanfar Shahgholian & Majid Moazzami & Seyed Mohammad Kazemi, 2022. "Long-Term Electricity Demand Forecasting in the Steel Complex Micro-Grid Electricity Supply Chain—A Coupled Approach," Energies, MDPI, vol. 15(21), pages 1-17, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5478-:d:1512214. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.