IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v62y2021i4d10.1007_s00362-020-01165-5.html
   My bibliography  Save this article

Estimating variances in time series kriging using convex optimization and empirical BLUPs

Author

Listed:
  • Martina Hančová

    (Pavol Jozef Šafárik University)

  • Andrej Gajdoš

    (Pavol Jozef Šafárik University)

  • Jozef Hanč

    (Pavol Jozef Šafárik University)

  • Gabriela Vozáriková

    (Pavol Jozef Šafárik University)

Abstract

We revisit and update estimating variances, fundamental quantities in a time series forecasting approach called kriging, in time series models known as FDSLRMs, whose observations can be described by a linear mixed model (LMM). As a result of applying the convex optimization, we resolved two open problems in FDSLRM research: (1) theoretical existence and equivalence between two standard estimation methods—least squares estimators, non-negative (M)DOOLSE, and maximum likelihood estimators, (RE)MLE, (2) and a practical lack of free available computational implementation for FDSLRM. As for computing (RE)MLE in the case of n observed time series values, we also discovered a new algorithm of order $${\mathcal {O}}(n)$$ O ( n ) , which at the default precision is $$10^7$$ 10 7 times more accurate and $$n^2$$ n 2 times faster than the best current Python(or R)-based computational packages, namely CVXPY, CVXR, nlme, sommer and mixed. The LMM framework led us to the proposal of a two-stage estimation method of variance components based on the empirical (plug-in) best linear unbiased predictions of unobservable random components in FDSLRM. The method, providing non-negative invariant estimators with a simple explicit analytic form and performance comparable with (RE)MLE in the Gaussian case, can be used for any absolutely continuous probability distribution of time series data. We illustrate our results via applications and simulations on three real data sets (electricity consumption, tourism and cyber security), which are easily available, reproducible, sharable and modifiable in the form of interactive Jupyter notebooks.

Suggested Citation

  • Martina Hančová & Andrej Gajdoš & Jozef Hanč & Gabriela Vozáriková, 2021. "Estimating variances in time series kriging using convex optimization and empirical BLUPs," Statistical Papers, Springer, vol. 62(4), pages 1899-1938, August.
  • Handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-020-01165-5
    DOI: 10.1007/s00362-020-01165-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-020-01165-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00362-020-01165-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Julio M. Singer & Francisco M.M. Rocha & Juvêncio S. Nobre, 2017. "Graphical Tools for Detecting Departures from Linear Mixed Model Assumptions and Some Remedial Measures," International Statistical Review, International Statistical Institute, vol. 85(2), pages 290-324, August.
    2. Amemiya, Takeshi, 1977. "A note on a heteroscedastic model," Journal of Econometrics, Elsevier, vol. 6(3), pages 365-370, November.
    3. Piotr Zwiernik & Caroline Uhler & Donald Richards, 2017. "Maximum likelihood estimation for linear Gaussian covariance models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1269-1292, September.
    4. Lynn LaMotte, 2007. "A direct derivation of the REML likelihood function," Statistical Papers, Springer, vol. 48(2), pages 321-327, April.
    5. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    6. Koenker, Roger & Mizera, Ivan, 2014. "Convex Optimization in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i05).
    7. František Štulajter & Viktor Witkovský, 2004. "Estimation of variances in orthogonal finite discrete spectrum linear regression models," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 60(2), pages 105-118, September.
    8. Giovanny Covarrubias-Pazaran, 2016. "Genome-Assisted Prediction of Quantitative Traits Using the R Package sommer," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-15, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luciano Rogério Braatz de Andrade & Massaine Bandeira e Sousa & Eder Jorge Oliveira & Marcos Deon Vilela de Resende & Camila Ferreira Azevedo, 2019. "Cassava yield traits predicted by genomic selection methods," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-22, November.
    2. Gaotian Zhang & Nicole M. Roberto & Daehan Lee & Steffen R. Hahnel & Erik C. Andersen, 2022. "The impact of species-wide gene expression variation on Caenorhabditis elegans complex traits," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    3. Mitchell J. Feldmann & Dominique D. A. Pincot & Glenn S. Cole & Steven J. Knapp, 2024. "Genetic gains underpinning a little-known strawberry Green Revolution," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    4. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    5. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    6. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
    7. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    8. Morán-Ordóñez, Alejandra & Ameztegui, Aitor & De Cáceres, Miquel & de-Miguel, Sergio & Lefèvre, François & Brotons, Lluís & Coll, Lluís, 2020. "Future trade-offs and synergies among ecosystem services in Mediterranean forests under global change scenarios," Ecosystem Services, Elsevier, vol. 45(C).
    9. Damian M. Herz & Manuel Bange & Gabriel Gonzalez-Escamilla & Miriam Auer & Keyoumars Ashkan & Petra Fischer & Huiling Tan & Rafal Bogacz & Muthuraman Muthuraman & Sergiu Groppa & Peter Brown, 2022. "Dynamic control of decision and movement speed in the human basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    10. Dongyan Liu & Chongran Zhou & John K. Keesing & Oscar Serrano & Axel Werner & Yin Fang & Yingjun Chen & Pere Masque & Janine Kinloch & Aleksey Sadekov & Yan Du, 2022. "Wildfires enhance phytoplankton production in tropical oceans," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    11. Zhaogeng Yang & Yanhui Li & Peijin Hu & Jun Ma & Yi Song, 2020. "Prevalence of Anemia and its Associated Factors among Chinese 9-, 12-, and 14-Year-Old Children: Results from 2014 Chinese National Survey on Students Constitution and Health," IJERPH, MDPI, vol. 17(5), pages 1-10, February.
    12. Marco Lopez-Cruz & Fernando M. Aguate & Jacob D. Washburn & Natalia Leon & Shawn M. Kaeppler & Dayane Cristina Lima & Ruijuan Tan & Addie Thompson & Laurence Willard Bretonne & Gustavo los Campos, 2023. "Leveraging data from the Genomes-to-Fields Initiative to investigate genotype-by-environment interactions in maize in North America," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    13. Baumann, Elias & Kern, Jana & Lessmann, Stefan, 2019. "Usage Continuance in Software-as-a-Service," IRTG 1792 Discussion Papers 2019-005, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    14. repec:cup:judgdm:v:16:y:2021:i:1:p:201-237 is not listed on IDEAS
    15. C. Gabriel Hidalgo Pizango & Eurídice N. Honorio Coronado & Jhon del Águila-Pasquel & Gerardo Flores Llampazo & Johan de Jong & César J. Córdova Oroche & José M. Reyna Huaymacari & Steve J. Carver & D, 2022. "Sustainable palm fruit harvesting as a pathway to conserve Amazon peatland forests," Nature Sustainability, Nature, vol. 5(6), pages 479-487, June.
    16. Lin-Lin Wang & Zachary Y. Huang & Wen-Fei Dai & Yong-Ping Yang & Yuan-Wen Duan, 2024. "Mixed effects of honey bees on pollination function in the Tibetan alpine grasslands," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    17. Szefer Elena & Lu Donghuan & Nathoo Farouk & Beg Mirza Faisal & Graham Jinko, 2017. "Multivariate association between single-nucleotide polymorphisms in Alzgene linkage regions and structural changes in the brain: discovery, refinement and validation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 16(5-6), pages 367-386, December.
    18. Julien Collet & Samantha C Patrick & Henri Weimerskirch, 2017. "A comparative analysis of the behavioral response to fishing boats in two albatross species," Behavioral Ecology, International Society for Behavioral Ecology, vol. 28(5), pages 1337-1347.
    19. Sean Coogan & Zhixian Sui & David Raubenheimer, 2018. "Gluttony and guilt: monthly trends in internet search query data are comparable with national-level energy intake and dieting behavior," Palgrave Communications, Palgrave Macmillan, vol. 4(1), pages 1-9, December.
    20. Darcy Steeg Morris & Kimberly F. Sellers, 2022. "A Flexible Mixed Model for Clustered Count Data," Stats, MDPI, vol. 5(1), pages 1-18, January.
    21. Katrijn Delaruelle, 2023. "Migration-related inequalities in loneliness across age groups: a cross-national comparative study in Europe," European Journal of Ageing, Springer, vol. 20(1), pages 1-17, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-020-01165-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.