IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v62y2021i4d10.1007_s00362-020-01165-5.html
   My bibliography  Save this article

Estimating variances in time series kriging using convex optimization and empirical BLUPs

Author

Listed:
  • Martina Hančová

    (Pavol Jozef Šafárik University)

  • Andrej Gajdoš

    (Pavol Jozef Šafárik University)

  • Jozef Hanč

    (Pavol Jozef Šafárik University)

  • Gabriela Vozáriková

    (Pavol Jozef Šafárik University)

Abstract

We revisit and update estimating variances, fundamental quantities in a time series forecasting approach called kriging, in time series models known as FDSLRMs, whose observations can be described by a linear mixed model (LMM). As a result of applying the convex optimization, we resolved two open problems in FDSLRM research: (1) theoretical existence and equivalence between two standard estimation methods—least squares estimators, non-negative (M)DOOLSE, and maximum likelihood estimators, (RE)MLE, (2) and a practical lack of free available computational implementation for FDSLRM. As for computing (RE)MLE in the case of n observed time series values, we also discovered a new algorithm of order $${\mathcal {O}}(n)$$ O ( n ) , which at the default precision is $$10^7$$ 10 7 times more accurate and $$n^2$$ n 2 times faster than the best current Python(or R)-based computational packages, namely CVXPY, CVXR, nlme, sommer and mixed. The LMM framework led us to the proposal of a two-stage estimation method of variance components based on the empirical (plug-in) best linear unbiased predictions of unobservable random components in FDSLRM. The method, providing non-negative invariant estimators with a simple explicit analytic form and performance comparable with (RE)MLE in the Gaussian case, can be used for any absolutely continuous probability distribution of time series data. We illustrate our results via applications and simulations on three real data sets (electricity consumption, tourism and cyber security), which are easily available, reproducible, sharable and modifiable in the form of interactive Jupyter notebooks.

Suggested Citation

  • Martina Hančová & Andrej Gajdoš & Jozef Hanč & Gabriela Vozáriková, 2021. "Estimating variances in time series kriging using convex optimization and empirical BLUPs," Statistical Papers, Springer, vol. 62(4), pages 1899-1938, August.
  • Handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-020-01165-5
    DOI: 10.1007/s00362-020-01165-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-020-01165-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00362-020-01165-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Julio M. Singer & Francisco M.M. Rocha & Juvêncio S. Nobre, 2017. "Graphical Tools for Detecting Departures from Linear Mixed Model Assumptions and Some Remedial Measures," International Statistical Review, International Statistical Institute, vol. 85(2), pages 290-324, August.
    2. Amemiya, Takeshi, 1977. "A note on a heteroscedastic model," Journal of Econometrics, Elsevier, vol. 6(3), pages 365-370, November.
    3. Piotr Zwiernik & Caroline Uhler & Donald Richards, 2017. "Maximum likelihood estimation for linear Gaussian covariance models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1269-1292, September.
    4. Lynn LaMotte, 2007. "A direct derivation of the REML likelihood function," Statistical Papers, Springer, vol. 48(2), pages 321-327, April.
    5. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    6. Koenker, Roger & Mizera, Ivan, 2014. "Convex Optimization in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i05).
    7. František Štulajter & Viktor Witkovský, 2004. "Estimation of variances in orthogonal finite discrete spectrum linear regression models," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 60(2), pages 105-118, September.
    8. Giovanny Covarrubias-Pazaran, 2016. "Genome-Assisted Prediction of Quantitative Traits Using the R Package sommer," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-15, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luciano Rogério Braatz de Andrade & Massaine Bandeira e Sousa & Eder Jorge Oliveira & Marcos Deon Vilela de Resende & Camila Ferreira Azevedo, 2019. "Cassava yield traits predicted by genomic selection methods," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-22, November.
    2. Gaotian Zhang & Nicole M. Roberto & Daehan Lee & Steffen R. Hahnel & Erik C. Andersen, 2022. "The impact of species-wide gene expression variation on Caenorhabditis elegans complex traits," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    3. Mitchell J. Feldmann & Dominique D. A. Pincot & Glenn S. Cole & Steven J. Knapp, 2024. "Genetic gains underpinning a little-known strawberry Green Revolution," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    4. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    5. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    6. Christopher Hassall & Michael Nisbet & Evan Norcliffe & He Wang, 2024. "The Potential Health Benefits of Urban Tree Planting Suggested through Immersive Environments," Land, MDPI, vol. 13(3), pages 1-12, February.
    7. Jie Zhao & Ji Chen & Damien Beillouin & Hans Lambers & Yadong Yang & Pete Smith & Zhaohai Zeng & Jørgen E. Olesen & Huadong Zang, 2022. "Global systematic review with meta-analysis reveals yield advantage of legume-based rotations and its drivers," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    8. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
    9. F J Heather & D Z Childs & A M Darnaude & J L Blanchard, 2018. "Using an integral projection model to assess the effect of temperature on the growth of gilthead seabream Sparus aurata," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-19, May.
    10. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    11. Morán-Ordóñez, Alejandra & Ameztegui, Aitor & De Cáceres, Miquel & de-Miguel, Sergio & Lefèvre, François & Brotons, Lluís & Coll, Lluís, 2020. "Future trade-offs and synergies among ecosystem services in Mediterranean forests under global change scenarios," Ecosystem Services, Elsevier, vol. 45(C).
    12. Jack McDonnell & Thomas McKenna & Kathryn A. Yurkonis & Deirdre Hennessy & Rafael Andrade Moral & Caroline Brophy, 2023. "A Mixed Model for Assessing the Effect of Numerous Plant Species Interactions on Grassland Biodiversity and Ecosystem Function Relationships," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 28(1), pages 1-19, March.
    13. Ana Pinto & Tong Yin & Marion Reichenbach & Raghavendra Bhatta & Pradeep Kumar Malik & Eva Schlecht & Sven König, 2020. "Enteric Methane Emissions of Dairy Cattle Considering Breed Composition, Pasture Management, Housing Conditions and Feeding Characteristics along a Rural-Urban Gradient in a Rising Megacity," Agriculture, MDPI, vol. 10(12), pages 1-18, December.
    14. Damian M. Herz & Manuel Bange & Gabriel Gonzalez-Escamilla & Miriam Auer & Keyoumars Ashkan & Petra Fischer & Huiling Tan & Rafal Bogacz & Muthuraman Muthuraman & Sergiu Groppa & Peter Brown, 2022. "Dynamic control of decision and movement speed in the human basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    15. Kathrin Stenchly & Marc Victor Hansen & Katharina Stein & Andreas Buerkert & Wilhelm Loewenstein, 2018. "Income Vulnerability of West African Farming Households to Losses in Pollination Services: A Case Study from Ouagadougou, Burkina Faso," Sustainability, MDPI, vol. 10(11), pages 1-12, November.
    16. Dongyan Liu & Chongran Zhou & John K. Keesing & Oscar Serrano & Axel Werner & Yin Fang & Yingjun Chen & Pere Masque & Janine Kinloch & Aleksey Sadekov & Yan Du, 2022. "Wildfires enhance phytoplankton production in tropical oceans," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    17. Zhaogeng Yang & Yanhui Li & Peijin Hu & Jun Ma & Yi Song, 2020. "Prevalence of Anemia and its Associated Factors among Chinese 9-, 12-, and 14-Year-Old Children: Results from 2014 Chinese National Survey on Students Constitution and Health," IJERPH, MDPI, vol. 17(5), pages 1-10, February.
    18. Marco Lopez-Cruz & Fernando M. Aguate & Jacob D. Washburn & Natalia Leon & Shawn M. Kaeppler & Dayane Cristina Lima & Ruijuan Tan & Addie Thompson & Laurence Willard Bretonne & Gustavo los Campos, 2023. "Leveraging data from the Genomes-to-Fields Initiative to investigate genotype-by-environment interactions in maize in North America," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    19. Baumann, Elias & Kern, Jana & Lessmann, Stefan, 2019. "Usage Continuance in Software-as-a-Service," IRTG 1792 Discussion Papers 2019-005, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    20. Alexandra M. Cheney & Stephanann M. Costello & Nicholas V. Pinkham & Annie Waldum & Susan C. Broadaway & Maria Cotrina-Vidal & Marc Mergy & Brian Tripet & Douglas J. Kominsky & Heather M. Grifka-Walk , 2023. "Gut microbiome dysbiosis drives metabolic dysfunction in Familial dysautonomia," Nature Communications, Nature, vol. 14(1), pages 1-12, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-020-01165-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.