IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v131y2019icp104-115.html
   My bibliography  Save this article

Partially observed functional data: The case of systematically missing parts

Author

Listed:
  • Liebl, Dominik
  • Rameseder, Stefan

Abstract

New estimators for the mean and the covariance function for partially observed functional data are proposed using a detour via the fundamental theorem of calculus. The new estimators allow for a consistent estimation of the mean and covariance function under specific violations of the missing-completely-at-random assumption. The requirements of the estimation procedure can be tested using a sequential multiple hypothesis test procedure. An extensive simulation study compares the new estimators with the classical estimators from the literature in different missing data scenarios. The proposed methodology is motivated by the practical problem of estimating the mean price curve in the German Control Reserve Market. In this auction market, price curves are only partially observable, and the underlying missing data mechanism depends on systematic trading strategies which clearly violate the missing-completely-at-random assumption. In contrast to the classical estimators, the new estimators lead to useful estimates of the mean and covariance functions. Supplementary materials are provided online.11Supplementary materials: R-package PartiallyFD and R-scripts for reproducing the simulation study and the real data application.

Suggested Citation

  • Liebl, Dominik & Rameseder, Stefan, 2019. "Partially observed functional data: The case of systematically missing parts," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 104-115.
    • Handle: RePEc:eee:csdana:v:131:y:2019:i:c:p:104-115
    DOI: 10.1016/j.csda.2018.08.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947318301944
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2018.08.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jonathan E. Gellar & Elizabeth Colantuoni & Dale M. Needham & Ciprian M. Crainiceanu, 2014. "Variable-Domain Functional Regression for Modeling ICU Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1425-1439, December.
    2. Dirk Engelmann & Veronika Grimm, 2009. "Bidding Behaviour in Multi-Unit Auctions - An Experimental Investigation," Economic Journal, Royal Economic Society, vol. 119(537), pages 855-882, April.
    3. Liebl, Dominik, 2013. "Modeling and Forecasting Electricity Spot Prices: A Functional Data Perspective," MPRA Paper 50881, University Library of Munich, Germany.
    4. David Kraus, 2015. "Components and completion of partially observed functional data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(4), pages 777-801, September.
    5. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    6. Yao, Fang & Muller, Hans-Georg & Wang, Jane-Ling, 2005. "Functional Data Analysis for Sparse Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 577-590, June.
    7. Aurore Delaigle & Peter Hall, 2013. "Classification Using Censored Functional Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(504), pages 1269-1283, December.
    8. A. Delaigle & P. Hall, 2016. "Approximating fragmented functional data by segments of Markov chains," Biometrika, Biometrika Trust, vol. 103(4), pages 779-799.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kraus, David, 2019. "Inferential procedures for partially observed functional data," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 583-603.
    2. Kraus, David & Stefanucci, Marco, 2020. "Ridge reconstruction of partially observed functional data is asymptotically optimal," Statistics & Probability Letters, Elsevier, vol. 165(C).
    3. Antonio Elías & Raúl Jiménez & Han Lin Shang, 2023. "Depth-based reconstruction method for incomplete functional data," Computational Statistics, Springer, vol. 38(3), pages 1507-1535, September.
    4. Yao, Binhong & Li, Peixing, 2023. "Covariance estimation error of incomplete functional data under RKHS framework," Applied Mathematics and Computation, Elsevier, vol. 443(C).
    5. Jianing Fan & Hans‐Georg Müller, 2022. "Conditional distribution regression for functional responses," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(2), pages 502-524, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kraus, David, 2019. "Inferential procedures for partially observed functional data," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 583-603.
    2. Marco Stefanucci & Laura M. Sangalli & Pierpaolo Brutti, 2018. "PCA‐based discrimination of partially observed functional data, with an application to AneuRisk65 data set," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 246-264, August.
    3. Kraus, David & Stefanucci, Marco, 2020. "Ridge reconstruction of partially observed functional data is asymptotically optimal," Statistics & Probability Letters, Elsevier, vol. 165(C).
    4. Antonio Elías & Raúl Jiménez & Han Lin Shang, 2023. "Depth-based reconstruction method for incomplete functional data," Computational Statistics, Springer, vol. 38(3), pages 1507-1535, September.
    5. A. Delaigle & P. Hall, 2016. "Approximating fragmented functional data by segments of Markov chains," Biometrika, Biometrika Trust, vol. 103(4), pages 779-799.
    6. Mojirsheibani, Majid & Shaw, Crystal, 2018. "Classification with incomplete functional covariates," Statistics & Probability Letters, Elsevier, vol. 139(C), pages 40-46.
    7. repec:cte:wsrepe:24606 is not listed on IDEAS
    8. Golovkine, Steven & Klutchnikoff, Nicolas & Patilea, Valentin, 2022. "Clustering multivariate functional data using unsupervised binary trees," Computational Statistics & Data Analysis, Elsevier, vol. 168(C).
    9. Jianing Fan & Hans‐Georg Müller, 2022. "Conditional distribution regression for functional responses," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(2), pages 502-524, June.
    10. Yang, Yang & Shang, Han Lin & Raymer, James, 2024. "Forecasting Australian fertility by age, region, and birthplace," International Journal of Forecasting, Elsevier, vol. 40(2), pages 532-548.
    11. Antonio Elías & Raúl Jiménez & J. E. Yukich, 2023. "Localization processes for functional data analysis," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(2), pages 485-517, June.
    12. Sven Otto & Nazarii Salish, 2022. "Approximate Factor Models for Functional Time Series," Papers 2201.02532, arXiv.org, revised May 2024.
    13. Matteo Fontana & Massimo Tavoni & Simone Vantini, 2019. "Functional Data Analysis of high-frequency load curves reveals drivers of residential electricity consumption," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-16, June.
    14. Valentina Masarotto & Victor M. Panaretos & Yoav Zemel, 2019. "Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 81(1), pages 172-213, February.
    15. Chen, Lu-Hung & Jiang, Ci-Ren, 2018. "Sensible functional linear discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 126(C), pages 39-52.
    16. Farzad Sabzikar & Piotr Kokoszka, 2023. "Tempered functional time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 44(3), pages 280-293, May.
    17. Shu Jiang & Yijun Xie & Graham A. Colditz, 2021. "Functional ensemble survival tree: Dynamic prediction of Alzheimer’s disease progression accommodating multiple time‐varying covariates," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 66-79, January.
    18. Paganoni, Anna M. & Sangalli, Laura M., 2019. "A Depth for Censured Functional Data," DES - Working Papers. Statistics and Econometrics. WS 28579, Universidad Carlos III de Madrid. Departamento de Estadística.
    19. Elías, Antonio & Jiménez, Raúl & Shang, Han Lin, 2022. "On projection methods for functional time series forecasting," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    20. Park, Yeonjoo & Kim, Hyunsung & Lim, Yaeji, 2023. "Functional principal component analysis for partially observed elliptical process," Computational Statistics & Data Analysis, Elsevier, vol. 184(C).
    21. Ana-Maria Staicu & Yingxing Li & Ciprian M. Crainiceanu & David Ruppert, 2014. "Likelihood Ratio Tests for Dependent Data with Applications to Longitudinal and Functional Data Analysis," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(4), pages 932-949, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:131:y:2019:i:c:p:104-115. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.