IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2014-16.html
   My bibliography  Save this paper

Low-dimensional decomposition, smoothing and forecasting of sparse functional data

Author

Listed:
  • Alexander Dokumentov
  • Rob J Hyndman

Abstract

We propose a new generic method ROPES (Regularized Optimization for Prediction and Estimation with Sparse data) for decomposing, smoothing and forecasting two-dimensional sparse data. In some ways, ROPES is similar to Ridge Regression, the LASSO, Principal Component Analysis (PCA) and Maximum-Margin Matrix Factorisation (MMMF). Using this new approach, we propose a practical method of forecasting mortality rates, as well as a new method for interpolating and extrapolating sparse longitudinal data. We also show how to calculate prediction intervals for the resulting estimates.

Suggested Citation

  • Alexander Dokumentov & Rob J Hyndman, 2014. "Low-dimensional decomposition, smoothing and forecasting of sparse functional data," Monash Econometrics and Business Statistics Working Papers 16/14, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2014-16
    as

    Download full text from publisher

    File URL: http://business.monash.edu/econometrics-and-business-statistics/research/publications/ebs/wp16-14.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Han Lin Shang & Heather Booth & Rob Hyndman, 2011. "Point and interval forecasts of mortality rates and life expectancy: A comparison of ten principal component methods," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 25(5), pages 173-214.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexander Dokumentov & Rob J. Hyndman, 2015. "STR: A Seasonal-Trend Decomposition Procedure Based on Regression," Monash Econometrics and Business Statistics Working Papers 13/15, Monash University, Department of Econometrics and Business Statistics.
    2. Gillard, Jonathan & Usevich, Konstantin, 2018. "Structured low-rank matrix completion for forecasting in time series analysis," International Journal of Forecasting, Elsevier, vol. 34(4), pages 582-597.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahbab Mohammad Fazle Rabbi & Stefano Mazzuco, 2021. "Mortality Forecasting with the Lee–Carter Method: Adjusting for Smoothing and Lifespan Disparity," European Journal of Population, Springer;European Association for Population Studies, vol. 37(1), pages 97-120, March.
    2. OlaOluwa Simon Yaya & Luis Alberiko Gil-Alana, 2020. "Modelling Long-Range Dependence and Non-linearity in the Infant Mortality Rates of African Countries," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 26(3), pages 303-315, August.
    3. Feng, Lingbing & Shi, Yanlin & Chang, Le, 2021. "Forecasting mortality with a hyperbolic spatial temporal VAR model," International Journal of Forecasting, Elsevier, vol. 37(1), pages 255-273.
    4. Ugofilippo Basellini & Søren Kjærgaard & Carlo Giovanni Camarda, 2020. "An age-at-death distribution approach to forecast cohort mortality," Working Papers axafx5_3agsuwaphvlfk, French Institute for Demographic Studies.
    5. Man Chung Fung & Gareth W. Peters & Pavel V. Shevchenko, 2016. "A unified approach to mortality modelling using state-space framework: characterisation, identification, estimation and forecasting," Papers 1605.09484, arXiv.org.
    6. Tickle Leonie & Booth Heather, 2014. "The Longevity Prospects of Australian Seniors: An Evaluation of Forecast Method and Outcome," Asia-Pacific Journal of Risk and Insurance, De Gruyter, vol. 8(2), pages 259-292, July.
    7. Yaya, OlaOluwa S & Adekoya, Oluwasegun B. & Babatunde, Oluwagbenga T., 2021. "Testing Fractional Persistence and Nonlinearity in Infant Mortality Rates of Asia Countries," MPRA Paper 109368, University Library of Munich, Germany.
    8. Ana Debón & Steven Haberman & Francisco Montes & Edoardo Otranto, 2021. "Do Different Models Induce Changes in Mortality Indicators? That Is a Key Question for Extending the Lee-Carter Model," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    9. Basellini, Ugofilippo & Kjærgaard, Søren & Camarda, Carlo Giovanni, 2020. "An age-at-death distribution approach to forecast cohort mortality," Insurance: Mathematics and Economics, Elsevier, vol. 91(C), pages 129-143.
    10. F. Peters & J. P. Mackenbach & W. J. Nusselder, 2016. "Does the Impact of the Tobacco Epidemic Explain Structural Changes in the Decline of Mortality?," European Journal of Population, Springer;European Association for Population Studies, vol. 32(5), pages 687-702, December.
    11. Rob J Hyndman & Yijun Zeng & Han Lin Shang, 2020. "Forecasting the Old-Age Dependency Ratio to Determine a Sustainable Pension Age," Monash Econometrics and Business Statistics Working Papers 31/20, Monash University, Department of Econometrics and Business Statistics.
    12. Nan Li & Ronald Lee & Patrick Gerland, 2013. "Extending the Lee-Carter Method to Model the Rotation of Age Patterns of Mortality Decline for Long-Term Projections," Demography, Springer;Population Association of America (PAA), vol. 50(6), pages 2037-2051, December.
    13. Fanny Janssen & Leo Wissen & Anton Kunst, 2013. "Including the Smoking Epidemic in Internationally Coherent Mortality Projections," Demography, Springer;Population Association of America (PAA), vol. 50(4), pages 1341-1362, August.
    14. Han Lin Shang & Steven Haberman, 2020. "Retiree Mortality Forecasting: A Partial Age-Range or a Full Age-Range Model?," Risks, MDPI, vol. 8(3), pages 1-11, July.
    15. repec:hum:wpaper:sfb649dp2015-007 is not listed on IDEAS
    16. Li, Hong & Shi, Yanlin, 2021. "Forecasting mortality with international linkages: A global vector-autoregression approach," Insurance: Mathematics and Economics, Elsevier, vol. 100(C), pages 59-75.
    17. Phillip A. Jang & David S. Matteson, 2023. "Spatial correlation in weather forecast accuracy: a functional time series approach," Computational Statistics, Springer, vol. 38(3), pages 1215-1229, September.
    18. Yue Li & Chengmeng Zhang & Yan Tong & Yalu Zhang & Gong Chen, 2022. "Prediction of the Old-Age Dependency Ratio in Chinese Cities Using DMSP/OLS Nighttime Light Data," IJERPH, MDPI, vol. 19(12), pages 1-23, June.
    19. Francesco Billari & Rebecca Graziani & Eugenio Melilli, 2014. "Stochastic Population Forecasting Based on Combinations of Expert Evaluations Within the Bayesian Paradigm," Demography, Springer;Population Association of America (PAA), vol. 51(5), pages 1933-1954, October.
    20. Ricarda Duerst & Jonas Schöley & Christina Bohk-Ewald, 2023. "A validation workflow for mortality forecasting," MPIDR Working Papers WP-2023-020, Max Planck Institute for Demographic Research, Rostock, Germany.
    21. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.

    More about this item

    Keywords

    Tikhonov regularisation; Smoothing; Forecasting; Ridge regression; PCA; LASSO; Maximum-margin matrix factorisation; Mortality rates; Sparse longitudinal data;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2014-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.