IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2014-16.html
   My bibliography  Save this paper

Low-dimensional decomposition, smoothing and forecasting of sparse functional data

Author

Listed:
  • Alexander Dokumentov
  • Rob J Hyndman

Abstract

We propose a new generic method ROPES (Regularized Optimization for Prediction and Estimation with Sparse data) for decomposing, smoothing and forecasting two-dimensional sparse data. In some ways, ROPES is similar to Ridge Regression, the LASSO, Principal Component Analysis (PCA) and Maximum-Margin Matrix Factorisation (MMMF). Using this new approach, we propose a practical method of forecasting mortality rates, as well as a new method for interpolating and extrapolating sparse longitudinal data. We also show how to calculate prediction intervals for the resulting estimates.

Suggested Citation

  • Alexander Dokumentov & Rob J Hyndman, 2014. "Low-dimensional decomposition, smoothing and forecasting of sparse functional data," Monash Econometrics and Business Statistics Working Papers 16/14, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2014-16
    as

    Download full text from publisher

    File URL: http://business.monash.edu/econometrics-and-business-statistics/research/publications/ebs/wp16-14.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Han Lin Shang & Heather Booth & Rob Hyndman, 2011. "Point and interval forecasts of mortality rates and life expectancy: A comparison of ten principal component methods," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 25(5), pages 173-214.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexander Dokumentov & Rob J. Hyndman, 2015. "STR: A Seasonal-Trend Decomposition Procedure Based on Regression," Monash Econometrics and Business Statistics Working Papers 13/15, Monash University, Department of Econometrics and Business Statistics.
    2. Gillard, Jonathan & Usevich, Konstantin, 2018. "Structured low-rank matrix completion for forecasting in time series analysis," International Journal of Forecasting, Elsevier, vol. 34(4), pages 582-597.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rob Hyndman & Heather Booth & Farah Yasmeen, 2013. "Coherent Mortality Forecasting: The Product-Ratio Method With Functional Time Series Models," Demography, Springer;Population Association of America (PAA), vol. 50(1), pages 261-283, February.
    2. Fang, Lei & Härdle, Wolfgang Karl, 2015. "Stochastic population analysis: A functional data approach," SFB 649 Discussion Papers 2015-007, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    3. Ahbab Mohammad Fazle Rabbi & Stefano Mazzuco, 2021. "Mortality Forecasting with the Lee–Carter Method: Adjusting for Smoothing and Lifespan Disparity," European Journal of Population, Springer;European Association for Population Studies, vol. 37(1), pages 97-120, March.
    4. OlaOluwa Simon Yaya & Luis Alberiko Gil-Alana, 2020. "Modelling Long-Range Dependence and Non-linearity in the Infant Mortality Rates of African Countries," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 26(3), pages 303-315, August.
    5. Bravo, Jorge M. & Ayuso, Mercedes & Holzmann, Robert & Palmer, Edward, 2021. "Addressing the life expectancy gap in pension policy," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 200-221.
    6. Feng, Lingbing & Shi, Yanlin & Chang, Le, 2021. "Forecasting mortality with a hyperbolic spatial temporal VAR model," International Journal of Forecasting, Elsevier, vol. 37(1), pages 255-273.
    7. Basellini, Ugofilippo & Camarda, Carlo Giovanni & Booth, Heather, 2023. "Thirty years on: A review of the Lee–Carter method for forecasting mortality," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1033-1049.
    8. Ugofilippo Basellini & Søren Kjærgaard & Carlo Giovanni Camarda, 2020. "An age-at-death distribution approach to forecast cohort mortality," Working Papers axafx5_3agsuwaphvlfk, French Institute for Demographic Studies.
    9. Man Chung Fung & Gareth W. Peters & Pavel V. Shevchenko, 2016. "A unified approach to mortality modelling using state-space framework: characterisation, identification, estimation and forecasting," Papers 1605.09484, arXiv.org.
    10. Tickle Leonie & Booth Heather, 2014. "The Longevity Prospects of Australian Seniors: An Evaluation of Forecast Method and Outcome," Asia-Pacific Journal of Risk and Insurance, De Gruyter, vol. 8(2), pages 259-292, July.
    11. Yaya, OlaOluwa S & Adekoya, Oluwasegun B. & Babatunde, Oluwagbenga T., 2021. "Testing Fractional Persistence and Nonlinearity in Infant Mortality Rates of Asia Countries," MPRA Paper 109368, University Library of Munich, Germany.
    12. Jorge M. Uribe & Helena Chuliá & Montserrat Guillen, 2018. "Trends in the Quantiles of the Life Table Survivorship Function," European Journal of Population, Springer;European Association for Population Studies, vol. 34(5), pages 793-817, December.
    13. OlaOluwa S. Yaya & Luis A. Gil-Alana & Acheampong Y. Amoateng, 2019. "Under-5 Mortality Rates in G7 Countries: Analysis of Fractional Persistence, Structural Breaks and Nonlinear Time Trends," European Journal of Population, Springer;European Association for Population Studies, vol. 35(4), pages 675-694, October.
    14. Ana Debón & Steven Haberman & Francisco Montes & Edoardo Otranto, 2021. "Do Different Models Induce Changes in Mortality Indicators? That Is a Key Question for Extending the Lee-Carter Model," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    15. Basellini, Ugofilippo & Kjærgaard, Søren & Camarda, Carlo Giovanni, 2020. "An age-at-death distribution approach to forecast cohort mortality," Insurance: Mathematics and Economics, Elsevier, vol. 91(C), pages 129-143.
    16. F. Peters & J. P. Mackenbach & W. J. Nusselder, 2016. "Does the Impact of the Tobacco Epidemic Explain Structural Changes in the Decline of Mortality?," European Journal of Population, Springer;European Association for Population Studies, vol. 32(5), pages 687-702, December.
    17. Rob J Hyndman & Yijun Zeng & Han Lin Shang, 2020. "Forecasting the Old-Age Dependency Ratio to Determine a Sustainable Pension Age," Monash Econometrics and Business Statistics Working Papers 31/20, Monash University, Department of Econometrics and Business Statistics.
    18. Apostolos Bozikas & Georgios Pitselis, 2018. "An Empirical Study on Stochastic Mortality Modelling under the Age-Period-Cohort Framework: The Case of Greece with Applications to Insurance Pricing," Risks, MDPI, vol. 6(2), pages 1-34, April.
    19. Lenny Stoeldraijer & Coen van Duin & Leo van Wissen & Fanny Janssen, 2013. "Impact of different mortality forecasting methods and explicit assumptions on projected future life expectancy: The case of the Netherlands," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(13), pages 323-354.
    20. Han Lin Shang, 2012. "Point and interval forecasts of age-specific fertility rates: a comparison of functional principal component methods," Monash Econometrics and Business Statistics Working Papers 10/12, Monash University, Department of Econometrics and Business Statistics.

    More about this item

    Keywords

    Tikhonov regularisation; Smoothing; Forecasting; Ridge regression; PCA; LASSO; Maximum-margin matrix factorisation; Mortality rates; Sparse longitudinal data;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2014-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.