IDEAS home Printed from https://ideas.repec.org/a/gam/jstats/v4y2021i4p54-942d680930.html
   My bibliography  Save this article

Estimating the RMSE of Small Area Estimates without the Tears

Author

Listed:
  • Diane Hindmarsh

    (National Institute for Applied Statistics Research Australia, School of Mathematics and Applied Statistics, Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, NSW 2252, Australia)

  • David Steel

    (National Institute for Applied Statistics Research Australia, School of Mathematics and Applied Statistics, Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, NSW 2252, Australia)

Abstract

Small area estimation (SAE) methods can provide information that conventional direct survey estimation methods cannot. The use of small area estimates based on linear and generalized linear mixed models is still very limited, possibly because of the perceived complexity of estimating the root mean square errors (RMSEs) of the estimates. This paper outlines a study used to determine the conditions under which the estimated RMSEs, produced as part of statistical output (‘plug-in’ estimates of RMSEs) could be considered appropriate for a practical application of SAE methods where one of the main requirements was to use SAS software. We first show that the estimated RMSEs created using an EBLUP model in SAS and those obtained using a parametric bootstrap are similar to the published estimated RMSEs for the corn data in the seminal paper by Battese, Harter and Fuller. We then compare plug-in estimates of RMSEs from SAS procedures used to create EBLUP and EBP estimators against estimates of RMSEs obtained from a parametric bootstrap. For this comparison we created estimates of current smoking in males for 153 local government areas (LGAs) using data from the NSW Population Health Survey in Australia. Demographic variables from the survey data were included as covariates, with LGA-level population proportions, obtained mainly from the Australian Census used for prediction. For the EBLUP, the estimated plug-in estimates of RMSEs can be used, provided the sample size for the small area is more than seven. For the EBP, the plug-in estimates of RMSEs are suitable for all in-sample areas; out-of-sample areas need to use estimated RMSEs that use the parametric bootstrap.

Suggested Citation

  • Diane Hindmarsh & David Steel, 2021. "Estimating the RMSE of Small Area Estimates without the Tears," Stats, MDPI, vol. 4(4), pages 1-12, November.
    • Handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:54-942:d:680930
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-905X/4/4/54/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-905X/4/4/54/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gonzalez-Manteiga, W. & Lombardia, M.J. & Molina, I. & Morales, D. & Santamaria, L., 2007. "Estimation of the mean squared error of predictors of small area linear parameters under a logistic mixed model," Computational Statistics & Data Analysis, Elsevier, vol. 51(5), pages 2720-2733, February.
    2. Isabel Molina & Ewa Strzalkowska‐Kominiak, 2020. "Estimation of proportions in small areas: application to the labour force using the Swiss Census Structural Survey," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 281-310, January.
    3. Soumendra N. Lahiri & Tapabrata Maiti & Myron Katzoff & Van Parsons, 2007. "Resampling-based empirical prediction: an application to small area estimation," Biometrika, Biometrika Trust, vol. 94(2), pages 469-485.
    4. Peter Hall & Tapabrata Maiti, 2006. "On parametric bootstrap methods for small area prediction," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(2), pages 221-238, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Miguel Boubeta & María José Lombardía & Domingo Morales, 2016. "Empirical best prediction under area-level Poisson mixed models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(3), pages 548-569, September.
    2. Angelo Moretti, 2023. "Estimation of small area proportions under a bivariate logistic mixed model," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3663-3684, August.
    3. Tomasz .Zk{a}d{l}o & Adam Chwila, 2024. "A step towards the integration of machine learning and small area estimation," Papers 2402.07521, arXiv.org.
    4. Isabel Molina & Ayoub Saei & M. José Lombardía, 2007. "Small area estimates of labour force participation under a multinomial logit mixed model," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 975-1000, October.
    5. G. Bertarelli & R. Chambers & N. Salvati, 2021. "Outlier robust small domain estimation via bias correction and robust bootstrapping," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 331-357, March.
    6. Ralf Münnich & Jan Burgard & Martin Vogt, 2013. "Small Area-Statistik: Methoden und Anwendungen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 6(3), pages 149-191, March.
    7. Isabel Molina & Nicola Salvati & Monica Pratesi, 2009. "Bootstrap for estimating the MSE of the Spatial EBLUP," Computational Statistics, Springer, vol. 24(3), pages 441-458, August.
    8. Patrick Krennmair & Timo Schmid, 2022. "Flexible domain prediction using mixed effects random forests," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1865-1894, November.
    9. repec:csb:stintr:v:17:y:2016:i:1:p:9-24 is not listed on IDEAS
    10. María Dolores Esteban & María José Lombardía & Esther López-Vizcaíno & Domingo Morales & Agustín Pérez, 2020. "Small area estimation of proportions under area-level compositional mixed models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(3), pages 793-818, September.
    11. Erciulescu Andreea L. & Fuller Wayne A., 2016. "Small Area Prediction Under Alternative Model Specifications," Statistics in Transition New Series, Statistics Poland, vol. 17(1), pages 9-24, March.
    12. Ralf Münnich & Jan Pablo Burgard & Siegfried Gabler & Matthias Ganninger & Jan-Philipp Kolb, 2016. "Small Area Estimation In The German Census 2011," Statistics in Transition New Series, Polish Statistical Association, vol. 17(1), pages 25-40, March.
    13. Tomasz Ża̧dło, 2015. "On longitudinal moving average model for prediction of subpopulation total," Statistical Papers, Springer, vol. 56(3), pages 749-771, August.
    14. Peter Hall & Tapabrata Maiti, 2008. "Non‐parametric inference for clustered binary and count data when only summary information is available," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(4), pages 725-738, September.
    15. M. D. Ugarte & A. F. Militino & T. Goicoa, 2008. "Adjusting economic estimates in business surveys," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(11), pages 1253-1265.
    16. Cristina Rueda & José Menéndez & Federico Gómez, 2010. "Small area estimators based on restricted mixed models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 19(3), pages 558-579, November.
    17. Molina, Isabel & Salvati, Nicola & Pratesi, Monica, 2007. "Bootstrap for estimating the mean squared error of the spatial EBLUP," DES - Working Papers. Statistics and Econometrics. WS ws073408, Universidad Carlos III de Madrid. Departamento de Estadística.
    18. Torabi, Mahmoud & Rao, J.N.K., 2014. "On small area estimation under a sub-area level model," Journal of Multivariate Analysis, Elsevier, vol. 127(C), pages 36-55.
    19. María Dolores Esteban & María José Lombardía & Esther López‐Vizcaíno & Domingo Morales & Agustín Pérez, 2022. "Empirical best prediction of small area bivariate parameters," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(4), pages 1699-1727, December.
    20. Boubeta, Miguel & Lombardía, María José & Morales, Domingo, 2017. "Poisson mixed models for studying the poverty in small areas," Computational Statistics & Data Analysis, Elsevier, vol. 107(C), pages 32-47.
    21. María José Lombardía & Esther López-Vizcaíno & Cristina Rueda, 2021. "Selection model for domains across time: application to labour force survey by economic activities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(1), pages 228-254, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:54-942:d:680930. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.