IDEAS home Printed from https://ideas.repec.org/a/gam/jstats/v4y2021i4p54-942d680930.html
   My bibliography  Save this article

Estimating the RMSE of Small Area Estimates without the Tears

Author

Listed:
  • Diane Hindmarsh

    (National Institute for Applied Statistics Research Australia, School of Mathematics and Applied Statistics, Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, NSW 2252, Australia)

  • David Steel

    (National Institute for Applied Statistics Research Australia, School of Mathematics and Applied Statistics, Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, NSW 2252, Australia)

Abstract

Small area estimation (SAE) methods can provide information that conventional direct survey estimation methods cannot. The use of small area estimates based on linear and generalized linear mixed models is still very limited, possibly because of the perceived complexity of estimating the root mean square errors (RMSEs) of the estimates. This paper outlines a study used to determine the conditions under which the estimated RMSEs, produced as part of statistical output (‘plug-in’ estimates of RMSEs) could be considered appropriate for a practical application of SAE methods where one of the main requirements was to use SAS software. We first show that the estimated RMSEs created using an EBLUP model in SAS and those obtained using a parametric bootstrap are similar to the published estimated RMSEs for the corn data in the seminal paper by Battese, Harter and Fuller. We then compare plug-in estimates of RMSEs from SAS procedures used to create EBLUP and EBP estimators against estimates of RMSEs obtained from a parametric bootstrap. For this comparison we created estimates of current smoking in males for 153 local government areas (LGAs) using data from the NSW Population Health Survey in Australia. Demographic variables from the survey data were included as covariates, with LGA-level population proportions, obtained mainly from the Australian Census used for prediction. For the EBLUP, the estimated plug-in estimates of RMSEs can be used, provided the sample size for the small area is more than seven. For the EBP, the plug-in estimates of RMSEs are suitable for all in-sample areas; out-of-sample areas need to use estimated RMSEs that use the parametric bootstrap.

Suggested Citation

  • Diane Hindmarsh & David Steel, 2021. "Estimating the RMSE of Small Area Estimates without the Tears," Stats, MDPI, vol. 4(4), pages 1-12, November.
    • Handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:54-942:d:680930
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-905X/4/4/54/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-905X/4/4/54/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gonzalez-Manteiga, W. & Lombardia, M.J. & Molina, I. & Morales, D. & Santamaria, L., 2007. "Estimation of the mean squared error of predictors of small area linear parameters under a logistic mixed model," Computational Statistics & Data Analysis, Elsevier, vol. 51(5), pages 2720-2733, February.
    2. Isabel Molina & Ewa Strzalkowska‐Kominiak, 2020. "Estimation of proportions in small areas: application to the labour force using the Swiss Census Structural Survey," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 281-310, January.
    3. Soumendra N. Lahiri & Tapabrata Maiti & Myron Katzoff & Van Parsons, 2007. "Resampling-based empirical prediction: an application to small area estimation," Biometrika, Biometrika Trust, vol. 94(2), pages 469-485.
    4. Peter Hall & Tapabrata Maiti, 2006. "On parametric bootstrap methods for small area prediction," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(2), pages 221-238, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. G. Bertarelli & R. Chambers & N. Salvati, 2021. "Outlier robust small domain estimation via bias correction and robust bootstrapping," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 331-357, March.
    2. Miguel Boubeta & María José Lombardía & Domingo Morales, 2016. "Empirical best prediction under area-level Poisson mixed models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(3), pages 548-569, September.
    3. Angelo Moretti, 2023. "Estimation of small area proportions under a bivariate logistic mixed model," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3663-3684, August.
    4. Ralf Münnich & Jan Burgard & Martin Vogt, 2013. "Small Area-Statistik: Methoden und Anwendungen," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 6(3), pages 149-191, March.
    5. Isabel Molina & Nicola Salvati & Monica Pratesi, 2009. "Bootstrap for estimating the MSE of the Spatial EBLUP," Computational Statistics, Springer, vol. 24(3), pages 441-458, August.
    6. Tomasz .Zk{a}d{l}o & Adam Chwila, 2024. "A step towards the integration of machine learning and small area estimation," Papers 2402.07521, arXiv.org.
    7. Isabel Molina & Ayoub Saei & M. José Lombardía, 2007. "Small area estimates of labour force participation under a multinomial logit mixed model," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 975-1000, October.
    8. Kubokawa, Tatsuya & Nagashima, Bui, 2012. "Parametric bootstrap methods for bias correction in linear mixed models," Journal of Multivariate Analysis, Elsevier, vol. 106(C), pages 1-16.
    9. González-Manteiga, W. & Lombardi­a, M.J. & Molina, I. & Morales, D. & Santamari­a, L., 2008. "Analytic and bootstrap approximations of prediction errors under a multivariate Fay-Herriot model," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5242-5252, August.
    10. Patrick Krennmair & Timo Schmid, 2022. "Flexible domain prediction using mixed effects random forests," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1865-1894, November.
    11. repec:csb:stintr:v:17:y:2016:i:1:p:9-24 is not listed on IDEAS
    12. María Dolores Esteban & María José Lombardía & Esther López-Vizcaíno & Domingo Morales & Agustín Pérez, 2020. "Small area estimation of proportions under area-level compositional mixed models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(3), pages 793-818, September.
    13. Erciulescu Andreea L. & Fuller Wayne A., 2016. "Small Area Prediction Under Alternative Model Specifications," Statistics in Transition New Series, Statistics Poland, vol. 17(1), pages 9-24, March.
    14. Torabi, Mahmoud, 2012. "Small area estimation using survey weights under a nested error linear regression model with structural measurement error," Journal of Multivariate Analysis, Elsevier, vol. 109(C), pages 52-60.
    15. Piyush Kant Rai & Sarla Pareek & Hemlata Joshi, 2017. "Met And Unmet Need For Contraception: Small Area Estimation For Rajasthan State Of India," Statistics in Transition New Series, Polish Statistical Association, vol. 18(2), pages 329-360, June.
    16. Hukum Chandra, 2021. "District-Level Estimates of Poverty Incidence for the State of West Bengal in India: Application of Small Area Estimation Technique Combining NSSO Survey and Census Data," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 19(2), pages 375-391, June.
    17. Ralf Münnich & Jan Pablo Burgard & Siegfried Gabler & Matthias Ganninger & Jan-Philipp Kolb, 2016. "Small Area Estimation In The German Census 2011," Statistics in Transition New Series, Polish Statistical Association, vol. 17(1), pages 25-40, March.
    18. Arnab Bhattacharjee & Eduardo Castro & Taps Maiti & João Marques, 2014. "Endogenous spatial structure and delineation of submarkets: A new framework with application to housing markets," SEEC Discussion Papers 1403, Spatial Economics and Econometrics Centre, Heriot Watt University.
    19. Żądło Tomasz, 2017. "On Asymmetry of Prediction Errors in Small Area Estimation," Statistics in Transition New Series, Statistics Poland, vol. 18(3), pages 413-432, September.
    20. Sanjoy Sinha & Abdus Sattar, 2015. "Inference in semi-parametric spline mixed models for longitudinal data," METRON, Springer;Sapienza Università di Roma, vol. 73(3), pages 377-395, December.
    21. Nicholas Longford, 2014. "Policy-related small-area estimation," Economics Working Papers 1427, Department of Economics and Business, Universitat Pompeu Fabra.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:54-942:d:680930. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.