IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp11412.html
   My bibliography  Save this paper

Estimating the Associations between SNAP and Food Insecurity, Obesity, and Food Purchases with Imperfect Administrative Measures of Participation

Author

Listed:
  • Courtemanche, Charles

    (University of Kentucky)

  • Denteh, Augustine

    (Georgia State University)

  • Tchernis, Rusty

    (Georgia State University)

Abstract

Administrative data are considered the "gold standard" when measuring program participation, but little evidence exists on the potential problems with administrative records or their implications for econometric estimates. We explore issues with administrative data using the FoodAPS, a unique dataset that contains two different administrative measures of Supplemental Nutrition Assistance Pro-gram (SNAP) participation as well as a survey-based measure. We first document substantial ambiguity in the two administrative participation variables and show that they disagree with each other almost as often as they disagree with self-reported participation.Estimated participation and misreporting rates can be meaningfully sensitive to choices made to resolve this ambiguity and disagreement. We then document similar sensitivity in regression estimates of the associations between SNAP and food insecurity, obesity, and the Healthy Eating Index. These results serve as a cautionary tale about uncritically relying on linked administrative records when conducting program evaluation research.

Suggested Citation

  • Courtemanche, Charles & Denteh, Augustine & Tchernis, Rusty, 2018. "Estimating the Associations between SNAP and Food Insecurity, Obesity, and Food Purchases with Imperfect Administrative Measures of Participation," IZA Discussion Papers 11412, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp11412
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp11412.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Brent Kreider, 2010. "Regression Coefficient Identification Decay in The Presence of Infrequent Classification Errors," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1017-1023, November.
    2. Aigner, Dennis J., 1973. "Regression with a binary independent variable subject to errors of observation," Journal of Econometrics, Elsevier, vol. 1(1), pages 49-59, March.
    3. Lorenzo Almada & Ian McCarthy & Rusty Tchernis, 2016. "What Can We Learn about the Effects of Food Stamps on Obesity in the Presence of Misreporting?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 98(4), pages 997-1017.
    4. repec:mpr:mprres:7859 is not listed on IDEAS
    5. Almada, Lorenzo N. & Tchernis, Rusty, 2018. "Measuring effects of SNAP on obesity at the intensive margin," Economics & Human Biology, Elsevier, vol. 31(C), pages 150-163.
    6. Nguimkeu, Pierre & Denteh, Augustine & Tchernis, Rusty, 2019. "On the estimation of treatment effects with endogenous misreporting," Journal of Econometrics, Elsevier, vol. 208(2), pages 487-506.
    7. Craig Gundersen & Victor Oliveira, 2001. "The Food Stamp Program and Food Insufficiency," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 83(4), pages 875-887.
    8. Arthur Lewbel, 2007. "Estimation of Average Treatment Effects with Misclassification," Econometrica, Econometric Society, vol. 75(2), pages 537-551, March.
    9. Bruce Meyer & Robert Goerge, 2011. "Errors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation," Working Papers 11-14, Center for Economic Studies, U.S. Census Bureau.
    10. Brent Kreider & John V. Pepper & Craig Gundersen & Dean Jolliffe, 2012. "Identifying the Effects of SNAP (Food Stamps) on Child Health Outcomes When Participation Is Endogenous and Misreported," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(499), pages 958-975, September.
    11. Nord, Mark & Prell, Mark, 2011. "Food Security Improved Following the 2009 ARRA Increase in SNAP Benefits," Economic Research Report 262242, United States Department of Agriculture, Economic Research Service.
    12. Elton Mykerezi & Bradford Mills, 2010. "The Impact of Food Stamp Program Participation on Household Food Insecurity," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(5), pages 1379-1391.
    13. Meyer, Bruce D. & Mittag, Nikolas & Goerge, Robert M., 2018. "Errors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation," IZA Discussion Papers 11776, Institute of Labor Economics (IZA).
    14. Chad D. Meyerhoefer & Yuriy Pylypchuk, 2008. "Does Participation in the Food Stamp Program Increase the Prevalence of Obesity and Health Care Spending?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 90(2), pages 287-305.
    15. Bruce D. Meyer & Wallace K. C. Mok & James X. Sullivan, 2015. "Household Surveys in Crisis," Journal of Economic Perspectives, American Economic Association, vol. 29(4), pages 199-226, Fall.
    16. Mittag, Nikolas, 2016. "Correcting for Misreporting of Government Benefits," IZA Discussion Papers 10266, Institute of Labor Economics (IZA).
    17. Bound, John & Brown, Charles & Mathiowetz, Nancy, 2001. "Measurement error in survey data," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 5, chapter 59, pages 3705-3843, Elsevier.
    18. Richard Volpe & Abigail Okrent & Ephraim Leibtag, 2013. "The Effect of Supercenter-format Stores on the Healthfulness of Consumers' Grocery Purchases," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 95(3), pages 568-589.
    19. Parke Wilde & Mark Nord, 2005. "The Effect of Food Stamps on Food Security: A Panel Data Approach," Review of Agricultural Economics, Agricultural and Applied Economics Association, vol. 27(3), pages 425-432.
    20. Gregory, Christian & Ver Ploeg, Michele & Andrews, Margaret & Coleman-Jensen, Alisha, 2013. "Supplemental Nutrition Assistance Program (SNAP) Participation Leads to Modest Changes in Diet Quality," Economic Research Report 262225, United States Department of Agriculture, Economic Research Service.
    21. repec:wly:soecon:v:82:4:y:2016:p:1266-1310 is not listed on IDEAS
    22. Huffman, Sonya Kostova & Jensen, Helen H., 2003. "Do Food Assistance Programs Improve Household Food Security?: Recent Evidence From The United States," 2003 Annual meeting, July 27-30, Montreal, Canada 22219, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    23. Nikolas Mittag, 2013. "A Method Of Correcting For Misreporting Applied To The Food Stamp Program," Working Papers 13-28, Center for Economic Studies, U.S. Census Bureau.
    24. James Mabli & Jim Ohls & Lisa Dragoset & Laura Castner & Betsy Santos, "undated". "Measuring the Effect of Supplemental Nutrition Assistance Program (SNAP) Participation on Food Security," Mathematica Policy Research Reports 69d901432c7a46779666a240a, Mathematica Policy Research.
    25. Lucie Schmidt & Lara Shore-Sheppard & Tara Watson, 2016. "The Effect of Safety-Net Programs on Food Insecurity," Journal of Human Resources, University of Wisconsin Press, vol. 51(3), pages 589-614.
    26. Meyer, Bruce D. & Mittag, Nikolas, 2018. "Misreporting of Government Transfers: How Important Are Survey Design and Geography?," IZA Discussion Papers 12038, Institute of Labor Economics (IZA).
    27. AIGNER, Dennis J., 1973. "Regression with a binary independent variable subject to errors of observation," LIDAM Reprints CORE 130, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    28. Marianne P. Bitler & Janet Currie, 2005. "Does WIC work? The effects of WIC on pregnancy and birth outcomes," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 24(1), pages 73-91.
    29. repec:mpr:mprres:8081 is not listed on IDEAS
    30. repec:mpr:mprres:7860 is not listed on IDEAS
    31. Janet Currie, 2003. "US Food and Nutrition Programs," NBER Chapters, in: Means-Tested Transfer Programs in the United States, pages 199-290, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Meyer, Bruce D. & Mittag, Nikolas & Wu, Derek, 2024. "Race, Ethnicity, and Measurement Error," IZA Discussion Papers 17349, Institute of Labor Economics (IZA).
    2. Bruce D. Meyer & Nikolas Mittag & Derek Wu, 2024. "Race, Ethnicity, and Measurement Error," NBER Chapters, in: Race, Ethnicity, and Economic Statistics for the 21st Century, National Bureau of Economic Research, Inc.
    3. Brent Kreider & John V. Pepper & Manan Roy, 2020. "Does The Women, Infants, And Children Program Improve Infant Health Outcomes?," Economic Inquiry, Western Economic Association International, vol. 58(4), pages 1731-1756, October.
    4. Helen H. Jensen & Brent Kreider & Oleksandr Zhylyevskyy, 2019. "Investigating Treatment Effects of Participating Jointly in SNAP and WIC when the Treatment Is Validated Only for SNAP," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 124-155, July.
    5. Meyer, Bruce D. & Mittag, Nikolas, 2018. "Misreporting of Government Transfers: How Important Are Survey Design and Geography?," IZA Discussion Papers 12038, Institute of Labor Economics (IZA).
    6. Wu, Kaidi, 2020. "The Effect of SNAP on Dietary Quality: Evidence from FoodAPS," 2020 Annual Meeting, July 26-28, Kansas City, Missouri 304462, Agricultural and Applied Economics Association.
    7. Sarah Tahamont & Zubin Jelveh & Aaron Chalfin & Shi Yan & Benjamin Hansen, 2019. "Administrative Data Linking and Statistical Power Problems in Randomized Experiments," NBER Working Papers 25657, National Bureau of Economic Research, Inc.
    8. Marianne P. Bitler & Christian Gregory, 2019. "Food Access, Program Participation, and Health: Research Using FoodAPS," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 9-17, July.
    9. Ding Liu & Daniel L. Millimet, 2021. "Bounding the joint distribution of disability and employment with misclassification," Health Economics, John Wiley & Sons, Ltd., vol. 30(7), pages 1628-1647, July.
    10. Nathalie Mathieu‐Bolh, 2022. "The elusive link between income and obesity," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 935-968, September.
    11. Luis Ayala & Ana Pérez & Mercedes Prieto-Alaiz, 2022. "The impact of different data sources on the level and structure of income inequality," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 13(3), pages 583-611, September.
    12. Bruce D. Meyer & Nikolas Mittag, 2019. "Misreporting of Government Transfers: How Important Are Survey Design and Geography?," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 230-253, July.
    13. Jinglin Feng & Linlin Fan & Edward C. Jaenicke, 2024. "The distributional impact of SNAP on dietary quality," Agricultural Economics, International Association of Agricultural Economists, vol. 55(1), pages 104-139, January.
    14. Celhay, Pablo & Meyer, Bruce D. & Mittag, Nikolas, 2024. "What leads to measurement errors? Evidence from reports of program participation in three surveys," Journal of Econometrics, Elsevier, vol. 238(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nguimkeu, Pierre & Denteh, Augustine & Tchernis, Rusty, 2019. "On the estimation of treatment effects with endogenous misreporting," Journal of Econometrics, Elsevier, vol. 208(2), pages 487-506.
    2. Tommasi, Denni & Zhang, Lina, 2024. "Bounding program benefits when participation is misreported," Journal of Econometrics, Elsevier, vol. 238(1).
    3. Kasahara, Hiroyuki & Shimotsu, Katsumi, 2022. "Identification Of Regression Models With A Misclassified And Endogenous Binary Regressor," Econometric Theory, Cambridge University Press, vol. 38(6), pages 1117-1139, December.
    4. Lorenzo Almada & Ian McCarthy & Rusty Tchernis, 2016. "What Can We Learn about the Effects of Food Stamps on Obesity in the Presence of Misreporting?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 98(4), pages 997-1017.
    5. Helen H. Jensen & Brent Kreider & Oleksandr Zhylyevskyy, 2019. "Investigating Treatment Effects of Participating Jointly in SNAP and WIC when the Treatment Is Validated Only for SNAP," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 124-155, July.
    6. Susan Chen & Le Wang, 2021. "SNAP participation, diet quality, and obesity: robust evidence with estimation techniques without external instrumental variables," Empirical Economics, Springer, vol. 61(3), pages 1641-1667, September.
    7. Kyung Min Kang & Robert A. Moffitt, 2019. "The Effect of SNAP and School Food Programs on Food Security, Diet Quality, and Food Spending: Sensitivity to Program Reporting Error," Southern Economic Journal, John Wiley & Sons, vol. 86(1), pages 156-201, July.
    8. Akanksha Negi & Digvijay Singh Negi, 2022. "Difference-in-Differences with a Misclassified Treatment," Papers 2208.02412, arXiv.org.
    9. Swann, Christopher A., 2017. "Household history, SNAP participation, and food insecurity," Food Policy, Elsevier, vol. 73(C), pages 1-9.
    10. Martijn van Hasselt & Christopher R. Bollinger & Jeremy W. Bray, 2022. "A Bayesian approach to account for misclassification in prevalence and trend estimation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 351-367, March.
    11. Steven J. Haider & Melvin Stephens Jr., 2020. "Correcting for Misclassified Binary Regressors Using Instrumental Variables," NBER Working Papers 27797, National Bureau of Economic Research, Inc.
    12. Francis J. DiTraglia & Camilo Garcia-Jimeno, 2020. "Identifying the effect of a mis-classified, binary, endogenous regressor," Papers 2011.07272, arXiv.org.
    13. Bronchetti, Erin T. & Christensen, Garret & Hoynes, Hilary W., 2019. "Local food prices, SNAP purchasing power, and child health," Journal of Health Economics, Elsevier, vol. 68(C).
    14. DiTraglia, Francis J. & García-Jimeno, Camilo, 2019. "Identifying the effect of a mis-classified, binary, endogenous regressor," Journal of Econometrics, Elsevier, vol. 209(2), pages 376-390.
    15. Acerenza, Santiago & Ban, Kyunghoon & Kedagni, Desire, 2021. "Marginal Treatment Effects with Misclassified Treatment," ISU General Staff Papers 202106180700001132, Iowa State University, Department of Economics.
    16. Brent Kreider & John V. Pepper & Manan Roy, 2020. "Does The Women, Infants, And Children Program Improve Infant Health Outcomes?," Economic Inquiry, Western Economic Association International, vol. 58(4), pages 1731-1756, October.
    17. Battistin, Erich & De Nadai, Michele & Sianesi, Barbara, 2014. "Misreported schooling, multiple measures and returns to educational qualifications," Journal of Econometrics, Elsevier, vol. 181(2), pages 136-150.
    18. Wang, Julia Shu-Huah & Zhao, Xi & Nam, Jaehyun, 2021. "The effects of welfare participation on parenting stress and parental engagement using an instrumental variables approach: Evidence from the Supplemental Nutrition Assistance Program," Children and Youth Services Review, Elsevier, vol. 121(C).
    19. Pourya Valizadeh & Travis A Smith, 2020. "How Did The American Recovery and Reinvestment Act Affect the Material Well‐Being of SNAP Participants? A Distributional Approach," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 42(3), pages 455-476, September.
    20. Almada, Lorenzo N. & Tchernis, Rusty, 2018. "Measuring effects of SNAP on obesity at the intensive margin," Economics & Human Biology, Elsevier, vol. 31(C), pages 150-163.

    More about this item

    Keywords

    food expenditures; food purchases; body mass index; obesity; food insecurity; SNAP; food stamps; Supplemental Nutrition Assistance Program; healthy eating index; misreporting; measurement error;
    All these keywords.

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • H51 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Health
    • I12 - Health, Education, and Welfare - - Health - - - Health Behavior
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp11412. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.