IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v49y2020i3p637-671.html
   My bibliography  Save this article

Optimizing Count Responses in Surveys: A Machine-learning Approach

Author

Listed:
  • Qiang Fu
  • Xin Guo
  • Kenneth C. Land

Abstract

Count responses with grouping and right censoring have long been used in surveys to study a variety of behaviors, status, and attitudes. Yet grouping or right-censoring decisions of count responses still rely on arbitrary choices made by researchers. We develop a new method for evaluating grouping and right-censoring decisions of count responses from a (semisupervised) machine-learning perspective. This article uses Poisson multinomial mixture models to conceptualize the data-generating process of count responses with grouping and right censoring and demonstrates the link between grouping-scheme choices and asymptotic distributions of the Poisson mixture. To search for the optimal grouping scheme maximizing objective functions of the Fisher information (matrix), an innovative three-step M algorithm is then proposed to process infinitely many grouping schemes based on Bayesian A-, D-, and E-optimalities. A new R package is developed to implement this algorithm and evaluate grouping schemes of count responses. Results show that an optimal grouping scheme not only leads to a more efficient sampling design but also outperforms a nonoptimal one even if the latter has more groups.

Suggested Citation

  • Qiang Fu & Xin Guo & Kenneth C. Land, 2020. "Optimizing Count Responses in Surveys: A Machine-learning Approach," Sociological Methods & Research, , vol. 49(3), pages 637-671, August.
    • Handle: RePEc:sae:somere:v:49:y:2020:i:3:p:637-671
    DOI: 10.1177/0049124117747302
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124117747302
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124117747302?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Qiang Fu & Kenneth C. Land & Vicki L. Lamb, 2016. "Violent Physical Bullying Victimization at School: Has There Been a Recent Increase in Exposure or Intensity? An Age-Period-Cohort Analysis in the United States, 1991 to 2012," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 9(2), pages 485-513, June.
    2. Daniel B. Hall, 2000. "Zero-Inflated Poisson and Binomial Regression with Random Effects: A Case Study," Biometrics, The International Biometric Society, vol. 56(4), pages 1030-1039, December.
    3. Bharati Basu & Felix Famoye, 2004. "Domestic violence against women, and their economic dependence: A count data analysis," Review of Political Economy, Taylor & Francis Journals, vol. 16(4), pages 457-472.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qiang Fu & Tian‐Yi Zhou & Xin Guo, 2021. "Modified Poisson regression analysis of grouped and right‐censored counts," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(4), pages 1347-1367, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    2. Augusto Mendoza Calderón, 2017. "El Efecto del Empleo sobre la Violencia Doméstica: Evidencia para las Mujeres Peruanas," Working Papers 99, Peruvian Economic Association.
    3. Cho, Daegon & Hwang, Youngdeok & Park, Jongwon, 2018. "More buzz, more vibes: Impact of social media on concert distribution," Journal of Economic Behavior & Organization, Elsevier, vol. 156(C), pages 103-113.
    4. Greene, William, 2007. "Functional Form and Heterogeneity in Models for Count Data," Foundations and Trends(R) in Econometrics, now publishers, vol. 1(2), pages 113-218, August.
    5. Das, Ujjwal & Das, Kalyan, 2018. "Inference on zero inflated ordinal models with semiparametric link," Computational Statistics & Data Analysis, Elsevier, vol. 128(C), pages 104-115.
    6. Niklas Elert, 2014. "What determines entry? Evidence from Sweden," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 53(1), pages 55-92, August.
    7. Sarah Brown & Pulak Ghosh & Bhuvanesh Pareek & Karl Taylor, 2017. "Financial Hardship and Saving Behaviour: Bayesian Analysis of British Panel Data," Working Papers 2017011, The University of Sheffield, Department of Economics.
    8. Hua Xin & Yuhlong Lio & Hsien-Ching Chen & Tzong-Ru Tsai, 2024. "Zero-Inflated Binary Classification Model with Elastic Net Regularization," Mathematics, MDPI, vol. 12(19), pages 1-17, September.
    9. Yanling Li & Zita Oravecz & Shuai Zhou & Yosef Bodovski & Ian J. Barnett & Guangqing Chi & Yuan Zhou & Naomi P. Friedman & Scott I. Vrieze & Sy-Miin Chow, 2022. "Bayesian Forecasting with a Regime-Switching Zero-Inflated Multilevel Poisson Regression Model: An Application to Adolescent Alcohol Use with Spatial Covariates," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 376-402, June.
    10. Payandeh Najafabadi Amir T. & MohammadPour Saeed, 2018. "A k-Inflated Negative Binomial Mixture Regression Model: Application to Rate–Making Systems," Asia-Pacific Journal of Risk and Insurance, De Gruyter, vol. 12(2), pages 1-31, July.
    11. Abbas Moghimbeigi & Mohammed Reza Eshraghian & Kazem Mohammad & Brian Mcardle, 2008. "Multilevel zero-inflated negative binomial regression modeling for over-dispersed count data with extra zeros," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(10), pages 1193-1202.
    12. Yanlin Tang & Liya Xiang & Zhongyi Zhu, 2014. "Risk Factor Selection in Rate Making: EM Adaptive LASSO for Zero‐Inflated Poisson Regression Models," Risk Analysis, John Wiley & Sons, vol. 34(6), pages 1112-1127, June.
    13. Harald Oberhofer & Michael Pfaffermayr, 2014. "Two-Part Models for Fractional Responses Defined as Ratios of Integers," Econometrics, MDPI, vol. 2(3), pages 1-22, September.
    14. Zhang, Tonglin, 2019. "General Gaussian estimation," Journal of Multivariate Analysis, Elsevier, vol. 169(C), pages 234-247.
    15. Damgaard, Christian, 2008. "Modelling pin-point plant cover data along an environmental gradient," Ecological Modelling, Elsevier, vol. 214(2), pages 404-410.
    16. Xie, Feng-Chang & Wei, Bo-Cheng & Lin, Jin-Guan, 2009. "Score tests for zero-inflated generalized Poisson mixed regression models," Computational Statistics & Data Analysis, Elsevier, vol. 53(9), pages 3478-3489, July.
    17. Soutik Ghosal & Timothy S. Lau & Jeremy Gaskins & Maiying Kong, 2020. "A hierarchical mixed effect hurdle model for spatiotemporal count data and its application to identifying factors impacting health professional shortages," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1121-1144, November.
    18. Liu, Juxin & Ma, Yanyuan & Johnstone, Jill, 2020. "A goodness-of-fit test for zero-inflated Poisson mixed effects models in tree abundance studies," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    19. Robert Richardson, 2022. "Spatial Generalized Linear Models with Non-Gaussian Translation Processes," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(1), pages 4-21, March.
    20. Ren, Haiying & Zhao, Yuhui, 2021. "Technology opportunity discovery based on constructing, evaluating, and searching knowledge networks," Technovation, Elsevier, vol. 101(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:49:y:2020:i:3:p:637-671. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.