IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v35y2011i3p269-303.html
   My bibliography  Save this article

Methodological Considerations in Using Complex Survey Data

Author

Listed:
  • Debbie L. Hahs-Vaughn
  • Christine M. McWayne
  • Rebecca J. Bulotsky-Shearer
  • Xiaoli Wen
  • Ann-Marie Faria

Abstract

Complex survey data are collected by means other than simple random samples. This creates two analytical issues: nonindependence and unequal selection probability. Failing to address these issues results in underestimated standard errors and biased parameter estimates. Using data from the nationally representative Head Start Family and Child Experiences Survey (FACES; 1997 and 2000 cohorts), three diverse multilevel models are presented that illustrate differences in results depending on addressing or ignoring the complex sampling issues. Limitations of using complex survey data are reported, along with recommendations for reporting complex sample results.

Suggested Citation

  • Debbie L. Hahs-Vaughn & Christine M. McWayne & Rebecca J. Bulotsky-Shearer & Xiaoli Wen & Ann-Marie Faria, 2011. "Methodological Considerations in Using Complex Survey Data," Evaluation Review, , vol. 35(3), pages 269-303, June.
    • Handle: RePEc:sae:evarev:v:35:y:2011:i:3:p:269-303
    DOI: 10.1177/0193841X11412071
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X11412071
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X11412071?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Vaughn, Gerald F., 1998. "In Review," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 13(2), pages 1-1.
    2. Schenker N. & Gentleman J. F., 2001. "On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals," The American Statistician, American Statistical Association, vol. 55, pages 182-186, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kull, Melissa A. & Coley, Rebekah Levine, 2015. "Early physical health conditions and school readiness skills in a prospective birth cohort of U.S. children," Social Science & Medicine, Elsevier, vol. 142(C), pages 145-153.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Domínguez-Torreiro, Marcos & Soliño, Mario, 2011. "Provided and perceived status quo in choice experiments: Implications for valuing the outputs of multifunctional rural areas," Ecological Economics, Elsevier, vol. 70(12), pages 2523-2531.
    2. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    3. Amy E. Wagler, 2014. "Confidence Intervals for Assessing Heterogeneity in Generalized Linear Mixed Models," Journal of Educational and Behavioral Statistics, , vol. 39(3), pages 167-179, June.
    4. Laura Lindberg & Kathryn Kost & Isaac Maddow-Zimet & Sheila Desai & Mia Zolna, 2020. "Abortion Reporting in the United States: An Assessment of Three National Fertility Surveys," Demography, Springer;Population Association of America (PAA), vol. 57(3), pages 899-925, June.
    5. Yuqing Feng & Jing Wei & Maogui Hu & Chengdong Xu & Tao Li & Jinfeng Wang & Wei Chen, 2022. "Lagged Effects of Exposure to Air Pollutants on the Risk of Pulmonary Tuberculosis in a Highly Polluted Region," IJERPH, MDPI, vol. 19(9), pages 1-13, May.
    6. Sabine D Klein & Loredana Torchetti & Martin Frei-Erb & Ursula Wolf, 2015. "Usage of Complementary Medicine in Switzerland: Results of the Swiss Health Survey 2012 and Development Since 2007," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-10, October.
    7. Carson Reeling & Leah H. Palm-Forster & Richard T. Melstrom, 2019. "Policy Instruments and Incentives for Coordinated Habitat Conservation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(3), pages 791-813, July.
    8. Polina Zvavitch & Michael S. Rendall & Constanza Hurtado-Acuna & Rachel M. Shattuck, 2021. "Contraceptive Consistency and Poverty After Birth," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 40(6), pages 1277-1311, December.
    9. El Benni, Nadja & Finger, Robert, 2013. "The effect of agricultural policy reforms on income inequality in Swiss agriculture - An analysis for valley, hill and mountain regions," Journal of Policy Modeling, Elsevier, vol. 35(4), pages 638-651.
    10. Colagrossi, Marco & d’Hombres, Béatrice & Schnepf, Sylke V, 2020. "Like (grand)parent, like child? Multigenerational mobility across the EU," European Economic Review, Elsevier, vol. 130(C).
    11. Hagger, Martin S. & Polet, Juho & Lintunen, Taru, 2018. "The reasoned action approach applied to health behavior: Role of past behavior and tests of some key moderators using meta-analytic structural equation modeling," Social Science & Medicine, Elsevier, vol. 213(C), pages 85-94.
    12. Andrew M. Johnson & Tahirou Abdoulaye & Bamikole Ayedun & Joan R. Fulton & Nicole J. Olynk Widmar & Akande Adebowale & Ranajit Bandyopadhyay & Victor Manyong, 2020. "Willingness to pay of Nigerian poultry producers and feed millers for aflatoxin‐safe maize," Agribusiness, John Wiley & Sons, Ltd., vol. 36(2), pages 299-317, April.
    13. McKendree, Melissa G.S. & Tonsor, Glynn T. & Schulz, Lee L., 2021. "Management of Multiple Sources of Risk in Livestock Production," Journal of Agricultural and Applied Economics, Cambridge University Press, vol. 53(1), pages 75-93, February.
    14. Vigren, Andreas, 2018. "How many want to drive the bus? Analyzing the number of bids for public transport bus contracts," Transport Policy, Elsevier, vol. 72(C), pages 138-147.
    15. Fransham, Mark, 2020. "Neighbourhood gentrification, displacement, and poverty dynamics in post-recession England," LSE Research Online Documents on Economics 103905, London School of Economics and Political Science, LSE Library.
    16. Samson Gebremedhin & Tilahun Bekele, 2021. "Gestational weight gain in sub-Saharan Africa: Estimation based on pseudo-cohort design," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-12, May.
    17. Heiko Rauhut, 2013. "Beliefs about Lying and Spreading of Dishonesty: Undetected Lies and Their Constructive and Destructive Social Dynamics in Dice Experiments," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-8, November.
    18. Francesco Guolo & Elisa Stivanello & Lorenzo Pizzi & Teodoro Georgiadis & Letizia Cremonini & Muriel Assunta Musti & Marianna Nardino & Filippo Ferretti & Paolo Marzaroli & Vincenza Perlangeli & Paolo, 2022. "Emergency Department Visits and Summer Temperatures in Bologna, Northern Italy, 2010–2019: A Case-Crossover Study and Geographically Weighted Regression Methods," IJERPH, MDPI, vol. 19(23), pages 1-15, November.
    19. Yarela Flores Arévalo & Roberto D. Ponce Oliva & Francisco J. Fernández & Felipe Vásquez-Lavin, 2021. "Sensitivity of Water Price Elasticity Estimates to Different Data Aggregation Levels," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 35(6), pages 2039-2052, April.
    20. Zajacova, Anna & Hummer, Robert A., 2009. "Gender differences in education effects on all-cause mortality for white and black adults in the United States," Social Science & Medicine, Elsevier, vol. 69(4), pages 529-537, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:35:y:2011:i:3:p:269-303. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.