IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v27y1998i1p103-133.html
   My bibliography  Save this article

An Empirical Evaluation of In-Depth Probes Used to Pretest Survey Questions

Author

Listed:
  • WILLIAM FODDY

    (Monash University)

Abstract

A growing number of survey methodologists have advocated the use of in-depth probes for identifying comprehension problems, inadequate response categories, the perspectives adopted by respondents, and the strategies respondents employ when retrieving information from memory. This article reports the results of an empirical evaluation of the relative effectiveness of a number of the probes that have been used for the first three of these tasks. This work confirms that the traditional practice of field testing a questionnaire on a subsample drawn from the population of interest is not an effective way of detecting shortcomings in survey questions. It also indicates that the effectiveness of a probe is directly related to its specificity; the most effective comprehension probes are those directed at exploring the ways in which respondents interpret key concepts; probes designed to get at respondents' interpretations of key concepts are the most effective means of identifying perspectives respondents have adopted; and well-educated respondents are most likely to make substantive responses to in-depth probes.

Suggested Citation

  • William Foddy, 1998. "An Empirical Evaluation of In-Depth Probes Used to Pretest Survey Questions," Sociological Methods & Research, , vol. 27(1), pages 103-133, August.
    • Handle: RePEc:sae:somere:v:27:y:1998:i:1:p:103-133
    DOI: 10.1177/0049124198027001003
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124198027001003
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124198027001003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jobe, J.B. & Mingay, D.J., 1989. "Cognitive research improves questionnaires," American Journal of Public Health, American Public Health Association, vol. 79(8), pages 1053-1055.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dürnberger, Andrea & Drasch, Katrin & Matthes, Britta, 2010. "Kontextgestützte Abfrage in Retrospektiverhebungen: Ein kognitiver Pretest zu Erinnerungsprozessen bei Weiterbildungsereignissen," IAB-Discussion Paper 201020, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    2. Verhagen, Tibert & Meents, Selmar, 2007. "A Framework for Developing Semantic Differentials in IS research: Assessing the Meaning of Electronic Marketplace Quality (EMQ)," Serie Research Memoranda 0016, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Annette S. Crisanti & Jennifer Earheart & Megan Deissinger & Kathryn Lowerre & Julie G. Salvador, 2022. "Implementation Challenges and Recommendations for Employing Peer Support Workers in Emergency Departments to Support Patients Presenting after an Opioid-Related Overdose," IJERPH, MDPI, vol. 19(9), pages 1-13, April.
    2. Weijters, Bert & Geuens, Maggie & Schillewaert, Niels, 2009. "The proximity effect: The role of inter-item distance on reverse-item bias," International Journal of Research in Marketing, Elsevier, vol. 26(1), pages 2-12.
    3. Cheng-Han Leng & Hung-Yu Huang & Grace Yao, 2020. "A Social Desirability Item Response Theory Model: Retrieve–Deceive–Transfer," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 56-74, March.
    4. Anja Mohorko & Valentina Hlebec, 2015. "Effect of a first-time interviewer on cognitive interview quality," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(5), pages 1897-1918, September.
    5. Helsper, Ellen & Schneider, Luc & van Deursen, Alexander J. A. M. & van Laar, Ester, 2020. "The youth skills digital indicator: report on the conceptualisation and development of the ySkills digital skills measure," LSE Research Online Documents on Economics 108878, London School of Economics and Political Science, LSE Library.
    6. William Foddy, 1996. "The in-depth testing of survey questions: A critical appraisal of methods," Quality & Quantity: International Journal of Methodology, Springer, vol. 30(4), pages 361-370, November.
    7. Parra, Diana C. & Gomez, Luis F. & Sarmiento, Olga L. & Buchner, David & Brownson, Ross & Schimd, Thomas & Gomez, Viviola & Lobelo, Felipe, 2010. "Perceived and objective neighborhood environment attributes and health related quality of life among the elderly in Bogotá, Colombia," Social Science & Medicine, Elsevier, vol. 70(7), pages 1070-1076, April.
    8. Lazo, Jeffrey K & McClain, Katherine T, 1996. "Community perceptions, environmental impacts, and energy policy : Rail shipment of coal," Energy Policy, Elsevier, vol. 24(6), pages 531-540, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:27:y:1998:i:1:p:103-133. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.