IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v185y2022i3p872-890.html
   My bibliography  Save this article

Innovating the collection of open‐ended answers: The linguistic and content characteristics of written and oral answers to political attitude questions

Author

Listed:
  • Konstantin Gavras
  • Jan Karem Höhne
  • Annelies G. Blom
  • Harald Schoen

Abstract

The rapid increase in smartphone surveys and technological developments open novel opportunities for collecting survey answers. One of these opportunities is the use of open‐ended questions with requests for oral instead of written answers, which may facilitate the answer process and result in more in‐depth and unfiltered information. Whereas it is now possible to collect oral answers on smartphones, we still lack studies on the impact of this novel answer format on the characteristics of respondents' answers. In this study, we compare the linguistic and content characteristics of written versus oral answers to political attitude questions. For this purpose, we conducted an experiment in a smartphone survey (N = 2402) and randomly assigned respondents to an answer format (written or oral). Oral answers were collected via the open source ‘SurveyVoice (SVoice)’ tool, whereas written answers were typed in via the smartphone keypad. Applying length analysis, lexical structure analysis, sentiment analysis and structural topic models, our results reveal that written and oral answers differ substantially from each other in terms of lengths, structures, sentiments and topics. We find evidence that written answers are characterized by an intentional and conscious answering, whereas oral answers are characterized by an intuitive and spontaneous answering.

Suggested Citation

  • Konstantin Gavras & Jan Karem Höhne & Annelies G. Blom & Harald Schoen, 2022. "Innovating the collection of open‐ended answers: The linguistic and content characteristics of written and oral answers to political attitude questions," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 872-890, July.
    • Handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:872-890
    DOI: 10.1111/rssa.12807
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12807
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12807?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Proksch, Sven-Oliver & Wratil, Christopher & Wäckerle, Jens, 2019. "Testing the Validity of Automatic Speech Recognition for Political Text Analysis," Political Analysis, Cambridge University Press, vol. 27(3), pages 339-359, July.
    2. Anna DeCastellarnau, 2018. "A classification of response scale characteristics that affect data quality: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1523-1559, July.
    3. Melanie Revilla & Carlos Ochoa, 2016. "Open narrative questions in PC and smartphones: is the device playing a role?," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(6), pages 2495-2513, November.
    4. Laura K. Nelson & Derek Burk & Marcel Knudsen & Leslie McCall, 2021. "The Future of Coding: A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods," Sociological Methods & Research, , vol. 50(1), pages 202-237, February.
    5. Christopher Claassen, 2020. "Does Public Support Help Democracy Survive?," American Journal of Political Science, John Wiley & Sons, vol. 64(1), pages 118-134, January.
    6. Lodge, Milton & McGraw, Kathleen M. & Stroh, Patrick, 1989. "An Impression-Driven Model of Candidate Evaluation," American Political Science Review, Cambridge University Press, vol. 83(2), pages 399-419, June.
    7. Achen, Christopher H., 1975. "Mass Political Attitudes and the Survey Response," American Political Science Review, Cambridge University Press, vol. 69(4), pages 1218-1231, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hank C. Jenkins-Smith & Neil J. Mitchell & Kerry G. Herron, 2004. "Foreign and Domestic Policy Belief Structures in the U.S. and British Publics," Journal of Conflict Resolution, Peace Science Society (International), vol. 48(3), pages 287-309, June.
    2. Bernhardt, Lea & Dewenter, Ralf & Thomas, Tobias, 2023. "Measuring partisan media bias in US newscasts from 2001 to 2012," European Journal of Political Economy, Elsevier, vol. 78(C).
    3. Lourdes ROJAS RUBIO, 2022. "Inequality, Corruption and Support for Democracy," THEMA Working Papers 2022-20, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    4. Duane Alwin, 1989. "Problems in the estimation and interpretation of the reliability of survey data," Quality & Quantity: International Journal of Methodology, Springer, vol. 23(3), pages 277-331, September.
    5. Niculaescu, Corina E. & Sangiorgi, Ivan & Bell, Adrian R., 2023. "Does personal experience with COVID-19 impact investment decisions? Evidence from a survey of US retail investors," International Review of Financial Analysis, Elsevier, vol. 88(C).
    6. S. Brent Jackson & Kathryn T. Stevenson & Lincoln R. Larson & M. Nils Peterson & Erin Seekamp, 2021. "Outdoor Activity Participation Improves Adolescents’ Mental Health and Well-Being during the COVID-19 Pandemic," IJERPH, MDPI, vol. 18(5), pages 1-18, March.
    7. Jon Roozenbeek & Stefan M. Herzog & Michael Geers & Ralf Kurvers & Mubashir Sultan & Sander van der Linden, 2022. "Susceptibility to misinformation is consistent across question framings and response modes and better explained by myside bias and partisanship than analytical thinking," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 17(3), pages 547-573, May.
    8. Joël Cariolle & Yasmine Elkhateeb & Mathilde Maurel, 2022. "(Mis-)information technology: Internet use and perception of democracy in Africa," Documents de travail du Centre d'Economie de la Sorbonne 22010, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    9. Enriqueta Aragones, 1997. "Negativity Effect and the Emergence of Ideologies," Journal of Theoretical Politics, , vol. 9(2), pages 189-210, April.
    10. Felderer, Barbara & Repke, Lydia & Weber, Wiebke & Schweisthal, jonas & Bothmann, Ludwig, 2024. "Predicting the Validity and Reliability of Survey Questions," OSF Preprints hkngd, Center for Open Science.
    11. Cavaillé, Charlotte & Chen, Daniel L. & Van Der Straeten, Karine, 2022. "Who Cares? Measuring Preference Intensity in a Polarized Environment," IAST Working Papers 22-130, Institute for Advanced Study in Toulouse (IAST).
    12. John Patty & Roberto Weber, 2007. "Letting the good times roll: A theory of voter inference and experimental evidence," Public Choice, Springer, vol. 130(3), pages 293-310, March.
    13. Jennifer Oser & Marc Hooghe & Zsuzsa Bakk & Roberto Mari, 2023. "Changing citizenship norms among adolescents, 1999-2009-2016: A two-step latent class approach with measurement equivalence testing," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(5), pages 4915-4933, October.
    14. Daniel Caballero-Julia & Philippe Campillo, 2021. "Epistemological Considerations of Text Mining: Implications for Systematic Literature Review," Mathematics, MDPI, vol. 9(16), pages 1-26, August.
    15. repec:cup:judgdm:v:17:y:2022:i:3:p:547-573 is not listed on IDEAS
    16. Lee, Taeku & Oliver, J. Eric, 2002. "Public Opinion and the Politics of America's Obesity Epidemic," Working Paper Series rwp02-017, Harvard University, John F. Kennedy School of Government.
    17. Choi, Young Rok & Phan, Phillip H. & Choi, Jaepil, 2020. "Formal governance, interfirm coordination, and performance in partnerships: An empirical investigation of a mediation model," European Management Journal, Elsevier, vol. 38(3), pages 413-424.
    18. Aarti Sharma & Taghreed Abunada & Sawsan S. Said & Rana M. Kurdi & Atiyeh M. Abdallah & Marawan Abu-Madi, 2022. "Clinical Practicum Assessment for Biomedical Science Program from Graduates’ Perspective," IJERPH, MDPI, vol. 19(19), pages 1-10, September.
    19. Mónica D. Oliveira & Inês Mataloto & Panos Kanavos, 2019. "Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 891-918, August.
    20. Silvia Russo & Håkan Stattin, 2017. "Stability and Change in Youths’ Political Interest," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 132(2), pages 643-658, June.
    21. Alex Luscombe & Kevin Dick & Kevin Walby, 2022. "Algorithmic thinking in the public interest: navigating technical, legal, and ethical hurdles to web scraping in the social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(3), pages 1023-1044, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:872-890. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.