IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0182156.html
   My bibliography  Save this article

"What else are you worried about?" – Integrating textual responses into quantitative social science research

Author

Listed:
  • Julia M Rohrer
  • Martin Brümmer
  • Stefan C Schmukle
  • Jan Goebel
  • Gert G Wagner

Abstract

Open-ended questions have routinely been included in large-scale survey and panel studies, yet there is some perplexity about how to actually incorporate the answers to such questions into quantitative social science research. Tools developed recently in the domain of natural language processing offer a wide range of options for the automated analysis of such textual data, but their implementation has lagged behind. In this study, we demonstrate straightforward procedures that can be applied to process and analyze textual data for the purposes of quantitative social science research. Using more than 35,000 textual answers to the question “What else are you worried about?” from participants of the German Socio-economic Panel Study (SOEP), we (1) analyzed characteristics of respondents that determined whether they answered the open-ended question, (2) used the textual data to detect relevant topics that were reported by the respondents, and (3) linked the features of the respondents to the worries they reported in their textual data. The potential uses as well as the limitations of the automated analysis of textual data are discussed.

Suggested Citation

  • Julia M Rohrer & Martin Brümmer & Stefan C Schmukle & Jan Goebel & Gert G Wagner, 2017. ""What else are you worried about?" – Integrating textual responses into quantitative social science research," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-34, July.
    • Handle: RePEc:plo:pone00:0182156
    DOI: 10.1371/journal.pone.0182156
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0182156
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0182156&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0182156?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    2. Jo Garcia & Julie Evans & Maggie Reshaw, 2004. "``Is There Anything Else You Would Like to Tell Us'' – Methodological Issues in the Use of Free-Text Comments from Postal Surveys," Quality & Quantity: International Journal of Methodology, Springer, vol. 38(2), pages 113-125, April.
    3. Efstathios Stamatatos, 2009. "A survey of modern authorship attribution methods," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(3), pages 538-556, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Julia M. Rohrer & Martin Bruemmer & Jürgen Schupp & Gert G. Wagner, 2017. "Worries across Time and Age in Germany: Bringing Together Open- and Close-Ended Questions," Discussion Papers of DIW Berlin 1671, DIW Berlin, German Institute for Economic Research.
    2. Patrick Meyer & Fenja M Schophaus & Thomas Glassen & Jasmin Riedl & Julia M Rohrer & Gert G Wagner & Timo von Oertzen, 2019. "Using the Dirichlet process to form clusters of people’s concerns in the context of future party identification," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-20, March.
    3. Zhang, Yucheng & Xu, Shan & Zhang, Long & Yang, Mengxi, 2021. "Big data and human resource management research: An integrative review and new directions for future research," Journal of Business Research, Elsevier, vol. 133(C), pages 34-50.
    4. Rohrer, Julia M. & Brümmer, Martin & Schupp, Jürgen & Wagner, Gert G., 2021. "Worries across time and age in the German Socio-Economic Panel study," Journal of Economic Behavior & Organization, Elsevier, vol. 181(C), pages 332-343.
    5. Gert G. Wagner & Martin Bruemmer & Axel Glemser & Julia Rohrer & Jürgen Schupp, 2017. "Dimensions of Quality of Life in Germany: Measured by Plain Text Responses in a Representative Survey (SOEP)," SOEPpapers on Multidisciplinary Panel Data Research 893, DIW Berlin, The German Socio-Economic Panel (SOEP).
    6. Alan Piper & Ian Jackson, 2017. "She’s leaving home: a large sample investigation of the empty nest syndrome," Danish-German Working Papers 006, Europa-Universität Flensburg, International Institute of Management (IIM);University of Southern Denmark, Department of Border Region Studies (IFG).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ballandonne, Matthieu & Cersosimo, Igor, 2022. "Towards a “Text as Data” Approach in the History of Economics: An Application to Adam Smith’s Classics," OSF Preprints mg3zb, Center for Open Science.
    2. Ballandonne, Matthieu & Cersosimo, Igor, 2022. "Towards a “Text as Data” Approach in the History of Economics: An Application to Adam Smith’s Classics," OSF Preprints mg3zb_v1, Center for Open Science.
    3. H Andrew Schwartz & Johannes C Eichstaedt & Margaret L Kern & Lukasz Dziurzynski & Stephanie M Ramones & Megha Agrawal & Achal Shah & Michal Kosinski & David Stillwell & Martin E P Seligman & Lyle H U, 2013. "Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-16, September.
    4. Bernhardt, Lea & Dewenter, Ralf & Thomas, Tobias, 2023. "Measuring partisan media bias in US newscasts from 2001 to 2012," European Journal of Political Economy, Elsevier, vol. 78(C).
    5. Ntentas, Raphael, 2021. "Quantifying political populism and examining the link with economic insecurity: evidence from Greece," LSE Research Online Documents on Economics 112579, London School of Economics and Political Science, LSE Library.
    6. Lin, Annie E. & Young, Jimmy A. & Guarino, Jeannine E., 2022. "Mother-Daughter sexual abuse: An exploratory study of the experiences of survivors of MDSA using Reddit," Children and Youth Services Review, Elsevier, vol. 138(C).
    7. Rybinski, Krzysztof, 2020. "The forecasting power of the multi-language narrative of sell-side research: A machine learning evaluation," Finance Research Letters, Elsevier, vol. 34(C).
    8. Rauh, Christian, 2015. "Communicating supranational governance? The salience of EU affairs in the German Bundestag, 1991–2013," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 16(1), pages 116-138.
    9. Grajzl, Peter & Murrell, Peter, 2021. "A machine-learning history of English caselaw and legal ideas prior to the Industrial Revolution I: generating and interpreting the estimates," Journal of Institutional Economics, Cambridge University Press, vol. 17(1), pages 1-19, February.
    10. David Bholat & Stephen Hans & Pedro Santos & Cheryl Schonhardt-Bailey, 2015. "Text mining for central banks," Handbooks, Centre for Central Banking Studies, Bank of England, number 33, April.
    11. Julia Seiermann, 2018. "Only Words? How Power in Trade Agreement Texts Affects International Trade Flows," UNCTAD Blue Series Papers 80, United Nations Conference on Trade and Development.
    12. Sami Diaf & Jörg Döpke & Ulrich Fritsche & Ida Rockenbach, 2020. "Sharks and minnows in a shoal of words: Measuring latent ideological positions of German economic research institutes based on text mining techniques," Macroeconomics and Finance Series 202001, University of Hamburg, Department of Socioeconomics.
    13. Dehler-Holland, Joris & Schumacher, Kira & Fichtner, Wolf, 2021. "Topic Modeling Uncovers Shifts in Media Framing of the German Renewable Energy Act," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 2(1).
    14. Weiss, Max & Zoorob, Michael, 2021. "Political frames of public health crises: Discussing the opioid epidemic in the US Congress," Social Science & Medicine, Elsevier, vol. 281(C).
    15. Maschke, Andreas, 2024. "Talking exports: The representation of Germany's current account in newspaper media," MPIfG Discussion Paper 24/1, Max Planck Institute for the Study of Societies.
    16. Arthur Dyevre & Nicolas Lampach, 2021. "Issue attention on international courts: Evidence from the European Court of Justice," The Review of International Organizations, Springer, vol. 16(4), pages 793-815, October.
    17. Dewenter, Ralf & Dulleck, Uwe & Thomas, Tobias, 2018. "The political coverage index and its application to government capture," Research Papers 6, EcoAustria – Institute for Economic Research.
    18. Pastwa, Anna M. & Shrestha, Prabal & Thewissen, James & Torsin, Wouter, 2021. "Unpacking the black box of ICO white papers: a topic modeling approach," LIDAM Discussion Papers LFIN 2021018, Université catholique de Louvain, Louvain Finance (LFIN).
    19. Maksym Polyakov & Morteza Chalak & Md. Sayed Iftekhar & Ram Pandit & Sorada Tapsuwan & Fan Zhang & Chunbo Ma, 2018. "Authorship, Collaboration, Topics, and Research Gaps in Environmental and Resource Economics 1991–2015," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 71(1), pages 217-239, September.
    20. Parijat Chakrabarti & Margaret Frye, 2017. "A mixed-methods framework for analyzing text data: Integrating computational techniques with qualitative methods in demography," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 37(42), pages 1351-1382.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0182156. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.