IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v51y2022i2p816-836.html
   My bibliography  Save this article

Relying on External Information Sources When Answering Knowledge Questions in Web Surveys

Author

Listed:
  • Tobias Gummer
  • Tanja Kunz

Abstract

Knowledge questions frequently are used in survey research to measure respondents’ topic-related cognitive ability and memory. However, in self-administered surveys, respondents can search external sources for additional information to answer a knowledge question correctly. In this case, the knowledge question measures accessible and procedural memory. Depending on what the knowledge question aims at, the validity of this measure is limited. Thus, in this study, we conducted three experiments using a web survey to investigate the effects of task difficulty, respondents’ ability, and respondents’ motivation on the likelihood of searching external sources for additional information as a form of over-optimizing response behavior when answering knowledge questions. We found that the respondents who are highly educated and more interested in a survey are more likely to invest additional efforts to answer knowledge questions correctly. Most importantly, our data showed that for these respondents, a more difficult question design further increases the likelihood of over-optimizing response behavior.

Suggested Citation

  • Tobias Gummer & Tanja Kunz, 2022. "Relying on External Information Sources When Answering Knowledge Questions in Web Surveys," Sociological Methods & Research, , vol. 51(2), pages 816-836, May.
    • Handle: RePEc:sae:somere:v:51:y:2022:i:2:p:816-836
    DOI: 10.1177/0049124119882470
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124119882470
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124119882470?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Markus Prior, 2005. "News vs. Entertainment: How Increasing Media Choice Widens Gaps in Political Knowledge and Turnout," American Journal of Political Science, John Wiley & Sons, vol. 49(3), pages 577-592, July.
    2. Vera Toepoel & Corrie Vis & Marcel Das & Arthur van Soest, 2009. "Design of Web Questionnaires," Sociological Methods & Research, , vol. 37(3), pages 371-392, February.
    3. Markus Prior & Arthur Lupia, 2008. "Money, Time, and Political Knowledge: Distinguishing Quick Recall and Political Learning Skills," American Journal of Political Science, John Wiley & Sons, vol. 52(1), pages 169-183, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. William D. Berry & Jacqueline H. R. DeMeritt & Justin Esarey, 2010. "Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential?," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 248-266, January.
    2. Samuele Poy & Simone Schüller, 2016. "Internet and Voting in the Web 2.0 Era: Evidence from a Local Broadband Policy," CESifo Working Paper Series 6129, CESifo.
    3. John V. Duca & Jason L. Saving, 2016. "Income Inequality and Political Polarization: Time Series Evidence Over Nine Decades," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 62(3), pages 445-466, September.
    4. Donati, Dante, 2023. "Mobile Internet access and political outcomes: Evidence from South Africa," Journal of Development Economics, Elsevier, vol. 162(C).
    5. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    6. Ruben Durante & Paolo Pinotti & Andrea Tesei, 2019. "The Political Legacy of Entertainment TV," American Economic Review, American Economic Association, vol. 109(7), pages 2497-2530, July.
    7. Stan Hok‐Wui Wong & Mathew Y. H. Wong, 2020. "“Distant Participation” and Youth Political Attitudes: Evidence from a Natural Experiment," Social Science Quarterly, Southwestern Social Science Association, vol. 101(4), pages 1489-1512, July.
    8. David K Levine, 2023. "True Myths," Levine's Working Paper Archive 11694000000000123, David K. Levine.
    9. Baumberg, Ben, 2016. "Benefit `myths'? The accuracy and inaccuracy of public beliefs about the benefits system," LSE Research Online Documents on Economics 103512, London School of Economics and Political Science, LSE Library.
    10. Lin Hu & Anqi Li & Ilya Segal, 2019. "The Politics of Personalized News Aggregation," Papers 1910.11405, arXiv.org, revised Nov 2022.
    11. repec:spo:wpmain:info:hdl:2441/16juu6v6rg8rq8nl0u1grb4jm6 is not listed on IDEAS
    12. Natali HELBERGER & Andra LEURDIJK & Silvain de MUNCK, 2010. "User Generated Diversity. Some reflections on how to improve the quality of amateur productions," Communications & Strategies, IDATE, Com&Strat dept., vol. 1(77), pages 55-78, 1st quart.
    13. Lisa Oswald, 2024. "More than news! Mapping the deliberative potential of a political online ecosystem with digital trace data," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-16, December.
    14. Julia Cagé, 2014. "Media Competition, Information Provision and Political Participation," SciencePo Working papers hal-03602440, HAL.
    15. Duca, John V. & Saving, Jason L., 2018. "What drives economic policy uncertainty in the long and short runs: European and U.S. evidence over several decades," Journal of Macroeconomics, Elsevier, vol. 55(C), pages 128-145.
    16. Julia Cagé & Nicolas Hervé & Marie-Luce Viaud, 2020. "The Production of Information in an Online World," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 87(5), pages 2126-2164.
    17. Brad R. Taylor, 2020. "The psychological foundations of rational ignorance: biased heuristics and decision costs," Constitutional Political Economy, Springer, vol. 31(1), pages 70-88, March.
    18. repec:hal:journl:hal-03533356 is not listed on IDEAS
    19. Bart Buelens & Jan A. van den Brakel, 2015. "Measurement Error Calibration in Mixed-mode Sample Surveys," Sociological Methods & Research, , vol. 44(3), pages 391-426, August.
    20. Bodo Knoll & Hans Pitlik & Martin Rode, 2023. "TV Consumption Patterns and the Impact of Media Freedom on Political Trust and Satisfaction with the Government," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 169(1), pages 323-340, September.
    21. Benjamin Ogden, 2017. "The Imperfect Beliefs Voting Model," Working Papers ECARES ECARES 2017-20, ULB -- Universite Libre de Bruxelles.
    22. John V. Duca & Jason L. Saving, 2012. "Has income inequality or media fragmentation increased political polarization?," Working Papers 1206, Federal Reserve Bank of Dallas.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:51:y:2022:i:2:p:816-836. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.