IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v5y2015i2p2158244015584617.html
   My bibliography  Save this article

It’s a Trap! Instructional Manipulation Checks Prompt Systematic Thinking on “Tricky†Tasks

Author

Listed:
  • David J. Hauser
  • Norbert Schwarz

Abstract

Instructional manipulation checks (IMCs) have become popular tools for identifying inattentive participants in online studies. IMCs function by attempting to trick inattentive participants into responding incorrectly. However, from a conversational perspective, question characteristics are part of the researcher’s contribution to the conversation, and IMCs may teach participants that there is “more than meets the eye,†prompting systematic thinking on subsequent tricky-seeming questions in an attempt to avoid being tricked. In two online studies, participants responded to a simple task either before or after completing an IMC. As expected, answering an IMC prior to the task improved performance on items that benefit from increased systematic thinking—namely, the Cognitive Reflection Test (Study 1), and a probabilistic reasoning task (Study 2). We conclude that IMCs change attention rather than merely measure attention and discuss implications for their use in online studies.

Suggested Citation

  • David J. Hauser & Norbert Schwarz, 2015. "It’s a Trap! Instructional Manipulation Checks Prompt Systematic Thinking on “Tricky†Tasks," SAGE Open, , vol. 5(2), pages 21582440155, April.
    • Handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015584617
    DOI: 10.1177/2158244015584617
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244015584617
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244015584617?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    2. Adam J. Berinsky & Michele F. Margolis & Michael W. Sances, 2014. "Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self‐Administered Surveys," American Journal of Political Science, John Wiley & Sons, vol. 58(3), pages 739-753, July.
    3. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Olbrich, Lukas & Sakshaug, Joseph W. & Lewandowski, Eric, 2024. "Evaluating methods to prevent and detect inattentive respondents in web surveys," SocArXiv py9gz, Center for Open Science.
    2. Eugene Y. Chan & Jack Lin, 2022. "Political ideology and psychological reactance: how serious should climate change be?," Climatic Change, Springer, vol. 172(1), pages 1-22, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Del Ponte, Alessandro & Li, Lianjun & Ang, Lina & Lim, Noah & Seow, Wei Jie, 2024. "Evaluating SoJump.com as a tool for online behavioral research in China," Journal of Behavioral and Experimental Finance, Elsevier, vol. 41(C).
    2. Justin F. Landy, 2016. "Representations of moral violations: Category members and associated features," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(5), pages 496-508, September.
    3. Insoo Cho & Peter F. Orazem, 2021. "How endogenous risk preferences and sample selection affect analysis of firm survival," Small Business Economics, Springer, vol. 56(4), pages 1309-1332, April.
    4. David J. Cooper & Krista Saral & Marie Claire Villeval, 2021. "Why Join a Team?," Management Science, INFORMS, vol. 67(11), pages 6980-6997, November.
    5. Zakaria Babutsidze & Nobuyuki Hanaki & Adam Zylbersztejn, 2019. "Digital Communication and Swift Trust," Post-Print halshs-02409314, HAL.
    6. Brice Corgnet & Roberto Hernán Gonzalez & Ricardo Mateo, 2015. "Cognitive Reflection and the Diligent Worker: An Experimental Study of Millennials," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-13, November.
    7. Francesco Capozza & Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2021. "Studying Information Acquisition in the Field: A Practical Guide and Review," CEBI working paper series 21-15, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    8. Luigi Guiso, 2015. "A Test of Narrow Framing and its Origin," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 1(1), pages 61-100, March.
    9. Breaban, Adriana & van de Kuilen, Gijs & Noussair, Charles, 2016. "Prudence, Personality, Cognitive Ability and Emotional State," Other publications TiSEM 9a01a5ab-e03d-49eb-9cd7-4, Tilburg University, School of Economics and Management.
    10. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    11. Fabian Herweg & Svenja Hippel & Daniel Müller & Fabio Römeis, 2024. "Axiom Preferences and Choice Mistakes under Risk," ECONtribute Discussion Papers Series 326, University of Bonn and University of Cologne, Germany.
    12. Schiff, Kaylyn Jackson & Schiff, Daniel S. & Bueno, Natalia, 2023. "The Liar's Dividend: The Impact of Deepfakes and Fake News on Trust in Political Discourse," SocArXiv x43ph, Center for Open Science.
    13. Fuchsman, Dillon & McGee, Josh B. & Zamarro, Gema, 2023. "Teachers’ willingness to pay for retirement benefits: A national stated preferences experiment," Economics of Education Review, Elsevier, vol. 92(C).
    14. Prokudina, Elena & Renneboog, Luc & Tobler, Philippe, 2015. "Does Confidence Predict Out-of-Domain Effort?," Discussion Paper 2015-055, Tilburg University, Center for Economic Research.
    15. Jordi Brandts & Isabel Busom & Cristina Lopez-Mayan & Judith Panadés, 2023. "Images Say More than Just Words: Effectiveness of Visual and Text Communication in Dispelling the Rent-Control Misconception," CESifo Working Paper Series 10537, CESifo.
    16. Noussair, C.N. & Tucker, S. & Xu, Yilong, 2014. "A Future Market Reduces Bubbles but Allows Greater Profit for More Sophisticated Traders," Other publications TiSEM 43ded173-9eee-48a4-8a15-6, Tilburg University, School of Economics and Management.
    17. Johannes Abeler & Felix Marklein, 2017. "Fungibility, Labels, and Consumption," Journal of the European Economic Association, European Economic Association, vol. 15(1), pages 99-127.
    18. Ek, Claes, 2017. "Some causes are more equal than others? The effect of similarity on substitution in charitable giving," Journal of Economic Behavior & Organization, Elsevier, vol. 136(C), pages 45-62.
    19. Vecchio, Riccardo & Caso, Gerarda & Cembalo, Luigi & Borrello, Massimiliano, 2020. "Is respondents’ inattention in online surveys a major issue for research?," Economia agro-alimentare / Food Economy, Italian Society of Agri-food Economics/Società Italiana di Economia Agro-Alimentare (SIEA), vol. 22(01), March.
    20. Lex Borghans & Angela Lee Duckworth & James J. Heckman & Bas ter Weel, 2008. "The Economics and Psychology of Personality Traits," Journal of Human Resources, University of Wisconsin Press, vol. 43(4).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015584617. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.