IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00237.html
   My bibliography  Save this paper

Do students behave rationally in multiple-choice tests? Evidence from a field experiment

Author

Listed:
  • Maria Espinosa
  • Javier Gardeazabal

Abstract

A disadvantage of multiple-choice tests is that students have incentives to guess. To discourage guessing, it is common to use scoring rules that either penalize wrong answers or reward omissions. These scoring rules are considered equivalent in psychometrics, although experimental evidence has not always been consistent with this claim. We model students' decisions and show, first, that equivalence holds only under risk neutrality and, second, that the two rules can be modified so that they become equivalent even under risk aversion. This paper presents the results of a filed experiment in which we analyze the decisions of subjects taking multiple-choice exams. The evidence suggests that differences between scoring rules are due to risk aversion as theory predicts. We also find that the number of omitted items depends on the scoring rule, knowledge, gender, and other covariates.

Suggested Citation

  • Maria Espinosa & Javier Gardeazabal, 2005. "Do students behave rationally in multiple-choice tests? Evidence from a field experiment," Natural Field Experiments 00237, The Field Experiments Website.
    • Handle: RePEc:feb:natura:00237
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00237.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. List, John & Millimet, Daniel, "undated". "Bounding the Impact of Market Experience on Rationality: Evidence from a Field Experiment with Imperfect Compliance," Departmental Working Papers 0505, Southern Methodist University, Department of Economics.
    2. Bertrand, Marianne & Karlan, Dean S. & Mullainathan, Sendhil & Shafir, Eldar & Zinman, Jonathan, 2005. "What's Psychology Worth? A Field Experiment in the Consumer Credit Market," Center Discussion Papers 28441, Yale University, Economic Growth Center.
    3. Michael S. Haigh & John A. List, 2005. "Do Professional Traders Exhibit Myopic Loss Aversion? An Experimental Analysis," Journal of Finance, American Finance Association, vol. 60(1), pages 523-534, February.
    4. William E. Becker & Carol Johnston, 1999. "The Relationship between Multiple Choice and Essay Response Questions in Assessing Economics Understanding," The Economic Record, The Economic Society of Australia, vol. 75(4), pages 348-357, December.
    5. Marco Haan & Bart Los & Yohanes Riyanto & Martin van Geest, 2002. "The Weakest Link - A Field Experiment in Rational Decision Making," Experimental 0203001, University Library of Munich, Germany.
    6. ,, 2008. "Risk taking and gender in hierarchies," Theoretical Economics, Econometric Society, vol. 3(4), December.
    7. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    8. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    9. repec:bla:ecorec:v:75:y:1999:i:231:p:348-57 is not listed on IDEAS
    10. repec:dgr:rugsom:02f20 is not listed on IDEAS
    11. Bredon, George, 2003. "Take-Home Tests in Economics," Economic Analysis and Policy, Elsevier, vol. 33(1), pages 52-60, March.
    12. Bram Cadsby, C. & Maynes, Elizabeth, 2005. "Gender, risk aversion, and the drawing power of equilibrium in an experimental corporate takeover game," Journal of Economic Behavior & Organization, Elsevier, vol. 56(1), pages 39-59, January.
    13. Walstad, William B & Becker, William E, 1994. "Achievement Differences on Multiple-Choice and Essay Tests in Economics," American Economic Review, American Economic Association, vol. 84(2), pages 193-196, May.
    14. Nixon Chan & Peter E. Kennedy, 2002. "Are Multiple-Choice Exams Easier for Economics Students? A Comparison of Multiple-Choice and “Equivalent” Constructed-Response Exam Questions," Southern Economic Journal, John Wiley & Sons, vol. 68(4), pages 957-971, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pelin Akyol & James Key & Kala Krishna, 2022. "Hit or Miss? Test Taking Behavior in Multiple Choice Exams," Annals of Economics and Statistics, GENES, issue 147, pages 3-50.
    2. Butler, Matthew J. & Cardon, James H. & Showalter, Mark H., 2017. "To choose or not to choose: An experiment in hedging strategies and risk preferences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 67(C), pages 14-19.
    3. Zapechelnyuk, Andriy, 2015. "An axiomatization of multiple-choice test scoring," Economics Letters, Elsevier, vol. 132(C), pages 24-27.
    4. ,, 2008. "Risk taking and gender in hierarchies," Theoretical Economics, Econometric Society, vol. 3(4), December.
    5. Montolio, Daniel & Taberner, Pere A., 2021. "Gender differences under test pressure and their impact on academic performance: A quasi-experimental design," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 1065-1090.
    6. Espinosa Maria Paz & Gardeazabal Javier, 2020. "The Gender-bias Effect of Test Scoring and Framing: A Concern for Personnel Selection and College Admission," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 20(3), pages 1-23, July.
    7. Nagore Iriberri & Pedro Rey-Biel, 2019. "Competitive Pressure Widens the Gender Gap in Performance: Evidence from a Two-stage Competition in Mathematics," The Economic Journal, Royal Economic Society, vol. 129(620), pages 1863-1893.
    8. Pau Balart & Lara Ezquerra & Iñigo Hernandez-Arenaz, 2022. "Framing effects on risk-taking behavior: evidence from a field experiment in multiple-choice tests," Experimental Economics, Springer;Economic Science Association, vol. 25(4), pages 1268-1297, September.
    9. J. Ignacio Conde-Ruiz & Juan José Ganuza & Manuel García, 2020. "Gender Gap and Multiple Choice Exams in Public Selection Processes," Hacienda Pública Española / Review of Public Economics, IEF, vol. 235(4), pages 11-28, December.
    10. Saygin, Perihan O. & Atwater, Ann, 2021. "Gender differences in leaving questions blank on high-stakes standardized tests," Economics of Education Review, Elsevier, vol. 84(C).
    11. David Budescu & Yuanchao Bo, 2015. "Analyzing Test-Taking Behavior: Decision Theory Meets Psychometric Theory," Psychometrika, Springer;The Psychometric Society, vol. 80(4), pages 1105-1122, December.
    12. Riener, Gerhard & Wagner, Valentin, 2017. "Shying away from demanding tasks? Experimental evidence on gender differences in answering multiple-choice questions," Economics of Education Review, Elsevier, vol. 59(C), pages 43-62.
    13. Iriberri, Nagore & Rey-Biel, Pedro, 2021. "Brave boys and play-it-safe girls: Gender differences in willingness to guess in a large scale natural field experiment," European Economic Review, Elsevier, vol. 131(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    2. Omar Al-Ubaydli & John List, 2016. "Field Experiments in Markets," Artefactual Field Experiments j0002, The Field Experiments Website.
    3. Shawn Cole & Xavier Gine & Jeremy Tobacman & Petia Topalova & Robert Townsend & James Vickery, 2013. "Barriers to Household Risk Management: Evidence from India," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 104-135, January.
    4. Stefano Cosma & Francesco Pattarin, 2012. "Attitudes, personality factors and household debt decisions: A study of consumer credit," Centro Studi di Banca e Finanza (CEFIN) (Center for Studies in Banking and Finance) 12021, Universita di Modena e Reggio Emilia, Dipartimento di Economia "Marco Biagi".
    5. Hong Luo & Julie Holland Mortimer, 2017. "Copyright Enforcement: Evidence from Two Field Experiments," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 26(2), pages 499-528, June.
    6. Bertrand, Marianne & Shafir, Eldar & Mullainathan, Sendhil, 2006. "Behavioral Economics and Marketing in Aid of Decision Making Among the Poor," Scholarly Articles 2962609, Harvard University Department of Economics.
    7. Roland G. Fryer, Jr & Steven D. Levitt & John List & Sally Sadoff, 2012. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment," NBER Working Papers 18237, National Bureau of Economic Research, Inc.
    8. Jim Engle-Warnick & Javier Escobal & Sonia Laszlo, 2006. "The Effect Of An Additional Alternative On Measured Risk Preferences In A Laboratory Experiment In Peru," Departmental Working Papers 2006-10, McGill University, Department of Economics.
    9. Dean Karlan & Jonathan Zinman, 2005. "Elasticities of Demand for Consumer Credit," Working Papers 926, Economic Growth Center, Yale University.
    10. Jim Engle‐Warnick & Javier Escobal & Sonia Laszlo, 2009. "How do additional alternatives affect individual choice under uncertainty?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 42(1), pages 113-140, February.
    11. Dean S. Karlan & Jonathan Zinman, 2008. "Credit Elasticities in Less-Developed Economies: Implications for Microfinance," American Economic Review, American Economic Association, vol. 98(3), pages 1040-1068, June.
    12. Bogdan Genchev & Julie Holland Mortimer, 2016. "Empirical Evidence on Conditional Pricing Practices," NBER Working Papers 22313, National Bureau of Economic Research, Inc.
    13. Sendhil Mullainathan & Joshua Schwartzstein & Andrei Shleifer, 2008. "Coarse Thinking and Persuasion," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(2), pages 577-619.
    14. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    15. Elbæk, Christian T. & Lystbæk, Martin Nørhede & Mitkidis, Panagiotis, 2022. "On the psychology of bonuses: The effects of loss aversion and Yerkes-Dodson law on performance in cognitively and mechanically demanding tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    16. Emmanuel Saez, 2009. "Details Matter: The Impact of Presentation and Information on the Take-Up of Financial Incentives for Retirement Saving," American Economic Journal: Economic Policy, American Economic Association, vol. 1(1), pages 204-228, February.
    17. Steven D. Levitt & John A. List & Susanne Neckermann & Sally Sadoff, 2016. "The Behavioralist Goes to School: Leveraging Behavioral Economics to Improve Educational Performance," American Economic Journal: Economic Policy, American Economic Association, vol. 8(4), pages 183-219, November.
    18. Dammert, Ana C. & Galdo, Jose C. & Galdo, Virgilio, 2014. "Preventing dengue through mobile phones: Evidence from a field experiment in Peru," Journal of Health Economics, Elsevier, vol. 35(C), pages 147-161.
    19. Pan, Yao & You, Jing, 2020. "Successful Social Programs over Local Political Cycles," MPRA Paper 98968, University Library of Munich, Germany.
    20. Kendra Strauss, 2009. "Cognition, Context, and Multimethod Approaches to Economic Decision Making," Environment and Planning A, , vol. 41(2), pages 302-317, February.

    More about this item

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D03 - Microeconomics - - General - - - Behavioral Microeconomics: Underlying Principles
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00237. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Francesca Pagnotta (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.