IDEAS home Printed from https://ideas.repec.org/a/wly/apecpp/v44y2022i1p110-128.html
   My bibliography  Save this article

Mitigating implicit bias in student evaluations: A randomized intervention

Author

Listed:
  • Brandon Genetin
  • Joyce Chen
  • Vladimir Kogan
  • Alan Kalish

Abstract

We conduct a randomized control trial to assess the efficacy of utilizing modified introductory language in student evaluations of instruction to mitigate implicit bias. Students are randomly assigned within courses to three treatment arms and shown so‐called “cheap talk” scripts referencing implicit bias, the high stakes associated with student evaluations, and the combination of the two. We analyze both the impact assignment of the treatment has on completion rates as well as the effect on average instructor rating. Our analysis indicates assignment has statistically significant effects on the likelihood of response for those assigned the combined treatment, though the effects are heterogeneous with respect to both instructor and student race/ethnicity and gender. We further find the high‐stakes treatment leads to higher average scores for racial/ethnic minority instructors with no significant effects from the implicit bias and combined scripts.

Suggested Citation

  • Brandon Genetin & Joyce Chen & Vladimir Kogan & Alan Kalish, 2022. "Mitigating implicit bias in student evaluations: A randomized intervention," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 44(1), pages 110-128, March.
  • Handle: RePEc:wly:apecpp:v:44:y:2022:i:1:p:110-128
    DOI: 10.1002/aepp.13217
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/aepp.13217
    Download Restriction: no

    File URL: https://libkey.io/10.1002/aepp.13217?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Firpo, Sergio & Foguel, Miguel N. & Jales, Hugo, 2020. "Balancing tests in stratified randomized controlled trials: A cautionary note," Economics Letters, Elsevier, vol. 186(C).
    2. Friederike Mengel & Jan Sauermann & Ulf Zölitz, 2019. "Gender Bias in Teaching Evaluations," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 535-566.
    3. Laura O. Taylor & Ronald G. Cummings, 1999. "Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method," American Economic Review, American Economic Association, vol. 89(3), pages 649-665, June.
    4. Michael A McPherson & R Todd Jewell & Myungsup Kim, 2009. "What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 35(1), pages 37-51.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert L. Moore & Hanna Song & James D. Whitney, 2021. "Do Students Discriminate? Exploring Differentials by Race and Sex in Class Enrollments and Student Ratings of Instructors," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 47(1), pages 135-162, January.
    2. Melanie Lefevre, 2011. "Willingness-to-pay for Local Milk-based Dairy Product in Senegal," CREPP Working Papers 1108, Centre de Recherche en Economie Publique et de la Population (CREPP) (Research Center on Public and Population Economics) HEC-Management School, University of Liège.
    3. Frode Alfnes & Chengyan Yue & Helen H. Jensen, 2010. "Cognitive dissonance as a means of reducing hypothetical bias," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 37(2), pages 147-163, June.
    4. John A. List & Robert P. Berrens & Alok K. Bohara & Joe Kerkvliet, 2004. "Examining the Role of Social Isolation on Stated Preferences," American Economic Review, American Economic Association, vol. 94(3), pages 741-752, June.
    5. Akinwehinmi, Oluwagbenga & Ogundari, Kolawole & Amos, Taiwo, 2021. "Consumers' Food Control Risk Perception and Preference for Government-Controlled Safety Certification in Emerging Food Markets," 2021 Conference, August 17-31, 2021, Virtual 315312, International Association of Agricultural Economists.
    6. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    7. Sarah Shandera & Jes L Matsick & David R Hunter & Louis Leblond, 2021. "RASE: Modeling cumulative disadvantage due to marginalized group status in academia," PLOS ONE, Public Library of Science, vol. 16(12), pages 1-20, December.
    8. Kanchanaroek, Yingluk & Termansen, Mette & Quinn, Claire, 2013. "Property rights regimes in complex fishery management systems: A choice experiment application," Ecological Economics, Elsevier, vol. 93(C), pages 363-373.
    9. Catherine M.H. Keske & Greta Lohman & John B. Loomis, 2013. "Do Respondents Report Willingness-to-Pay on a per Person or per Group Basis? A High Mountain Recreation Example," Tourism Economics, , vol. 19(1), pages 133-145, February.
    10. Blomquist, Glenn C. & Coomes, Paul A. & Jepsen, Christopher & Koford, Brandon C. & Troske, Kenneth R., 2014. "Estimating the social value of higher education: willingness to pay for community and technical colleges," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 5(1), pages 3-41, January.
    11. Barry Bozeman & Gordon Kingsley, 2013. "Research value mapping and evaluation: theory and application," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 7, pages 166-189, Edward Elgar Publishing.
    12. Felix Munoz-Garcia & Sherzod B. Akhundjanov, 2014. "Firm Preferences for Environmental Policy: Industry Uniform or Firm Specific?," Working Papers 2014-8, School of Economic Sciences, Washington State University.
    13. Johanna Lena Dahlhausen & Cam Rungie & Jutta Roosen, 2018. "Value of labeling credence attributes—common structures and individual preferences," Agricultural Economics, International Association of Agricultural Economists, vol. 49(6), pages 741-751, November.
    14. Ayllón, Sara, 2022. "Online teaching and gender bias," Economics of Education Review, Elsevier, vol. 89(C).
    15. Moser, Riccarda & Raffaelli, Roberta, 2011. "Exploiting cut-off information to incorporate context effect: a discrete choice experiment on small fruits in a Alpine region," 2011 International Congress, August 30-September 2, 2011, Zurich, Switzerland 114646, European Association of Agricultural Economists.
    16. John Robst & Jennifer VanGilder, 2016. "The relationship between faculty characteristics and the use of norm- and criteria-based grading," Cogent Economics & Finance, Taylor & Francis Journals, vol. 4(1), pages 1127746-112, December.
    17. Britwum, Kofi & Yiannaka, Amalia, 2019. "Consumer willingness to pay for food safety interventions: The role of message framing and issue involvement," Food Policy, Elsevier, vol. 86(C), pages 1-1.
    18. Adalja, Aaron & Hanson, James & Towe, Charles & Tselepidakis, Elina, 2015. "An Examination of Consumer Willingness to Pay for Local Products," Agricultural and Resource Economics Review, Cambridge University Press, vol. 44(3), pages 253-274, December.
    19. Nicolas Jacquemet & Alexander James & Stéphane Luchini & Jason Shogren, 2011. "Social Psychology and Environmental Economics: A New Look at ex ante Corrections of Biased Preference Evaluation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 48(3), pages 413-433, March.
    20. Paul Heidhues & Botond KH{o}szegi & Philipp Strack, 2019. "Overconfidence and Prejudice," Papers 1909.08497, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:apecpp:v:44:y:2022:i:1:p:110-128. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2040-5804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.