IDEAS home Printed from https://ideas.repec.org/a/spr/reihed/v65y2024i7d10.1007_s11162-024-09804-8.html
   My bibliography  Save this article

Student Ratings of Instruction: Updating Measures to Reflect Recent Scholarship

Author

Listed:
  • Kathleen D. Dyer

    (California State University)

  • Dermot Donnelly-Hermosillo

    (California State University)

Abstract

This study aimed to demonstrate how one university worked to overcome some of the measurement problems associated with legacy student rating instruments through the creation and investigation of a new student rating instrument based on the most current scholarship on teaching and learning. Measurement problems with legacy instruments include asking about consumer satisfaction (including the use of global ratings) rather than directly assessing the quality of teaching, asking students for self-reports of learning, and asking students to make judgments about the internal state of their instructors. A new instrument was created to intentionally reduce these problems. The new instrument and its predecessor were both administered by 54 instructors in 81 classes and completed by 2,013 students. The following semester, the new instrument was administered university-wide including 1,450 instructors, 3,669 classes, and 58,320 students. The findings indicate that the new instrument created by this process is both reliable and valid but does not reflect multidimensionality. There is also no compelling evidence of bias according to gender or race. This work illustrates the process by which new and better instruments might be created and tested in order to replace flawed legacy instruments in higher education.

Suggested Citation

  • Kathleen D. Dyer & Dermot Donnelly-Hermosillo, 2024. "Student Ratings of Instruction: Updating Measures to Reflect Recent Scholarship," Research in Higher Education, Springer;Association for Institutional Research, vol. 65(7), pages 1587-1613, November.
    • Handle: RePEc:spr:reihed:v:65:y:2024:i:7:d:10.1007_s11162-024-09804-8
    DOI: 10.1007/s11162-024-09804-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11162-024-09804-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11162-024-09804-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bandiera, Oriana & Larcinese, Valentino & Rasul, Imran, 2015. "Blissful ignorance? A natural experiment on the effect of feedback on students' performance," Labour Economics, Elsevier, vol. 34(C), pages 13-25.
    2. Hamermesh, Daniel S. & Parker, Amy, 2005. "Beauty in the classroom: instructors' pulchritude and putative pedagogical productivity," Economics of Education Review, Elsevier, vol. 24(4), pages 369-376, August.
    3. Boring, Anne, 2017. "Gender biases in student evaluations of teaching," Journal of Public Economics, Elsevier, vol. 145(C), pages 27-41.
    4. Patti Miles & Deanna House, 2015. "The Tail Wagging the Dog; An Overdue Examination of Student Teaching Evaluations," International Journal of Higher Education, Sciedu Press, vol. 4(2), pages 116-116, May.
    5. Alexei Karas, 2021. "The effect of class size on grades and course evaluations: Evidence from multisection courses," Bulletin of Economic Research, Wiley Blackwell, vol. 73(4), pages 624-642, October.
    6. Henson Martin & Pugna Irina Bogdana, 2019. "Organising Principles for the Internationalization of Higher Education," Proceedings of the International Conference on Business Excellence, Sciendo, vol. 13(1), pages 784-791, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Friederike Mengel & Jan Sauermann & Ulf Zölitz, 2019. "Gender Bias in Teaching Evaluations," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 535-566.
    2. Paredes, Valentina & Pino, Francisco J. & Díaz, David, 2024. "Does facial structure explain differences in student evaluations of teaching? The role of fWHR as a proxy for perceived dominance," Economics & Human Biology, Elsevier, vol. 54(C).
    3. Robert L. Moore & Hanna Song & James D. Whitney, 2021. "Do Students Discriminate? Exploring Differentials by Race and Sex in Class Enrollments and Student Ratings of Instructors," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 47(1), pages 135-162, January.
    4. Abel, Martin, 2019. "Do Workers Discriminate against Female Bosses?," IZA Discussion Papers 12611, Institute of Labor Economics (IZA).
    5. Babin, J. Jobu & Hussey, Andrew, 2023. "Gender penalties and solidarity — Teaching evaluation differentials in and out of STEM," Economics Letters, Elsevier, vol. 226(C).
    6. Alkusari, Haneen & Datta Gupta, Nabanita & Etcoff, Nancy, 2024. "In the eye of the promoter? How faculty ratings of attractiveness matter for junior academic careers," Journal of Economic Behavior & Organization, Elsevier, vol. 220(C), pages 645-659.
    7. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    8. Gwen-Jiro Clochard & Guillaume Hollard & Julia Wirtz, 2022. "More effort or better technologies? On the effect of relative performance feedback," Bristol Economics Discussion Papers 22/767, School of Economics, University of Bristol, UK.
    9. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    10. Vanessa, Mertins & Jeworrek, Sabrina & Vlassopoulos, Michael, 2018. ""The Good News about Bad News": Feedback about Past Organisational Failure Bad ist Impact in Worker Productivity," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181644, Verein für Socialpolitik / German Economic Association.
    11. Caliendo, Marco & Gehrsitz, Markus, 2016. "Obesity and the labor market: A fresh look at the weight penalty," Economics & Human Biology, Elsevier, vol. 23(C), pages 209-225.
    12. Ahmed, Shaker & Ranta, Mikko & Vähämaa, Emilia & Vähämaa, Sami, 2023. "Facial attractiveness and CEO compensation: Evidence from the banking industry," Journal of Economics and Business, Elsevier, vol. 123(C).
    13. Olivier Gergaud & Victor Ginsburgh & Florine Livat, 2012. "Success of Celebrities: Talent, Intelligence or Beauty?," Economics Bulletin, AccessEcon, vol. 32(4), pages 3120-3127.
    14. Ayllón, Sara, 2022. "Online teaching and gender bias," Economics of Education Review, Elsevier, vol. 89(C).
    15. Ale J. Hejase & Hussin J. Hejase & Rana S. Al Kaakour, 2014. "The Impact of Students’ Characteristics on their Perceptions of the Evaluation of Teaching Process," International Journal of Management Sciences, Research Academy of Social Sciences, vol. 4(2), pages 90-105.
    16. Julian House & Nicola Lacetera & Mario Macis & Nina Mazar, 2022. "Nudging the Nudger: A Field Experiment on the Effect of Performance Feedback to Service Agents on Increasing Organ Donor Registrations," CESifo Working Paper Series 10012, CESifo.
    17. Rigissa Megalokonomou & Yi Zhang, 2022. "How Good Am I? Effects and Mechanisms behind Salient Ranks," CESifo Working Paper Series 9991, CESifo.
    18. Boring, Anne & Philippe, Arnaud, 2021. "Reducing discrimination in the field: Evidence from an awareness raising intervention targeting gender biases in student evaluations of teaching," Journal of Public Economics, Elsevier, vol. 193(C).
    19. Elif S. Filiz, 2022. "Does it Payoff to be Blond in a Non-Blond Neighborhood? Eye Color, Hair Color, Ethnic Composition and Starting Wages," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 48(1), pages 122-146, January.
    20. Funk, Patricia & Iriberri, Nagore & Savio, Giulia, 2024. "Does scarcity of female instructors create demand for diversity among students? Evidence from an M-Turk experiment," Labour Economics, Elsevier, vol. 90(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reihed:v:65:y:2024:i:7:d:10.1007_s11162-024-09804-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.