IDEAS home Printed from https://ideas.repec.org/p/qld/uq2004/516.html
   My bibliography  Save this paper

Does the student evaluation of teaching instrument really measure instructors teaching effectiveness? An econometric analysis of students perceptions in economics courses

Author

Abstract

While the student evaluation of teaching (SET) has been an intensely researched area in higher education there has been little research using the individual student responses on their perceptions of instructors’ effectiveness (TEVAL) score. This research delivers a methodological breakthrough as it fills this gap by employing individual student responses from an elite Australian university and partial proportional odds model to investigate the influence of students’ perceptions of instructional attributes included in the SET instrument and other variables on TEVAL score. The findings indicate presentation and explanation of materials, and organization of classes were key determinants of TEVAL scores. Emphasis on thinking rather than memorising was less influential. Intermediate level courses and non-English speaking background instructors received lower TEVAL scores. Elective courses and instructors below associate professor attracted higher ratings. The SET instrument currently used fails to provide a valid measure teaching quality as it does little to measure the extent of students’ actual learning. This paper underscores the need for inclusion of variables typifying diversity of the student population such as academic performance, discipline destination, ethno-linguistic background, age, sex, indicators of students’ effort. It raises broader implications such as subscales, inclusion of items on course contents, intellectual challenge, real world applications, and problem-solving skills.

Suggested Citation

  • Mohammad Alauddin & Temesgen Kifle, 2014. "Does the student evaluation of teaching instrument really measure instructors teaching effectiveness? An econometric analysis of students perceptions in economics courses," Discussion Papers Series 516, School of Economics, University of Queensland, Australia.
    • Handle: RePEc:qld:uq2004:516
    as

    Download full text from publisher

    File URL: https://economics.uq.edu.au/files/45960/516.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Anthony Onwuegbuzie & Larry Daniel & Kathleen Collins, 2009. "A meta-validation model for assessing the score-validity of student teaching evaluations," Quality & Quantity: International Journal of Methodology, Springer, vol. 43(2), pages 197-209, March.
    2. Mason, Paul M. & Steagall, Jeffrey W. & Fabritius, Michael M., 2003. "The changing quality of business education," Economics of Education Review, Elsevier, vol. 22(6), pages 603-609, December.
    3. Andrew J. Martin & Rachel Wilson & Gregory Arief D. Liem & Paul Ginns, 2013. "Academic Momentum at University/College: Exploring the Roles of Prior Learning, Life Experience, and Ongoing Performance in Academic Achievement across Time," The Journal of Higher Education, Taylor & Francis Journals, vol. 84(5), pages 640-674, September.
    4. Philip Babcock & Mindy Marks, 2011. "The Falling Time Cost of College: Evidence from Half a Century of Time Use Data," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 468-478, May.
    5. Mohammad Alauddin & Hong Son Nghiem, 2010. "Do instructional attributes pose multicollinearity problems? An empirical exploration," Economic Analysis and Policy, Elsevier, vol. 40(3), pages 351-361, December.
    6. Scott E. Carrell & James E. West, 2010. "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors," Journal of Political Economy, University of Chicago Press, vol. 118(3), pages 409-432, June.
    7. George A. Akerlof, 1970. "The Market for "Lemons": Quality Uncertainty and the Market Mechanism," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 84(3), pages 488-500.
    8. J. Scott Long & Jeremy Freese, 2006. "Regression Models for Categorical Dependent Variables using Stata, 2nd Edition," Stata Press books, StataCorp LP, edition 2, number long2, March.
    9. Mason, Paul M. & Steagall, Jeffrey W. & Fabritius, Michael M., 1995. "Student evaluations of faculty: A new procedure for using aggregate measures of performance," Economics of Education Review, Elsevier, vol. 14(4), pages 403-416, December.
    10. Calin Valsan & Robert Sproule, 2008. "The Invisible Hands behind the Student Evaluation of Teaching: The Rise of the New Managerial Elite in the Governance of Higher Education," Journal of Economic Issues, Taylor & Francis Journals, vol. 42(4), pages 939-958, December.
    11. Griliches, Zvi, 1974. "Errors in Variables and Other Unobservables," Econometrica, Econometric Society, vol. 42(6), pages 971-998, November.
    12. Mohammad Alauddin & Clem Tisdell, 2000. "Changing Academic Environment And Teaching Of Economics At The University Level: Some Critical Issues Analysed With The Help Of Microeconomics," Economic Papers, The Economic Society of Australia, vol. 19(1), pages 1-17, March.
    13. David Colander, 2004. "The Art of Teaching Economics," International Review of Economic Education, Economics Network, University of Bristol, vol. 3(1), pages 63-76.
    14. Mohammad Alauddin & Adrian Ashman, 2014. "The changing academic environment and diversity in students study philosophy, beliefs and attitudes in higher education," Discussion Papers Series 511, School of Economics, University of Queensland, Australia.
    15. William E. Becker, 2000. "Teaching Economics in the 21st Century," Journal of Economic Perspectives, American Economic Association, vol. 14(1), pages 109-119, Winter.
    16. L. F. Jameson Boex, 2000. "Attributes of Effective Economics Instructors: An Analysis of Student Evaluations," The Journal of Economic Education, Taylor & Francis Journals, vol. 31(3), pages 211-227, September.
    17. R. Todd Jewell & Michael A. McPherson & Margie A. Tieslau, 2013. "Whose fault is it? Assigning blame for grade inflation in higher education," Applied Economics, Taylor & Francis Journals, vol. 45(9), pages 1185-1200, March.
    18. Sproule, Robert, 2002. "The underdetermination of instructor performance by data from the student evaluation of teaching," Economics of Education Review, Elsevier, vol. 21(3), pages 287-294, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Berezvai, Zombor & Lukáts, Gergely Dániel & Molontay, Roland, 2019. "A pénzügyi ösztönzők hatása az egyetemi oktatók osztályozási gyakorlatára [How financially rewarding student evaluation may affect grading behaviour. Evidence from a natural experiment]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 733-750.
    2. Fernando Suárez & Juan Carlos Mosquera Feijóo & Isabel Chiyón & Marcos García Alberti, 2021. "Flipped Learning in Engineering Modules Is More Than Watching Videos: The Development of Personal and Professional Skills," Sustainability, MDPI, vol. 13(21), pages 1-20, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohammad Alauddin & Clem Tisdell, 2007. "Factors That Affect Teaching Scores in Economics Instruction: Analysis of Student Evaluation of Teaching (SET) Data," Discussion Papers Series 353, School of Economics, University of Queensland, Australia.
    2. Sam Allgood & William B. Walstad & John J. Siegfried, 2015. "Research on Teaching Economics to Undergraduates," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 285-325, June.
    3. Mohammad Alauddin & Adrian Ashman, 2014. "The changing academic environment and diversity in students study philosophy, beliefs and attitudes in higher education," Discussion Papers Series 511, School of Economics, University of Queensland, Australia.
    4. Dr. Mohammad Alauddin & Professor John Foster, 2005. "Heterogenous clientele and product differentiation: teaching economics in a changing environment," Discussion Papers Series 340, School of Economics, University of Queensland, Australia.
    5. Dr. Mohammad Alauddin & Professor John Foster, 2005. "Teaching Economics at the University Level: Dynamics of Parameters and Implications," Discussion Papers Series 339, School of Economics, University of Queensland, Australia.
    6. Langbein, Laura, 2008. "Management by results: Student evaluation of faculty teaching and the mis-measurement of performance," Economics of Education Review, Elsevier, vol. 27(4), pages 417-428, August.
    7. Beleche, Trinidad & Fairris, David & Marks, Mindy, 2012. "Do course evaluations truly reflect student learning? Evidence from an objectively graded post-test," Economics of Education Review, Elsevier, vol. 31(5), pages 709-719.
    8. Kaganovich, Michael & Sarpca, Sinan & Su, Xuejuan, 2020. "Competition in Higher Education," Working Papers 2020-3, University of Alberta, Department of Economics.
    9. repec:spo:wpmain:info:hdl:2441/1seuirq4ak9b9bouu1j29ebui7 is not listed on IDEAS
    10. Anne Boring, 2015. "Gender Biases in Student Evaluations of Teachers," Working Papers hal-03470161, HAL.
    11. Pearce, John A., 2017. "How employers can stanch the hemorrhaging of collegiate GPA credibility," Business Horizons, Elsevier, vol. 60(1), pages 35-43.
    12. Paul Dalziel, 2011. "Schumpeter's 'Vision' and the Teaching of Principles of Economics to Resource Students," International Review of Economic Education, Economics Network, University of Bristol, vol. 10(2), pages 63-74.
    13. Armstrong, J. Scott, 2012. "Natural Learning in Higher Education," MPRA Paper 37648, University Library of Munich, Germany.
    14. Avdjiev, Stefan & Binder, Stephan & Sousa, Ricardo, 2021. "External debt composition and domestic credit cycles," Journal of International Money and Finance, Elsevier, vol. 115(C).
    15. Palali, Ali & van Elk, Roel & Bolhaar, Jonneke & Rud, Iryna, 2018. "Are good researchers also good teachers? The relationship between research quality and teaching quality," Economics of Education Review, Elsevier, vol. 64(C), pages 40-49.
    16. Rieger, Matthias & Voorvelt, Katherine, 2016. "Gender, ethnicity and teaching evaluations: Evidence from mixed teaching teamsAuthor-Name: Wagner, Natascha," Economics of Education Review, Elsevier, vol. 54(C), pages 79-94.
    17. Michael Kaganovich & Sinan Sarpca & Xuejuan Su, 2020. "Competition in Higher Education: A Survey," CESifo Working Paper Series 8220, CESifo.
    18. Ferdinand V. N. Houeho & Casim A. Hounkanrin & Boris O. K. Lokonon, 2024. "Main Channels of Job Search in the Beninese Labour Market: The Case of Unemployed Young," The Indian Journal of Labour Economics, Springer;The Indian Society of Labour Economics (ISLE), vol. 67(2), pages 571-592, June.
    19. Alauddin, Mohammad & Valadkhani, Abbas, 2003. "Causes and Implications of Declining Economics Major: A Focus on Australia," MPRA Paper 50393, University Library of Munich, Germany.
    20. Ali Palali & Roel van Elk & Jonneke Bolhaar & Iryna Rud, 2017. "Are good researchers also good teachers? The relationship between research quality and teaching quality," CPB Discussion Paper 347.rdf, CPB Netherlands Bureau for Economic Policy Analysis.
    21. Berezvai, Zombor & Lukáts, Gergely Dániel & Molontay, Roland, 2019. "A pénzügyi ösztönzők hatása az egyetemi oktatók osztályozási gyakorlatára [How financially rewarding student evaluation may affect grading behaviour. Evidence from a natural experiment]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 733-750.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qld:uq2004:516. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SOE IT (email available below). General contact details of provider: https://edirc.repec.org/data/decuqau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.