IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v11y2021i3p21582440211040121.html
   My bibliography  Save this article

Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance

Author

Listed:
  • Rikkert M. van der Lans
  • Ridwan Maulana
  • Michelle Helms-Lorenz
  • Carmen-María Fernández-García
  • Seyeoung Chun
  • Thelma de Jager
  • Yulia Irnidayanti
  • Mercedes Inda-Caro
  • Okhwa Lee
  • Thys Coetzee
  • Nurul Fadhilah
  • Meae Jeon
  • Peter Moorer

Abstract

This study examines measurement invariance of student perceptions of teaching quality collected in five countries: Indonesia (n students = 6,331), the Netherlands (n students = 6,738), South Africa (n students = 3,422), South Korea (n students = 6,997) and Spain (n students = 4,676). The administered questionnaire was the My Teacher Questionnaire (MTQ). Student perceived teachers’ teaching quality was estimated using the partial credit model (PCM). Tests for differential item functioning (DIF) were used to assess measurement invariance. Furthermore, if DIF was found, it was explored whether an application of a quasi-international calibration, which estimates country-unique parameters for DIF items, can provide more valid estimates for between-country comparisons. Results indicate the absence of non-uniform DIF, but presence of uniform DIF among most items. This suggests that direct comparisons of raw mean or sum scores between countries is not advisable. Details of the set of invariant items are provided. Furthermore, results suggest that the quasi-international calibration is promising, but also that this approach needs further exploration in the context of student perceptions of teaching quality.

Suggested Citation

  • Rikkert M. van der Lans & Ridwan Maulana & Michelle Helms-Lorenz & Carmen-María Fernández-García & Seyeoung Chun & Thelma de Jager & Yulia Irnidayanti & Mercedes Inda-Caro & Okhwa Lee & Thys Coetze, 2021. "Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance," SAGE Open, , vol. 11(3), pages 21582440211, August.
    • Handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121
    DOI: 10.1177/21582440211040121
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/21582440211040121
    Download Restriction: no
    File URL: https://libkey.io/10.1177/21582440211040121?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ralph Hippe & Maciej Jakubowski & Luisa De Sousa Lobo Borges de Araujo, 2018. "Regional inequalities in PISA: the case of Italy and Spain," JRC Research Reports JRC109057, Joint Research Centre.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. John Horn, 1965. "A rationale and test for the number of factors in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 30(2), pages 179-185, June.
    4. Mair, Patrick & Hatzinger, Reinhold, 2007. "Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i09).
    5. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    6. Silvia Bacci & Valeria Caviezel, 2011. "Multilevel IRT models for the university teaching evaluation," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2775-2791, February.
    7. Michael Browne, 1992. "Circumplex models for correlation matrices," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 469-497, December.
    8. Tazeen Fasih & Rythia Afkar & Heather Tomlinson, 2018. "Learning for All," World Bank Publications - Reports 29379, The World Bank Group.
    9. Isaac M. Mbiti, 2016. "The Need for Accountability in Education in Developing Countries," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 109-132, Summer.
    10. Choi, Seung W. & Gibbons, Laura E. & Crane, Paul K., 2011. "lordif: An R Package for Detecting Differential Item Functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 39(i08).
    11. Weeks, Jonathan P., 2010. "plink: An R Package for Linking Mixed-Format Tests Using IRT-Based Methods," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 35(i12).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    2. Carla Sabariego & Cornelia Oberhauser & Aleksandra Posarac & Jerome Bickenbach & Nenad Kostanjsek & Somnath Chatterji & Alana Officer & Michaela Coenen & Lay Chhan & Alarcos Cieza, 2015. "Measuring Disability: Comparing the Impact of Two Data Collection Approaches on Disability Rates," IJERPH, MDPI, vol. 12(9), pages 1-23, August.
    3. Brzezińska Justyna, 2018. "Item Response Theory Models in the Measurement Theory with the Use of ltm Package in R," Econometrics. Advances in Applied Data Analysis, Sciendo, vol. 22(1), pages 11-25, March.
    4. Kai Liu & Longfei Zhang & Dongbo Tu & Yan Cai, 2022. "Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students," SAGE Open, , vol. 12(4), pages 21582440221, December.
    5. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    6. Yikun Luo & Qipeng Chen & Jianyong Chen & Peida Zhan, 2024. "Development and validation of two shortened anxiety sensitive index-3 scales based on item response theory," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-7, December.
    7. Salzberger, Thomas & Koller, Monika, 2013. "Towards a new paradigm of measurement in marketing," Journal of Business Research, Elsevier, vol. 66(9), pages 1307-1317.
    8. repec:jss:jstsof:36:c01 is not listed on IDEAS
    9. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    10. Silvia Bacci & Michela Gnaldi, 2015. "A classification of university courses based on students’ satisfaction: an application of a two-level mixture item response model," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 927-940, May.
    11. repec:jss:jstsof:35:i12 is not listed on IDEAS
    12. Bacci, Silvia & Fabbricatore, Rosa & Iannario, Maria, 2023. "Multilevel IRT models for the analysis of satisfaction for distance learning during the Covid-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 86(C).
    13. Laura Maldonado-Murciano & Halley M. Pontes & Mark D. Griffiths & Maite Barrios & Juana Gómez-Benito & Georgina Guilera, 2020. "The Spanish Version of the Internet Gaming Disorder Scale-Short Form (IGDS9-SF): Further Examination Using Item Response Theory," IJERPH, MDPI, vol. 17(19), pages 1-14, September.
    14. Benjamin D. Schalet & Sangdon Lim & David Cella & Seung W. Choi, 2021. "Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 717-746, September.
    15. Sati Bozkurt & Gizem B. Ekitli & Christopher L. Thomas & Jerrell C. Cassady, 2017. "Validation of the Turkish Version of the Cognitive Test Anxiety Scale–Revised," SAGE Open, , vol. 7(1), pages 21582440166, January.
    16. Wenjia Wang & Mickaël Guedj & Viviane Bertrand & Julie Foucquier & Elisabeth Jouve & Daniel Commenges & Cécile Proust-Lima & Niall P Murphy & Olivier Blin & Laurent Magy & Daniel Cohen & Shahram Attar, 2017. "A Rasch Analysis of the Charcot-Marie-Tooth Neuropathy Score (CMTNS) in a Cohort of Charcot-Marie-Tooth Type 1A Patients," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-14, January.
    17. Galeotti, Marzio & Rubashkina, Yana & Salini, Silvia & Verdolini, Elena, 2018. "Environmental policy performance and its determinants: Application of a three-level random intercept model," Energy Policy, Elsevier, vol. 114(C), pages 134-144.
    18. Clemens Draxler & Andreas Kurz & Can Gürer & Jan Philipp Nolte, 2024. "An Improved Inferential Procedure to Evaluate Item Discriminations in a Conditional Maximum Likelihood Framework," Journal of Educational and Behavioral Statistics, , vol. 49(3), pages 403-430, June.
    19. Sara Fernandes & Guillaume Fond & Xavier Zendjidjian & Pierre Michel & Karine Baumstarck & Christophe Lançon & Ludovic Samalin & Pierre-Michel Llorca & Magali Coldefy & Pascal Auquier & Laurent Boyer , 2022. "Development and Calibration of the PREMIUM Item Bank for Measuring Respect and Dignity for Patients with Severe Mental Illness," Post-Print hal-03649277, HAL.
    20. Felix Zimmer & Clemens Draxler & Rudolf Debelak, 2023. "Power Analysis for the Wald, LR, Score, and Gradient Tests in a Marginal Maximum Likelihood Framework: Applications in IRT," Psychometrika, Springer;The Psychometric Society, vol. 88(4), pages 1249-1298, December.
    21. Marius Leckelt & Eunike Wetzel & Tanja M. Gerlach & Robert A. Ackerman & Joshua D. Miller & William J. Chopik & Lars Penke & Katharina Geukes & Albrecht C. P. Küfner & Roos Hutteman & David Richter & , 2016. "Validation of the Narcissistic Admiration and Rivalry Questionnaire Short Scale (NARQ-S) in Convenience and Representative Samples," SOEPpapers on Multidisciplinary Panel Data Research 884, DIW Berlin, The German Socio-Economic Panel (SOEP).
    22. Jürgen M. Pelikan & Thomas Link & Christa Straßmayr & Karin Waldherr & Tobias Alfers & Henrik Bøggild & Robert Griebler & Maria Lopatina & Dominika Mikšová & Marie Germund Nielsen & Sandra Peer & Mitj, 2022. "Measuring Comprehensive, General Health Literacy in the General Adult Population: The Development and Validation of the HLS 19 -Q12 Instrument in Seventeen Countries," IJERPH, MDPI, vol. 19(21), pages 1-31, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.