IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v11y2021i3p21582440211040121.html
   My bibliography  Save this article

Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance

Author

Listed:
  • Rikkert M. van der Lans
  • Ridwan Maulana
  • Michelle Helms-Lorenz
  • Carmen-María Fernández-García
  • Seyeoung Chun
  • Thelma de Jager
  • Yulia Irnidayanti
  • Mercedes Inda-Caro
  • Okhwa Lee
  • Thys Coetzee
  • Nurul Fadhilah
  • Meae Jeon
  • Peter Moorer

Abstract

This study examines measurement invariance of student perceptions of teaching quality collected in five countries: Indonesia (n students = 6,331), the Netherlands (n students = 6,738), South Africa (n students = 3,422), South Korea (n students = 6,997) and Spain (n students = 4,676). The administered questionnaire was the My Teacher Questionnaire (MTQ). Student perceived teachers’ teaching quality was estimated using the partial credit model (PCM). Tests for differential item functioning (DIF) were used to assess measurement invariance. Furthermore, if DIF was found, it was explored whether an application of a quasi-international calibration, which estimates country-unique parameters for DIF items, can provide more valid estimates for between-country comparisons. Results indicate the absence of non-uniform DIF, but presence of uniform DIF among most items. This suggests that direct comparisons of raw mean or sum scores between countries is not advisable. Details of the set of invariant items are provided. Furthermore, results suggest that the quasi-international calibration is promising, but also that this approach needs further exploration in the context of student perceptions of teaching quality.

Suggested Citation

  • Rikkert M. van der Lans & Ridwan Maulana & Michelle Helms-Lorenz & Carmen-María Fernández-García & Seyeoung Chun & Thelma de Jager & Yulia Irnidayanti & Mercedes Inda-Caro & Okhwa Lee & Thys Coetze, 2021. "Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance," SAGE Open, , vol. 11(3), pages 21582440211, August.
    • Handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121
    DOI: 10.1177/21582440211040121
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/21582440211040121
    Download Restriction: no
    File URL: https://libkey.io/10.1177/21582440211040121?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ralph Hippe & Maciej Jakubowski & Luisa De Sousa Lobo Borges de Araujo, 2018. "Regional inequalities in PISA: the case of Italy and Spain," JRC Research Reports JRC109057, Joint Research Centre.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. John Horn, 1965. "A rationale and test for the number of factors in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 30(2), pages 179-185, June.
    4. Mair, Patrick & Hatzinger, Reinhold, 2007. "Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i09).
    5. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    6. Silvia Bacci & Valeria Caviezel, 2011. "Multilevel IRT models for the university teaching evaluation," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2775-2791, February.
    7. Michael Browne, 1992. "Circumplex models for correlation matrices," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 469-497, December.
    8. Tazeen Fasih & Rythia Afkar & Heather Tomlinson & Rythia Afkar, 2018. "Learning for All," World Bank Publications - Reports 29379, The World Bank Group.
    9. Isaac M. Mbiti, 2016. "The Need for Accountability in Education in Developing Countries," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 109-132, Summer.
    10. Choi, Seung W. & Gibbons, Laura E. & Crane, Paul K., 2011. "lordif: An R Package for Detecting Differential Item Functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 39(i08).
    11. Weeks, Jonathan P., 2010. "plink: An R Package for Linking Mixed-Format Tests Using IRT-Based Methods," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 35(i12).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    2. Kai Liu & Longfei Zhang & Dongbo Tu & Yan Cai, 2022. "Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students," SAGE Open, , vol. 12(4), pages 21582440221, December.
    3. Carla Sabariego & Cornelia Oberhauser & Aleksandra Posarac & Jerome Bickenbach & Nenad Kostanjsek & Somnath Chatterji & Alana Officer & Michaela Coenen & Lay Chhan & Alarcos Cieza, 2015. "Measuring Disability: Comparing the Impact of Two Data Collection Approaches on Disability Rates," IJERPH, MDPI, vol. 12(9), pages 1-23, August.
    4. Brzezińska Justyna, 2018. "Item Response Theory Models in the Measurement Theory with the Use of ltm Package in R," Econometrics. Advances in Applied Data Analysis, Sciendo, vol. 22(1), pages 11-25, March.
    5. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    6. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    7. Yikun Luo & Qipeng Chen & Jianyong Chen & Peida Zhan, 2024. "Development and validation of two shortened anxiety sensitive index-3 scales based on item response theory," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-7, December.
    8. Salzberger, Thomas & Koller, Monika, 2013. "Towards a new paradigm of measurement in marketing," Journal of Business Research, Elsevier, vol. 66(9), pages 1307-1317.
    9. Joakim Wallmark & James O. Ramsay & Juan Li & Marie Wiberg, 2024. "Analyzing Polytomous Test Data: A Comparison Between an Information-Based IRT Model and the Generalized Partial Credit Model," Journal of Educational and Behavioral Statistics, , vol. 49(5), pages 753-779, October.
    10. Wenling Liu & Ziping Xu & Tianan Yang, 2018. "Health Effects of Air Pollution in China," IJERPH, MDPI, vol. 15(7), pages 1-15, July.
    11. repec:jss:jstsof:36:c01 is not listed on IDEAS
    12. Jeanne A. Teresi & Katja Ocepek-Welikson & John A. Toner & Marjorie Kleinman & Mildred Ramirez & Joseph P. Eimicke & Barry J. Gurland & Albert Siu, 2017. "Methodological Issues in Measuring Subjective Well-Being and Quality-of-Life: Applications to Assessment of Affect in Older, Chronically and Cognitively Impaired, Ethnically Diverse Groups Using the F," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(2), pages 251-288, June.
    13. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    14. Silvia Bacci & Michela Gnaldi, 2015. "A classification of university courses based on students’ satisfaction: an application of a two-level mixture item response model," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 927-940, May.
    15. Galeotti, Marzio & Rubashkina, Yana & Salini, Silvia & Verdolini, Elena, 2018. "Environmental policy performance and its determinants: Application of a three-level random intercept model," Energy Policy, Elsevier, vol. 114(C), pages 134-144.
    16. Shaojie Wang & Won-Chan Lee & Huixia Ma, 2024. "The Chinese Version of the Couples Satisfaction Index: Psychometric Assessment and Differential Item Functioning Analysis with Item Response Theory," SAGE Open, , vol. 14(3), pages 21582440241, August.
    17. Muhammad Sholihin, 2024. "Developing the halal-sufficiency scale: a preliminary insight," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(4), pages 3585-3604, August.
    18. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    19. repec:jss:jstsof:35:i12 is not listed on IDEAS
    20. Bacci, Silvia & Fabbricatore, Rosa & Iannario, Maria, 2023. "Multilevel IRT models for the analysis of satisfaction for distance learning during the Covid-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 86(C).
    21. Gerhard Tutz, 2022. "Item Response Thresholds Models: A General Class of Models for Varying Types of Items," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1238-1269, December.
    22. Daphna Harel & Russell J. Steele, 2018. "An Information Matrix Test for the Collapsing of Categories Under the Partial Credit Model," Journal of Educational and Behavioral Statistics, , vol. 43(6), pages 721-750, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.