IDEAS home Printed from https://ideas.repec.org/a/gam/jscscx/v8y2019i2p46-d203505.html
   My bibliography  Save this article

Measurement Invariance of a Direct Behavior Rating Multi Item Scale across Occasions

Author

Listed:
  • Markus Gebhardt

    (Research in Inclusive Education, Faculty of Rehabilitation Science, Technical University of Dortmund, 44227 Dortmund, Germany)

  • Jeffrey M. DeVries

    (Research in Inclusive Education, Faculty of Rehabilitation Science, Technical University of Dortmund, 44227 Dortmund, Germany)

  • Jana Jungjohann

    (Research in Inclusive Education, Faculty of Rehabilitation Science, Technical University of Dortmund, 44227 Dortmund, Germany)

  • Gino Casale

    (Department of Special Education, University of Cologne, 50931 Cologne, Germany)

  • Andreas Gegenfurtner

    (Deggendorf Institute of Technology, Institute for Quality and Continuing Education, 94469 Deggendorf, Germany)

  • Jörg-Tobias Kuhn

    (Faculty of Rehabilitation Science, Educational Research Methods, Technical University of Dortmund, 44227 Dortmund, Germany)

Abstract

Direct Behavior Rating (DBR) as a behavioral progress monitoring tool can be designed as longitudinal assessment with only short intervals between measurement points. The reliability of these instruments has been mostly evaluated in observational studies with small samples based on generalizability theory. However, for a standardized use in the pedagogical field, a larger and broader sample is required in order to assess measurement invariance between different participant groups and over time. Therefore, we constructed a DBR, the Questionnaire for Monitoring Behavior in Schools (QMBS) with multiple items to measure the occurrence of specific externalizing and internalizing student classroom behaviors on a Likert scale (1 = never to 7 = always). In a pilot study, two trained raters observed 16 primary education students and rated the student behavior over all items with a satisfactory reliability. In the main study, 108 regular primary school students, 97 regular secondary students, and 14 students in a clinical setting were rated daily over one week (five measurement points). Item response theory (IRT) analyses confirmed the technical adequacy of the instrument and latent growth models demonstrated the instrument’s stability over time. Further development of the instrument and study designs to implement DBRs is discussed.

Suggested Citation

  • Markus Gebhardt & Jeffrey M. DeVries & Jana Jungjohann & Gino Casale & Andreas Gegenfurtner & Jörg-Tobias Kuhn, 2019. "Measurement Invariance of a Direct Behavior Rating Multi Item Scale across Occasions," Social Sciences, MDPI, vol. 8(2), pages 1-14, February.
    • Handle: RePEc:gam:jscscx:v:8:y:2019:i:2:p:46-:d:203505
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2076-0760/8/2/46/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2076-0760/8/2/46/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Thomas Warm, 1989. "Weighted likelihood estimation of ability in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 54(3), pages 427-450, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cherier Mae G. Logroño & Celso L. Tagadiad, 2023. "Instructional Leadership and Ethical Climate as Determinants of School Connectedness," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(1), pages 750-769, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:zbw:rwidps:0002 is not listed on IDEAS
    2. Dahmann, Sarah C., 2017. "How does education improve cognitive skills? Instructional time versus timing of instruction," Labour Economics, Elsevier, vol. 47(C), pages 35-47.
    3. Torberg Falch & Justina AV Fischer, 2008. "Does a generous welfare state crowd out student achievement? Panel data evidence from international student tests," TWI Research Paper Series 31, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    4. Steger, Diana & Schroeders, Ulrich & Wilhelm, Oliver, 2019. "On the dimensionality of crystallized intelligence: A smartphone-based assessment," Intelligence, Elsevier, vol. 72(C), pages 76-85.
    5. Janna Niens & Lisa Richter-Beuschel & Tobias C. Stubbe & Susanne Bögeholz, 2021. "Procedural Knowledge of Primary School Teachers in Madagascar for Teaching and Learning towards Land-Use- and Health-Related Sustainable Development Goals," Sustainability, MDPI, vol. 13(16), pages 1-36, August.
    6. Michela Battauz & Ruggero Bellio, 2011. "Structural Modeling of Measurement Error in Generalized Linear Models with Rasch Measures as Covariates," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 40-56, January.
    7. Xiang Liu & James Yang & Hui Soo Chae & Gary Natriello, 2020. "Power Divergence Family of Statistics for Person Parameters in IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 502-525, June.
    8. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    9. Marko Böhm & Jan Barkmann & Sabina Eggert & Claus H. Carstensen & Susanne Bögeholz, 2020. "Quantitative Modelling and Perspective Taking: Two Competencies of Decision Making for Sustainable Development," Sustainability, MDPI, vol. 12(17), pages 1-32, August.
    10. repec:zbw:rwidps:0023 is not listed on IDEAS
    11. Hammon, Angelina & Zinn, Sabine, 2020. "Multiple imputation of binary multilevel missing not at random data," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 69(3), pages 547-564.
    12. Schmidt, Christoph & Fertig, Michael, 2002. "The Role of Background Factors for Reading Literacy: Straight National scores in the Pisa 2000 Study," CEPR Discussion Papers 3544, C.E.P.R. Discussion Papers.
    13. Yang Liu & Jan Hannig & Abhishek Pal Majumder, 2019. "Second-Order Probability Matching Priors for the Person Parameter in Unidimensional IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 701-718, September.
    14. Robitzsch, Alexander, 2020. "About Still Nonignorable Consequences of (Partially) Ignoring Missing Item Responses in Large-scale Assessment," OSF Preprints hmy45, Center for Open Science.
    15. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    16. Fertig, Michael, 2003. "Educational Production, Endogenous Peer Group Formation and Class Composition - Evidence From the PISA 2000 Study," RWI Discussion Papers 2, RWI - Leibniz-Institut für Wirtschaftsforschung.
    17. Elina Tsigeman & Sebastian Silas & Klaus Frieler & Maxim Likhanov & Rebecca Gelding & Yulia Kovas & Daniel Müllensiefen, 2022. "The Jack and Jill Adaptive Working Memory Task: Construction, Calibration and Validation," PLOS ONE, Public Library of Science, vol. 17(1), pages 1-29, January.
    18. Seonghoon Kim, 2012. "A Note on the Reliability Coefficients for Item Response Model-Based Ability Estimates," Psychometrika, Springer;The Psychometric Society, vol. 77(1), pages 153-162, January.
    19. Falch, Torberg & Justina, Fischer, 2016. "Welfare state generosity and student performance: Evidence from international student tests 1980-2003," MPRA Paper 74553, University Library of Munich, Germany.
    20. Xiao Li & Hanchen Xu & Jinming Zhang & Hua-hua Chang, 2023. "Deep Reinforcement Learning for Adaptive Learning Systems," Journal of Educational and Behavioral Statistics, , vol. 48(2), pages 220-243, April.
    21. Martin Biehler & Heinz Holling & Philipp Doebler, 2015. "Saddlepoint Approximations of the Distribution of the Person Parameter in the Two Parameter Logistic Model," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 665-688, September.
    22. Piero Veronese & Eugenio Melilli, 2021. "Confidence Distribution for the Ability Parameter of the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 131-166, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jscscx:v:8:y:2019:i:2:p:46-:d:203505. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.