IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v48y2014i4p2355-2369.html
   My bibliography  Save this article

Estimating intercoder reliability: a structural equation modeling approach

Author

Listed:
  • Guangchao Feng

Abstract

Intercoder reliability is usually estimated with a summary index, and yet the limitations concerning the indexing approach have been well noted. This study critically reviewed all the existing major modeling approaches to estimating intercoder reliability, and empirically tested and further compared these approaches. It was found that latent variable modeling, also called the second-generation SEM, generally perform better than log-linear modeling, and is able to explain the paradox haunting some indices, and to spot the sources of disagreement among coders. Implications were discussed at last. Copyright Springer Science+Business Media Dordrecht 2014

Suggested Citation

  • Guangchao Feng, 2014. "Estimating intercoder reliability: a structural equation modeling approach," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(4), pages 2355-2369, July.
    • Handle: RePEc:spr:qualqt:v:48:y:2014:i:4:p:2355-2369
    DOI: 10.1007/s11135-014-0034-7
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11135-014-0034-7
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11135-014-0034-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stanley Sclove, 1987. "Application of model-selection criteria to some problems in multivariate analysis," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 333-343, September.
    2. Linzer, Drew A. & Lewis, Jeffrey B., 2011. "poLCA: An R Package for Polytomous Variable Latent Class Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i10).
    3. Agresti, Alan, 1989. "An agreement model with kappa as parameter," Statistics & Probability Letters, Elsevier, vol. 7(4), pages 271-273, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laurent Ott & Mehdi Farsi & Sylvain Weber, 2021. "Beyond political divides: analyzing public opinion on carbon taxation in Switzerland," Chapters, in: Axel Franzen & Sebastian Mader (ed.), Research Handbook on Environmental Sociology, chapter 17, pages 313-339, Edward Elgar Publishing.
    2. Lehrl, Simone & Smidt, Wilfried & Grosse, Christiane & Richter, David, 2014. "Patterns of Literacy and Numeracy Activities in Preschool and Their Relation to Structural Characteristics and Children's Home Activities," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 29(5), pages 577-597.
    3. Aline Riboli Marasca & Maurício Scopel Hoffmann & Anelise Reis Gaya & Denise Ruschel Bandeira, 2021. "Subjective Well-Being and Psychopathology Symptoms: Mental Health Profiles and their Relations with Academic Achievement in Brazilian Children," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 14(3), pages 1121-1137, June.
    4. Collart, Alba J. & Palma, Marco A., 2014. "What Motivates Individuals to Participate in Economic Experiments? A Latent Class Analysis with Unobserved Heterogeneity," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 170401, Agricultural and Applied Economics Association.
    5. Palma, Marco A. & Ness, Meghan L. & Anderson, David P., 2015. "Buying More than Taste? A Latent Class Analysis of Health and Prestige Determinants of Healthy Food," 2015 Conference (59th), February 10-13, 2015, Rotorua, New Zealand 202566, Australian Agricultural and Resource Economics Society.
    6. Adrian O’Hagan & Arthur White, 2019. "Improved model-based clustering performance using Bayesian initialization averaging," Computational Statistics, Springer, vol. 34(1), pages 201-231, March.
    7. Lisa Blaydes, 2023. "Assessing the Labor Conditions of Migrant Domestic Workers in the Arab Gulf States," ILR Review, Cornell University, ILR School, vol. 76(4), pages 724-747, August.
    8. Jindřich Špička & Zdeňka Náglová, 2022. "Consumer segmentation in the meat market - The case study of Czech Republic," Agricultural Economics, Czech Academy of Agricultural Sciences, vol. 68(2), pages 68-77.
    9. Nicholas T. Davis & Kirby Goidel & Yikai Zhao, 2021. "The Meanings of Democracy among Mass Publics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 153(3), pages 849-921, February.
    10. Carter, Virginia & Derudder, Ben & Henríquez, Cristián, 2021. "Assessing local governments’ perception of the potential implementation of biophilic urbanism in Chile: A latent class approach," Land Use Policy, Elsevier, vol. 101(C).
    11. Schreier, Alayna & Stenersen, Madeline R. & Strambler, Michael J. & Marshall, Tim & Bracey, Jeana & Kaufman, Joy S., 2023. "Needs of caregivers of youth enrolled in a statewide system of care: A latent class analysis," Children and Youth Services Review, Elsevier, vol. 147(C).
    12. Assem Abu Hatab & Padmaja Ravula & Swamikannu Nedumaran & Carl-Johan Lagerkvist, 2022. "Perceptions of the impacts of urban sprawl among urban and peri-urban dwellers of Hyderabad, India: a Latent class clustering analysis," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 24(11), pages 12787-12812, November.
    13. Peña, Guillermo & Puente-Ajovín, Miguel & Ramos, Arturo & Sanz-Gracia, Fernando, 2022. "Log-growth rates of CO2: An empirical analysis," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 588(C).
    14. Lorena Charrier & Paola Berchialla & Paola Dalmasso & Alberto Borraccino & Patrizia Lemma & Franco Cavallo, 2019. "Cigarette Smoking and Multiple Health Risk Behaviors: A Latent Class Regression Model to Identify a Profile of Young Adolescents," Risk Analysis, John Wiley & Sons, vol. 39(8), pages 1771-1782, August.
    15. Raphaela Grafiadeli & Heide Glaesmer & Birgit Wagner, 2022. "Loss-Related Characteristics and Symptoms of Depression, Prolonged Grief, and Posttraumatic Stress Following Suicide Bereavement," IJERPH, MDPI, vol. 19(16), pages 1-10, August.
    16. Yikun Luo & Qipeng Chen & Jianyong Chen & Peida Zhan, 2024. "Development and validation of two shortened anxiety sensitive index-3 scales based on item response theory," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-7, December.
    17. Paolo Brunori & Alain Trannoy & Caterina Francesca Guidi, 2021. "Ranking populations in terms of inequality of health opportunity: A flexible latent type approach," Health Economics, John Wiley & Sons, Ltd., vol. 30(2), pages 358-383, February.
    18. Kan, Kees-Jan & van der Maas, Han L.J. & Levine, Stephen Z., 2019. "Extending psychometric network analysis: Empirical evidence against g in favor of mutualism?," Intelligence, Elsevier, vol. 73(C), pages 52-62.
    19. Weissinger, Guy & Shelby Rivers, Alannah & Atte, Tita & Diamond, Guy, 2023. "Suicide risk screening in the school environment: Family factors and profiles," Children and Youth Services Review, Elsevier, vol. 145(C).
    20. William Magee & Sébastien St-Arnaud, 2012. "Models of the Joint Structure of Domain-Related and Global Distress: Implications for the Reconciliation of Quality of Life and Mental Health Perspectives," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 105(1), pages 161-185, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:48:y:2014:i:4:p:2355-2369. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.