IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v48y2014i4p2355-2369.html
   My bibliography  Save this article

Estimating intercoder reliability: a structural equation modeling approach

Author

Listed:
  • Guangchao Feng

Abstract

Intercoder reliability is usually estimated with a summary index, and yet the limitations concerning the indexing approach have been well noted. This study critically reviewed all the existing major modeling approaches to estimating intercoder reliability, and empirically tested and further compared these approaches. It was found that latent variable modeling, also called the second-generation SEM, generally perform better than log-linear modeling, and is able to explain the paradox haunting some indices, and to spot the sources of disagreement among coders. Implications were discussed at last. Copyright Springer Science+Business Media Dordrecht 2014

Suggested Citation

  • Guangchao Feng, 2014. "Estimating intercoder reliability: a structural equation modeling approach," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(4), pages 2355-2369, July.
    • Handle: RePEc:spr:qualqt:v:48:y:2014:i:4:p:2355-2369
    DOI: 10.1007/s11135-014-0034-7
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11135-014-0034-7
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11135-014-0034-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Linzer, Drew A. & Lewis, Jeffrey B., 2011. "poLCA: An R Package for Polytomous Variable Latent Class Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i10).
    2. Agresti, Alan, 1989. "An agreement model with kappa as parameter," Statistics & Probability Letters, Elsevier, vol. 7(4), pages 271-273, February.
    3. Stanley Sclove, 1987. "Application of model-selection criteria to some problems in multivariate analysis," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 333-343, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aline Riboli Marasca & Maurício Scopel Hoffmann & Anelise Reis Gaya & Denise Ruschel Bandeira, 2021. "Subjective Well-Being and Psychopathology Symptoms: Mental Health Profiles and their Relations with Academic Achievement in Brazilian Children," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 14(3), pages 1121-1137, June.
    2. Laurent Ott & Mehdi Farsi & Sylvain Weber, 2021. "Beyond political divides: analyzing public opinion on carbon taxation in Switzerland," Chapters, in: Axel Franzen & Sebastian Mader (ed.), Research Handbook on Environmental Sociology, chapter 17, pages 313-339, Edward Elgar Publishing.
    3. Collart, Alba J. & Palma, Marco A., 2014. "What Motivates Individuals to Participate in Economic Experiments? A Latent Class Analysis with Unobserved Heterogeneity," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 170401, Agricultural and Applied Economics Association.
    4. Lehrl, Simone & Smidt, Wilfried & Grosse, Christiane & Richter, David, 2014. "Patterns of Literacy and Numeracy Activities in Preschool and Their Relation to Structural Characteristics and Children's Home Activities," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 29(5), pages 577-597.
    5. Jiwon Lee & Midam An & Yongku Kim & Jung-In Seo, 2021. "Optimal Allocation for Electric Vehicle Charging Stations," Energies, MDPI, vol. 14(18), pages 1-10, September.
    6. Palma, Marco A. & Ness, Meghan L. & Anderson, David P., 2015. "Buying More than Taste? A Latent Class Analysis of Health and Prestige Determinants of Healthy Food," 2015 Conference (59th), February 10-13, 2015, Rotorua, New Zealand 202566, Australian Agricultural and Resource Economics Society.
    7. Alan Crane & Kevin Crotty, 2020. "How Skilled Are Security Analysts?," Journal of Finance, American Finance Association, vol. 75(3), pages 1629-1675, June.
    8. Adrian O’Hagan & Arthur White, 2019. "Improved model-based clustering performance using Bayesian initialization averaging," Computational Statistics, Springer, vol. 34(1), pages 201-231, March.
    9. Lisa Blaydes, 2023. "Assessing the Labor Conditions of Migrant Domestic Workers in the Arab Gulf States," ILR Review, Cornell University, ILR School, vol. 76(4), pages 724-747, August.
    10. Jindřich Špička & Zdeňka Náglová, 2022. "Consumer segmentation in the meat market - The case study of Czech Republic," Agricultural Economics, Czech Academy of Agricultural Sciences, vol. 68(2), pages 68-77.
    11. Nicholas T. Davis & Kirby Goidel & Yikai Zhao, 2021. "The Meanings of Democracy among Mass Publics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 153(3), pages 849-921, February.
    12. Marcus Heise & Astrid Fink & Jens Baumert & Christin Heidemann & Yong Du & Thomas Frese & Solveig Carmienke, 2021. "Patterns and associated factors of diabetes self-management: Results of a latent class analysis in a German population-based study," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-23, March.
    13. Urška Smrke & Nejc Plohl & Izidor Mlakar, 2022. "Aging Adults’ Motivation to Use Embodied Conversational Agents in Instrumental Activities of Daily Living: Results of Latent Profile Analysis," IJERPH, MDPI, vol. 19(4), pages 1-11, February.
    14. Hamparsum Bozdogan, 1987. "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 345-370, September.
    15. Carter, Virginia & Derudder, Ben & Henríquez, Cristián, 2021. "Assessing local governments’ perception of the potential implementation of biophilic urbanism in Chile: A latent class approach," Land Use Policy, Elsevier, vol. 101(C).
    16. Schreier, Alayna & Stenersen, Madeline R. & Strambler, Michael J. & Marshall, Tim & Bracey, Jeana & Kaufman, Joy S., 2023. "Needs of caregivers of youth enrolled in a statewide system of care: A latent class analysis," Children and Youth Services Review, Elsevier, vol. 147(C).
    17. Francesco BARTOLUCCI & Silvia BACCI & Claudia PIGINI, 2015. "A Misspecification Test for Finite-Mixture Logistic Models for Clustered Binary and Ordered Responses," Working Papers 410, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    18. Arjan S. Gosal & Janine A. McMahon & Katharine M. Bowgen & Catherine H. Hoppe & Guy Ziv, 2021. "Identifying and Mapping Groups of Protected Area Visitors by Environmental Awareness," Land, MDPI, vol. 10(6), pages 1-14, May.
    19. Assem Abu Hatab & Padmaja Ravula & Swamikannu Nedumaran & Carl-Johan Lagerkvist, 2022. "Perceptions of the impacts of urban sprawl among urban and peri-urban dwellers of Hyderabad, India: a Latent class clustering analysis," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 24(11), pages 12787-12812, November.
    20. Martin Eling & David Pankoke, 2016. "Costs and Benefits of Financial Regulation: An Empirical Assessment for Insurance Companies," The Geneva Papers on Risk and Insurance - Issues and Practice, Palgrave Macmillan;The Geneva Association, vol. 41(4), pages 529-554, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:48:y:2014:i:4:p:2355-2369. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.