IDEAS home Printed from https://ideas.repec.org/a/bla/stanee/v29y1975i3p101-123.html
   My bibliography  Save this article

A review of statistical methods in the analysis of data arising from observer reliability studies (Part I)

Author

Listed:
  • J. Richard Landis
  • Gary G. Koch

Abstract

Summary This paper reviews research situations in medicine, epidemiology and psychiatry, in psychological measurement and testing, and in sample surveys in which the observer(rater or interviewer) can be an important source of measurement error. Moreover, most of the statistical literature in observer variability is surveyed with attention given to a notational unification of the various models proposed. In the continuous data case, the usual analysis of variance (ANOVA) components of variance models are presented with an emphasis on the intraclass correlation coefficient as a measure of reliability. Other modified ANOVA models, response error models in sample surveys, and related multivariate extensions are also discussed. For the categorical data case, special attention is given to measures of agreement and tests of hypotheses when the data consist of dichotomous responses. In addition, similarities between the dichotomous and continous cases are illustrated in terms of intraclass correlation coefficients. Finally, measures of agreement, such as kappa and weighted‐kappa, are discussed in the context of nominal and ordinal data. A proposed unifying framework for the categorical data case is given in the form of concluding remarks.

Suggested Citation

  • J. Richard Landis & Gary G. Koch, 1975. "A review of statistical methods in the analysis of data arising from observer reliability studies (Part I)," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 29(3), pages 101-123, September.
  • Handle: RePEc:bla:stanee:v:29:y:1975:i:3:p:101-123
    DOI: 10.1111/j.1467-9574.1975.tb00254.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9574.1975.tb00254.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-9574.1975.tb00254.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Högberg, Hans & Svensson, Elisabeth, 2008. "An Overview of Methods in the Analysis of Dependent ordered catagorical Data: Assumptions and Implications," Working Papers 2008:7, Örebro University, School of Business.
    2. N. Lu & T. Chen & P. Wu & D. Gunzler & H. Zhang & H. He & X.M. Tu, 2014. "Functional response models for intraclass correlation coefficients," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(11), pages 2539-2556, November.
    3. Debby L Gerritsen & Nardi Steverink & Dinnus HM Frijters & Marcel E Ooms & Miel W Ribbe, 2010. "Social well‐being and its measurement in the nursing home, the SWON‐scale," Journal of Clinical Nursing, John Wiley & Sons, vol. 19(9‐10), pages 1243-1251, May.
    4. Högberg, Hans & Svensson, Elisabeth, 2008. "Comparison of methods in the analysis of dependent ordered catagorical data," Working Papers 2008:6, Örebro University, School of Business.
    5. Cheng, Song-Show & Cheng, Yu-Chun, 1998. "An ordered relation between the ANOVA estimator of the intraclass correlation and a kappa-type statistic in binary data," Statistics & Probability Letters, Elsevier, vol. 38(3), pages 275-280, June.
    6. Fabio Rapallo, 2005. "Algebraic exact inference for rater agreement models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 14(1), pages 45-66, February.
    7. Christof Schuster & David Smith, 2005. "Dispersion-weighted kappa: An integrative framework for metric and nominal scale agreement coefficients," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 135-146, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:stanee:v:29:y:1975:i:3:p:101-123. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0039-0402 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.