IDEAS home Printed from https://ideas.repec.org/a/bpj/ijbist/v9y2013i1p14n1.html
   My bibliography  Save this article

Assessing Agreement of Repeated Binary Measurements with an Application to the CDC’s Anthrax Vaccine Clinical Trial

Author

Listed:
  • Pan Yi

    (Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA; Logistics Health Inc, La Crosse, WI, USA)

  • Rose Charles E.

    (Division of Bacterial Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA)

  • Haber Michael

    (Department of Biostatistics and Bioinformatics, Emory University, Rollins School of Public Health, Atlanta, GA, USA)

  • Ma Yan

    (Biostatistics, Public Health Department, Hospital for Special Surgery, Weill Medical College of Cornell University, New York City, NY, USA)

  • Carrasco Josep L.

    (University of Barcelona, Barcelona, Spain)

  • Stewart Brock

    (Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA)

  • Keitel Wendy A.

    (Baylor College of Medicine, Houston, TX, USA)

  • Keyserling Harry

    (Emory University School of Medicine, Atlanta, GA, USA)

  • Jacobson Robert M.

    (Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA)

  • Poland Gregory

    (Mayo Clinic, Rochester, MN, USA)

  • McNeil Michael M.

    (Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention; Centers for Disease Control and Prevention, 1600 Clifton Road NE, Atlanta, GA 30333)

Abstract

Cohen’s kappa coefficient, which was introduced in 1960, serves as the most widely employed coefficient to assess inter-observer agreement for categorical outcomes. However, the original kappa can only be applied to cross-sectional binary measurements and, therefore, cannot be applied in the practical situation when the observers evaluate the same subjects at repeated time intervals. This study summarizes six methods of assessing agreement of repeated binary outcomes under different assumptions and discusses under which condition we should use the most appropriate method in practice. These approaches are illustrated using data from the CDC anthrax vaccine adsorbed (AVA) human clinical trial comparing the agreement for two solicited adverse events after AVA between the 1–3 day in-clinic medical record and the patient’s diary on the same day. We hope this article can inspire researchers to choose the most appropriate method to assess agreement for their own study with longitudinal binary data.

Suggested Citation

  • Pan Yi & Rose Charles E. & Haber Michael & Ma Yan & Carrasco Josep L. & Stewart Brock & Keitel Wendy A. & Keyserling Harry & Jacobson Robert M. & Poland Gregory & McNeil Michael M., 2013. "Assessing Agreement of Repeated Binary Measurements with an Application to the CDC’s Anthrax Vaccine Clinical Trial," The International Journal of Biostatistics, De Gruyter, vol. 9(1), pages 19-32, July.
    • Handle: RePEc:bpj:ijbist:v:9:y:2013:i:1:p:14:n:1
    DOI: 10.1515/ijb-2012-0001
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/ijb-2012-0001
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/ijb-2012-0001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Martin S. Ridout & Clarice G. B. Demétrio & David Firth, 1999. "Estimating Intraclass Correlation for Binary Data," Biometrics, The International Biometric Society, vol. 55(1), pages 137-148, March.
    2. Yan Ma & Wan Tang & Changyong Feng & Xin M. Tu, 2008. "Inference for Kappas for Longitudinal Study Data: Applications to Sexual Health Research," Biometrics, The International Biometric Society, vol. 64(3), pages 781-789, September.
    3. Stuart Lipsitz & Michael Parzen & Garrett Fitzmaurice & Neil Klar, 2003. "A two-stage logistic regression model for analyzing inter-rater agreement," Psychometrika, Springer;The Psychometric Society, vol. 68(2), pages 289-298, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. D. Gunzler & W. Tang & N. Lu & P. Wu & X. Tu, 2014. "A Class of Distribution-Free Models for Longitudinal Mediation Analysis," Psychometrika, Springer;The Psychometric Society, vol. 79(4), pages 543-568, October.
    2. McMahon, James M. & Pouget, Enrique R. & Tortu, Stephanie, 2006. "A guide for multilevel modeling of dyadic data with binary outcomes using SAS PROC NLMIXED," Computational Statistics & Data Analysis, Elsevier, vol. 50(12), pages 3663-3680, August.
    3. Yue, Chen & Chen, Shaojie & Sair, Haris I. & Airan, Raag & Caffo, Brian S., 2015. "Estimating a graphical intra-class correlation coefficient (GICC) using multivariate probit-linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 89(C), pages 126-133.
    4. Tounkara Fodé & Rivest Louis-Paul, 2014. "Some New Random Effect Models for Correlated Binary Responses," Dependence Modeling, De Gruyter, vol. 2(1), pages 1-15, December.
    5. Tsung-Shan Tsou & Wan-Chen Chen, 2013. "Estimation of intra-cluster correlation coefficient via the failure of Bartlett’s second identity," Computational Statistics, Springer, vol. 28(4), pages 1681-1698, August.
    6. Donald P. Green & Winston Lin & Claudia Gerber, 2018. "Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments," Evaluation Review, , vol. 42(4), pages 391-422, August.
    7. Janie McDonald & Patrick D. Gerard & Christopher S. McMahan & William R. Schucany, 2016. "Exact-Permutation-Based Sign Tests for Clustered Binary Data Via Weighted and Unweighted Test Statistics," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 21(4), pages 698-712, December.
    8. Mulawa, Marta & Yamanis, Thespina J. & Hill, Lauren M. & Balvanz, Peter & Kajula, Lusajo J. & Maman, Suzanne, 2016. "Evidence of social network influence on multiple HIV risk behaviors and normative beliefs among young Tanzanian men," Social Science & Medicine, Elsevier, vol. 153(C), pages 35-43.
    9. Guangyong Zou & Allan Donner, 2004. "Confidence Interval Estimation of the Intraclass Correlation Coefficient for Binary Outcome Data," Biometrics, The International Biometric Society, vol. 60(3), pages 807-811, September.
    10. Sandra M. Eldridge & Obioha C. Ukoumunne & John B. Carlin, 2009. "The Intra‐Cluster Correlation Coefficient in Cluster Randomized Trials: A Review of Definitions," International Statistical Review, International Statistical Institute, vol. 77(3), pages 378-394, December.
    11. Zhen Pang & Anthony Y. C. Kuk, 2007. "Test of Marginal Compatibility and Smoothing Methods for Exchangeable Binary Data with Unequal Cluster Sizes," Biometrics, The International Biometric Society, vol. 63(1), pages 218-227, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:ijbist:v:9:y:2013:i:1:p:14:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.