Doctors and Numbers

Background . Risk interpretation affects decision making. Yet, there is no valid assessment of how clinicians interpret the risk data that they commonly encounter. Objective . To establish the reliability and validity of a 20-item test of clinicians’ risk interpretation. Methods . The Critical Risk Interpretation Test (CRIT) measures clinicians’ abilities to 1) modify the interpretation based on meaningful differences in the outcome (e.g., disease specific v. all-cause mortality) and time period (e.g., lifetime v. 10-year mortality), 2) maintain a stable interpretation for different risk framings (e.g., relative v. absolute risk), and 3) correctly interpret how diagnostic testing modifies risk. There were 658 clinicians and medical trainees who participated: 116 nurse practitioners (NPs) at a national conference, 273 medical students at 1 institution, 148 residents in internal medicine at 2 institutions, and 121 internists at 1 institution. Participants completed a self-administered paper test during educational conferences. Seventeen evidence-based medicine experts took the test online and formally assessed content validity. Eighteen second-year medical students were recruited to take the test and a retest 3 weeks later to explore test-retest correlation. Results . Expert review supported test clarity and content validity. Factor analysis supported that the CRIT identifies at least 3 separable areas of clinician knowledge. Test-retest correlation was fair (intraclass correlation coefficient = 0.65; standard error = 0.15). Scores on our test correlated with other tests of related abilities. Mean test scores varied among groups, with differences in prior evidence-based medicine training and experience (93 for NPs, 101 for medical students, 101 for residents, 103 for academic internists, and 110 for physician experts; P