Medical students' confidence judgments using a factual database and personal memory: A comparison

In order to determine whether medical students can recognize when an information need has been fulfilled and when it has not, this study examined the quality of medical students' confidence estimates in answering short‐answer questions dealing with bacteriology, based upon their personal knowledge alone and what they were able to retrieve from a factual database in microbiology. Twelve students, assessed over three occasions, remained in the final sample. The results indicate that students displayed a positive relationship between their expertise in answering the questions and the amount of overconfidence they indicated (the opposite of the hard–easy effect) for the personal knowledge task using a partial credit format. For the database‐assisted task using the partial credit format, students showed less overconfidence in their answers with greater expertise in using the database. For both the personal knowledge and database‐assisted tasks using a binary format (all or nothing correct), the students displayed the opposite of the hard–easy effect. We conclude that the patterns of confidence estimates (in the form of Brier scores), and thus students' ability to recognize whether their information need has been fulfilled, differ with varying degrees of expertise in both the personal knowledge responses and the database‐assisted responses for both the partial credit and binary formats. Taking into consideration the fact that when subjects, in this case future medical practitioners, are extremely overconfident, they stop looking for information long before they have found material that is relevant, the results have broad implications for medical practice and information seeking.