IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v37y2017i6p715-724.html
   My bibliography  Save this article

The Potential of Collective Intelligence in Emergency Medicine: Pooling Medical Students’ Independent Decisions Improves Diagnostic Performance

Author

Listed:
  • Juliane E. Kämmer
  • Wolf E. Hautz
  • Stefan M. Herzog
  • Olga Kunina-Habenicht
  • Ralf H. J. M. Kurvers

Abstract

Background. Evidence suggests that pooling multiple independent diagnoses can improve diagnostic accuracy in well-defined tasks. We investigated whether this is also the case for diagnostics in emergency medicine, an ill-defined task environment where diagnostic errors are rife. Methods. A computer simulation study was conducted based on empirical data from 2 published experimental studies. In the computer experiments, 285 medical students independently diagnosed 6 simulated patients arriving at the emergency room with dyspnea. Participants’ diagnoses (n = 1,710), confidence ratings, and expertise levels were entered into a computer simulation. Virtual groups of different sizes were randomly created, and 3 collective intelligence rules (follow-the-plurality rule, follow-the-most-confident rule, and follow-the-most-senior rule) were applied to combine the independent decisions into a final diagnosis. For different group sizes, the performance levels (i.e., percentage of correct diagnoses) of the 3 collective intelligence rules were compared with each other and against the average individual accuracy. Results. For all collective intelligence rules, combining independent decisions substantially increased performance relative to average individual performance. For groups of 4 or fewer, the follow-the-most-confident rule outperformed the other rules; for larger groups, the follow-the-plurality rule performed best. For example, combining 5 independent decisions using the follow-the-plurality rule increased diagnostic accuracy by 22 percentage points. These results were robust across case difficulty and expertise level. Limitations of the study include the use of simulated patients diagnosed by medical students. Whether results generalize to clinical practice is currently unknown. Conclusion. Combining independent decisions may substantially improve the quality of diagnoses in emergency medicine and may thus enhance patient safety.

Suggested Citation

  • Juliane E. Kämmer & Wolf E. Hautz & Stefan M. Herzog & Olga Kunina-Habenicht & Ralf H. J. M. Kurvers, 2017. "The Potential of Collective Intelligence in Emergency Medicine: Pooling Medical Students’ Independent Decisions Improves Diagnostic Performance," Medical Decision Making, , vol. 37(6), pages 715-724, August.
    • Handle: RePEc:sae:medema:v:37:y:2017:i:6:p:715-724
    DOI: 10.1177/0272989X17696998
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X17696998
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X17696998?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert T. Clemen & Robert L. Winkler, 1985. "Limits for the Precision and Value of Information from Dependent Sources," Operations Research, INFORMS, vol. 33(2), pages 427-442, April.
    2. Robert L. Winkler & Roy M. Poses, 1993. "Evaluating and Combining Physicians' Probabilities of Survival in an Intensive Care Unit," Management Science, INFORMS, vol. 39(12), pages 1526-1543, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elaine C. Khoong & Sarah S. Nouri & Delphine S. Tuot & Shantanu Nundy & Valy Fontil & Urmimala Sarkar, 2022. "Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation," Medical Decision Making, , vol. 42(3), pages 293-302, April.
    2. Matthew D. Blanchard & Stefan M. Herzog & Juliane E. Kämmer & Nikolas Zöller & Olga Kostopoulou & Ralf H. J. M. Kurvers, 2024. "Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting," Medical Decision Making, , vol. 44(4), pages 451-462, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hurley, W. J. & Lior, D. U., 2002. "Combining expert judgment: On the performance of trimmed mean vote aggregation procedures in the presence of strategic voting," European Journal of Operational Research, Elsevier, vol. 140(1), pages 142-147, July.
    2. James K. Hammitt & Alexander I. Shlyakhter, 1999. "The Expected Value of Information and the Probability of Surprise," Risk Analysis, John Wiley & Sons, vol. 19(1), pages 135-152, February.
    3. James K. Hammitt & Yifan Zhang, 2013. "Combining Experts’ Judgments: Comparison of Algorithmic Methods Using Synthetic Data," Risk Analysis, John Wiley & Sons, vol. 33(1), pages 109-120, January.
    4. Joseph Lipscomb & Giovanni Parmigiani & Vic Hasselblad, 1998. "Combining Expert Judgment by Hierarchical Modeling: An Application to Physician Staffing," Management Science, INFORMS, vol. 44(2), pages 149-161, February.
    5. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    6. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    7. Fifić, Mario & Gigerenzer, Gerd, 2014. "Are two interviewers better than one?," Journal of Business Research, Elsevier, vol. 67(8), pages 1771-1779.
    8. Jaspersen, Johannes G., 2022. "Convex combinations in judgment aggregation," European Journal of Operational Research, Elsevier, vol. 299(2), pages 780-794.
    9. Ozer, Muammer, 2011. "Understanding the impacts of product knowledge and product type on the accuracy of intentions-based new product predictions," European Journal of Operational Research, Elsevier, vol. 211(2), pages 359-369, June.
    10. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    11. Mark Freeman & Ben Groom, 2015. "Using equity premium survey data to estimate future wealth," Review of Quantitative Finance and Accounting, Springer, vol. 45(4), pages 665-693, November.
    12. Ozer, Muammer, 2007. "Reducing the demand uncertainties at the fuzzy-front-end of developing new online services," Research Policy, Elsevier, vol. 36(9), pages 1372-1387, November.
    13. Gary J. Summers, 2021. "Friction and Decision Rules in Portfolio Decision Analysis," Decision Analysis, INFORMS, vol. 18(2), pages 101-120, June.
    14. Paola Monari & Patrizia Agati, 2001. "Fiducial inference in combining expert judgements," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 10(1), pages 81-97, January.
    15. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    16. Bian, Wenliang & Shang, Jennifer & Zhang, Juliang, 2016. "Two-way information sharing under supply chain competition," International Journal of Production Economics, Elsevier, vol. 178(C), pages 82-94.
    17. Elena Verdolini & Laura Díaz Anadón & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2018. "Future Prospects for Energy Technologies: Insights from Expert Elicitations," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 133-153.
    18. Fumie Yokota & Kimberly M. Thompson, 2004. "Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future," Risk Analysis, John Wiley & Sons, vol. 24(3), pages 635-650, June.
    19. Stewart, Thomas R. & Roebber, Paul J. & Bosart, Lance F., 1997. "The Importance of the Task in Analyzing Expert Judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 205-219, March.
    20. Vincenz Frey & Arnout van de Rijt, 2021. "Social Influence Undermines the Wisdom of the Crowd in Sequential Decision Making," Management Science, INFORMS, vol. 67(7), pages 4273-4286, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:37:y:2017:i:6:p:715-724. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.