IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0083528.html
   My bibliography  Save this article

A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making

Author

Listed:
  • Esther Kaufmann
  • Ulf-Dietrich Reips
  • Werner W Wittmann

Abstract

Achieving accurate judgment (‘judgmental achievement’) is of utmost importance in daily life across multiple domains. The lens model and the lens model equation provide useful frameworks for modeling components of judgmental achievement and for creating tools to help decision makers (e.g., physicians, teachers) reach better judgments (e.g., a correct diagnosis, an accurate estimation of intelligence). Previous meta-analyses of judgment and decision-making studies have attempted to evaluate overall judgmental achievement and have provided the basis for evaluating the success of bootstrapping (i.e., replacing judges by linear models that guide decision making). However, previous meta-analyses have failed to appropriately correct for a number of study design artifacts (e.g., measurement error, dichotomization), which may have potentially biased estimations (e.g., of the variability between studies) and led to erroneous interpretations (e.g., with regards to moderator variables). In the current study we therefore conduct the first psychometric meta-analysis of judgmental achievement studies that corrects for a number of study design artifacts. We identified 31 lens model studies (N = 1,151, k = 49) that met our inclusion criteria. We evaluated overall judgmental achievement as well as whether judgmental achievement depended on decision domain (e.g., medicine, education) and/or the level of expertise (expert vs. novice). We also evaluated whether using corrected estimates affected conclusions with regards to the success of bootstrapping with psychometrically-corrected models. Further, we introduce a new psychometric trim-and-fill method to estimate the effect sizes of potentially missing studies correct psychometric meta-analyses for effects of publication bias. Comparison of the results of the psychometric meta-analysis with the results of a traditional meta-analysis (which only corrected for sampling error) indicated that artifact correction leads to a) an increase in values of the lens model components, b) reduced heterogeneity between studies, and c) increases the success of bootstrapping. We argue that psychometric meta-analysis is useful for accurately evaluating human judgment and show the success of bootstrapping.

Suggested Citation

  • Esther Kaufmann & Ulf-Dietrich Reips & Werner W Wittmann, 2013. "A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-16, December.
  • Handle: RePEc:plo:pone00:0083528
    DOI: 10.1371/journal.pone.0083528
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0083528
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0083528&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0083528?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    2. repec:cup:judgdm:v:6:y:2011:i:8:p:870-881 is not listed on IDEAS
    3. Harvey, Nigel & Harries, Clare, 2004. "Effects of judges' forecasting on their later combination of forecasts for the same outcomes," International Journal of Forecasting, Elsevier, vol. 20(3), pages 391-409.
    4. Mear, Ross & Firth, Michael, 1987. "Assessing the accuracy of financial analyst security return predictions," Accounting, Organizations and Society, Elsevier, vol. 12(4), pages 331-340, June.
    5. Stewart, Thomas R. & Roebber, Paul J. & Bosart, Lance F., 1997. "The Importance of the Task in Analyzing Expert Judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 205-219, March.
    6. Ashton, Ah, 1982. "An Empirical-Study Of Budget-Related Predictions Of Corporate-Executives," Journal of Accounting Research, Wiley Blackwell, vol. 20(2), pages 440-449.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Esther Kaufmann & Werner W Wittmann, 2016. "The Success of Linear Bootstrapping Models: Decision Domain-, Expertise-, and Criterion-Specific Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-21, June.
    2. Kausel, Edgar E. & Culbertson, Satoris S. & Madrid, Hector P., 2016. "Overconfidence in personnel selection: When and why unstructured interview information can hurt hiring decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 137(C), pages 27-44.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Esther Kaufmann & Werner W Wittmann, 2016. "The Success of Linear Bootstrapping Models: Decision Domain-, Expertise-, and Criterion-Specific Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-21, June.
    2. Anqiang Huang & Han Qiao & Shouyang Wang & John Liu, 2016. "Improving Forecasting Performance by Exploiting Expert Knowledge: Evidence from Guangzhou Port," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(02), pages 387-401, March.
    3. Anqiang Huang & Kin Keung Lai & Han Qiao & Shouyang Wang & Zhenji Zhang, 2018. "Does Interval Knowledge Sharpen Forecasting Models? Evidence from China’s Typical Ports," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(02), pages 467-483, March.
    4. Qian Li & Yan Chen & Shikun Sun & Muyuan Zhu & Jing Xue & Zihan Gao & Jinfeng Zhao & Yihe Tang, 2022. "Research on Crop Irrigation Schedules Under Deficit Irrigation—A Meta-analysis," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(12), pages 4799-4817, September.
    5. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    6. Damiano Pizzol & Mike Trott & Igor Grabovac & Mario Antunes & Anna Claudia Colangelo & Simona Ippoliti & Cristian Petre Ilie & Anne Carrie & Nicola Veronese & Lee Smith, 2021. "Laparoscopy in Low-Income Countries: 10-Year Experience and Systematic Literature Review," IJERPH, MDPI, vol. 18(11), pages 1-11, May.
    7. Gehrig, Thomas & Güth, Werner & Leví0nský, René & Popova, Vera, 2010. "On the evolution of professional consulting," Journal of Economic Behavior & Organization, Elsevier, vol. 76(1), pages 113-126, October.
    8. Ünal, Zehra E. & Kartal, Gamze & Ulusoy, Serra & Ala, Aslı M. & Yilmaz, Munube & Geary, David C., 2023. "Relative contributions of g and basic domain-specific mathematics skills to complex mathematics competencies," Intelligence, Elsevier, vol. 101(C).
    9. Viktoria Maria Baumeister & Leonie Petra Kuen & Maike Bruckes & Gerhard Schewe, 2021. "The Relationship of Work-Related ICT Use With Well-being, Incorporating the Role of Resources and Demands: A Meta-Analysis," SAGE Open, , vol. 11(4), pages 21582440211, November.
    10. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    11. Chuang Yuan & Jing Wang & Michael Ying, 2016. "Predictive Value of Carotid Distensibility Coefficient for Cardiovascular Diseases and All-Cause Mortality: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-15, April.
    12. Francisco Javier Blanco-Encomienda & Rocío García-Cantero & María José Latorre-Medina, 2020. "Association between Work-Related Rumination, Work Environment and Employee Well-Being: A Meta-Analytic Study of Main and Moderator Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(3), pages 887-910, August.
    13. Andrew J. Rosman, 2011. "Auditors' going‐concern judgments: rigid, adaptive, or both?," Review of Accounting and Finance, Emerald Group Publishing Limited, vol. 10(1), pages 30-45, February.
    14. Pedro L. Cosio & Manuel Crespo-Posadas & Álvaro Velarde-Sotres & Mireia Pelaez, 2021. "Effect of Chronic Resistance Training on Circulating Irisin: Systematic Review and Meta-Analysis of Randomized Controlled Trials," IJERPH, MDPI, vol. 18(5), pages 1-16, March.
    15. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    16. Georgiou, George K. & Guo, Kan & Naveenkumar, Nithya & Vieira, Ana Paula Alves & Das, J.P., 2020. "PASS theory of intelligence and academic achievement: A meta-analytic review," Intelligence, Elsevier, vol. 79(C).
    17. Liliana R Demenescu & Rudie Kortekaas & Johan A den Boer & André Aleman, 2010. "Impaired Attribution of Emotion to Facial Expressions in Anxiety and Major Depression," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-5, December.
    18. Lohwasser, Todor S., 2020. "Meta-analyzing the relative performance of venture capital-backed firms," Discussion Papers of the Institute for Organisational Economics 4/2020, University of Münster, Institute for Organisational Economics.
    19. Nelson, Jon Paul, 2020. "Fixed-effect versus random-effects meta-analysis in economics: A study of pass-through rates for alcohol beverage excise taxes," Economics Discussion Papers 2020-1, Kiel Institute for the World Economy (IfW Kiel).
    20. Michael Wolfowicz & Yael Litmanovitz & David Weisburd & Badi Hasisi, 2021. "Cognitive and behavioral radicalization: A systematic review of the putative risk and protective factors," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0083528. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.