IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0083528.html
   My bibliography  Save this article

A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making

Author

Listed:
  • Esther Kaufmann
  • Ulf-Dietrich Reips
  • Werner W Wittmann

Abstract

Achieving accurate judgment (‘judgmental achievement’) is of utmost importance in daily life across multiple domains. The lens model and the lens model equation provide useful frameworks for modeling components of judgmental achievement and for creating tools to help decision makers (e.g., physicians, teachers) reach better judgments (e.g., a correct diagnosis, an accurate estimation of intelligence). Previous meta-analyses of judgment and decision-making studies have attempted to evaluate overall judgmental achievement and have provided the basis for evaluating the success of bootstrapping (i.e., replacing judges by linear models that guide decision making). However, previous meta-analyses have failed to appropriately correct for a number of study design artifacts (e.g., measurement error, dichotomization), which may have potentially biased estimations (e.g., of the variability between studies) and led to erroneous interpretations (e.g., with regards to moderator variables). In the current study we therefore conduct the first psychometric meta-analysis of judgmental achievement studies that corrects for a number of study design artifacts. We identified 31 lens model studies (N = 1,151, k = 49) that met our inclusion criteria. We evaluated overall judgmental achievement as well as whether judgmental achievement depended on decision domain (e.g., medicine, education) and/or the level of expertise (expert vs. novice). We also evaluated whether using corrected estimates affected conclusions with regards to the success of bootstrapping with psychometrically-corrected models. Further, we introduce a new psychometric trim-and-fill method to estimate the effect sizes of potentially missing studies correct psychometric meta-analyses for effects of publication bias. Comparison of the results of the psychometric meta-analysis with the results of a traditional meta-analysis (which only corrected for sampling error) indicated that artifact correction leads to a) an increase in values of the lens model components, b) reduced heterogeneity between studies, and c) increases the success of bootstrapping. We argue that psychometric meta-analysis is useful for accurately evaluating human judgment and show the success of bootstrapping.

Suggested Citation

  • Esther Kaufmann & Ulf-Dietrich Reips & Werner W Wittmann, 2013. "A Critical Meta-Analysis of Lens Model Studies in Human Judgment and Decision-Making," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-16, December.
  • Handle: RePEc:plo:pone00:0083528
    DOI: 10.1371/journal.pone.0083528
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0083528
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0083528&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0083528?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    2. repec:cup:judgdm:v:6:y:2011:i:8:p:870-881 is not listed on IDEAS
    3. Harvey, Nigel & Harries, Clare, 2004. "Effects of judges' forecasting on their later combination of forecasts for the same outcomes," International Journal of Forecasting, Elsevier, vol. 20(3), pages 391-409.
    4. Mear, Ross & Firth, Michael, 1987. "Assessing the accuracy of financial analyst security return predictions," Accounting, Organizations and Society, Elsevier, vol. 12(4), pages 331-340, June.
    5. Stewart, Thomas R. & Roebber, Paul J. & Bosart, Lance F., 1997. "The Importance of the Task in Analyzing Expert Judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 205-219, March.
    6. Ashton, Ah, 1982. "An Empirical-Study Of Budget-Related Predictions Of Corporate-Executives," Journal of Accounting Research, Wiley Blackwell, vol. 20(2), pages 440-449.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Esther Kaufmann & Werner W Wittmann, 2016. "The Success of Linear Bootstrapping Models: Decision Domain-, Expertise-, and Criterion-Specific Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-21, June.
    2. Kausel, Edgar E. & Culbertson, Satoris S. & Madrid, Hector P., 2016. "Overconfidence in personnel selection: When and why unstructured interview information can hurt hiring decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 137(C), pages 27-44.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Esther Kaufmann & Werner W Wittmann, 2016. "The Success of Linear Bootstrapping Models: Decision Domain-, Expertise-, and Criterion-Specific Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-21, June.
    2. Anqiang Huang & Han Qiao & Shouyang Wang & John Liu, 2016. "Improving Forecasting Performance by Exploiting Expert Knowledge: Evidence from Guangzhou Port," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(02), pages 387-401, March.
    3. Anqiang Huang & Kin Keung Lai & Han Qiao & Shouyang Wang & Zhenji Zhang, 2018. "Does Interval Knowledge Sharpen Forecasting Models? Evidence from China’s Typical Ports," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(02), pages 467-483, March.
    4. Qian Li & Yan Chen & Shikun Sun & Muyuan Zhu & Jing Xue & Zihan Gao & Jinfeng Zhao & Yihe Tang, 2022. "Research on Crop Irrigation Schedules Under Deficit Irrigation—A Meta-analysis," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(12), pages 4799-4817, September.
    5. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    6. Damiano Pizzol & Mike Trott & Igor Grabovac & Mario Antunes & Anna Claudia Colangelo & Simona Ippoliti & Cristian Petre Ilie & Anne Carrie & Nicola Veronese & Lee Smith, 2021. "Laparoscopy in Low-Income Countries: 10-Year Experience and Systematic Literature Review," IJERPH, MDPI, vol. 18(11), pages 1-11, May.
    7. Wolfgang Goymann & John C. Wingfield, 2014. "Male-to-female testosterone ratios, dimorphism, and life history—what does it really tell us?," Behavioral Ecology, International Society for Behavioral Ecology, vol. 25(4), pages 685-699.
    8. Gehrig, Thomas & Güth, Werner & Leví0nský, René & Popova, Vera, 2010. "On the evolution of professional consulting," Journal of Economic Behavior & Organization, Elsevier, vol. 76(1), pages 113-126, October.
    9. Alderotti, Giammarco & Rapallini, Chiara & Traverso, Silvio, 2023. "The Big Five personality traits and earnings: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 94(C).
    10. Ünal, Zehra E. & Kartal, Gamze & Ulusoy, Serra & Ala, Aslı M. & Yilmaz, Munube & Geary, David C., 2023. "Relative contributions of g and basic domain-specific mathematics skills to complex mathematics competencies," Intelligence, Elsevier, vol. 101(C).
    11. Daniele Zago & Maria Eugênia Andrighetto Canozzi & Júlio Otávio Jardim Barcellos, 2020. "Pregnant beef cow’s nutrition and its effects on postnatal weight and carcass quality of their progeny," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-20, August.
    12. Viktoria Maria Baumeister & Leonie Petra Kuen & Maike Bruckes & Gerhard Schewe, 2021. "The Relationship of Work-Related ICT Use With Well-being, Incorporating the Role of Resources and Demands: A Meta-Analysis," SAGE Open, , vol. 11(4), pages 21582440211, November.
    13. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    14. Angélica Pigola & Priscila Rezende Costa, 2022. "In search of understanding about knowledge and learning on innovation performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3995-4022, July.
    15. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    16. Chuang Yuan & Jing Wang & Michael Ying, 2016. "Predictive Value of Carotid Distensibility Coefficient for Cardiovascular Diseases and All-Cause Mortality: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-15, April.
    17. Francisco Javier Blanco-Encomienda & Rocío García-Cantero & María José Latorre-Medina, 2020. "Association between Work-Related Rumination, Work Environment and Employee Well-Being: A Meta-Analytic Study of Main and Moderator Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(3), pages 887-910, August.
    18. Andrew J. Rosman, 2011. "Auditors' going‐concern judgments: rigid, adaptive, or both?," Review of Accounting and Finance, Emerald Group Publishing Limited, vol. 10(1), pages 30-45, February.
    19. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    20. Pedro L. Cosio & Manuel Crespo-Posadas & Álvaro Velarde-Sotres & Mireia Pelaez, 2021. "Effect of Chronic Resistance Training on Circulating Irisin: Systematic Review and Meta-Analysis of Randomized Controlled Trials," IJERPH, MDPI, vol. 18(5), pages 1-16, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0083528. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.