IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0141468.html
   My bibliography  Save this article

The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance

Author

Listed:
  • Sven Kepes
  • Michael A McDaniel

Abstract

Introduction: Sensitivity analyses refer to investigations of the degree to which the results of a meta-analysis remain stable when conditions of the data or the analysis change. To the extent that results remain stable, one can refer to them as robust. Sensitivity analyses are rarely conducted in the organizational science literature. Despite conscientiousness being a valued predictor in employment selection, sensitivity analyses have not been conducted with respect to meta-analytic estimates of the correlation (i.e., validity) between conscientiousness and job performance. Methods: To address this deficiency, we reanalyzed the largest collection of conscientiousness validity data in the personnel selection literature and conducted a variety of sensitivity analyses. Results: Publication bias analyses demonstrated that the validity of conscientiousness is moderately overestimated (by around 30%; a correlation difference of about .06). The misestimation of the validity appears to be due primarily to suppression of small effects sizes in the journal literature. These inflated validity estimates result in an overestimate of the dollar utility of personnel selection by millions of dollars and should be of considerable concern for organizations. Conclusion: The fields of management and applied psychology seldom conduct sensitivity analyses. Through the use of sensitivity analyses, this paper documents that the existing literature overestimates the validity of conscientiousness in the prediction of job performance. Our data show that effect sizes from journal articles are largely responsible for this overestimation.

Suggested Citation

  • Sven Kepes & Michael A McDaniel, 2015. "The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-22, October.
  • Handle: RePEc:plo:pone00:0141468
    DOI: 10.1371/journal.pone.0141468
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141468
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0141468&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0141468?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. O'Boyle, Ernest H. & Rutherford, Matthew W. & Banks, George C., 2014. "Publication bias in entrepreneurship research: An examination of dominant relations to performance," Journal of Business Venturing, Elsevier, vol. 29(6), pages 773-784.
    2. Emily S Sena & H Bart van der Worp & Philip M W Bath & David W Howells & Malcolm R Macleod, 2010. "Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy," PLOS Biology, Public Library of Science, vol. 8(3), pages 1-8, March.
    3. Tom M. Palmer & Jaime L. Peters & Alex J. Sutton & Santiago G. Moreno, 2008. "Contour-enhanced funnel plots for meta-analysis," Stata Journal, StataCorp LP, vol. 8(2), pages 242-254, June.
    4. Michal Kicinski, 2013. "Publication Bias in Recent Meta-Analyses," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    5. Kepes, Sven & McDaniel, Michael A., 2013. "How Trustworthy Is the Scientific Literature in Industrial and Organizational Psychology?," Industrial and Organizational Psychology, Cambridge University Press, vol. 6(3), pages 252-268, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Serban, Andra & Kepes, Sven & Wang, Wenhao & Baldwin, Robert, 2023. "Cognitive ability and creativity: Typology contributions and a meta-analytic review," Intelligence, Elsevier, vol. 98(C).
    2. Sala, Giovanni & Aksayli, N. Deniz & Tatlidil, K. Semir & Gondo, Yasuyuki & Gobet, Fernand, 2019. "Working memory training does not enhance older adults' cognitive skills: A comprehensive meta-analysis," Intelligence, Elsevier, vol. 77(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael J. Crowther & Dean Langan & Alex J. Sutton, 2012. "Graphical augmentations to the funnel plot to assess the impact of a new study on an existing meta-analysis," Stata Journal, StataCorp LP, vol. 12(4), pages 605-622, December.
    2. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    3. Keim, Jan & Müller, Susan & Dey, Pascal, 2024. "Whatever the problem, entrepreneurship is the solution! Confronting the panacea myth of entrepreneurship with structural injustice," Journal of Business Venturing Insights, Elsevier, vol. 21(C).
    4. Jasper Brinkerink, 2023. "When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research," Entrepreneurship Theory and Practice, , vol. 47(2), pages 304-343, March.
    5. Ruxiu Tie & Tiansong Zhang & Huarui Fu & Limengmeng Wang & Yebo Wang & Ying He & Binsheng Wang & Ni Zhu & Shan Fu & Xiaoyu Lai & Jimin Shi & He Huang, 2014. "Association between DNMT3A Mutations and Prognosis of Adults with De Novo Acute Myeloid Leukemia: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    6. Tiago Severo Garcia & Tatiana Helena Rech & Cristiane Bauermann Leitão, 2017. "Pancreatic size and fat content in diabetes: A systematic review and meta-analysis of imaging studies," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-15, July.
    7. Shaylea Badovinac & Jodi Martin & Camille Guérin-Marion & Monica O’Neill & Rebecca Pillai Riddell & Jean-François Bureau & Rebecca Spiegel, 2018. "Associations between mother-preschooler attachment and maternal depression symptoms: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-27, October.
    8. Lisa Bero, 2018. "Meta-research matters: Meta-spin cycles, the blindness of bias, and rebuilding trust," PLOS Biology, Public Library of Science, vol. 16(4), pages 1-4, April.
    9. Chun-Yu Chang & Po-Chen Lin & Yung-Jiun Chien & Chien-Sheng Chen & Meng-Yu Wu, 2020. "Analysis of Chest-Compression Depth and Full Recoil in Two Infant Chest-Compression Techniques Performed by a Single Rescuer: Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 17(11), pages 1-17, June.
    10. Haris S Vasiliadis & Adriani Nikolakopoulou & Ian Shrier & Michael P Lunn & Ruth Brassington & Rob J P Scholten & Georgia Salanti, 2015. "Endoscopic and Open Release Similarly Safe for the Treatment of Carpal Tunnel Syndrome. A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-16, December.
    11. Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    12. Joanna Małecka & Teresa Łuczka & Jarmila Šebestová & Roman Šperka, 2017. "Economic Activity And Social Determinants Versus Etrepreneurship In Smes – Selected Aspects," Copernican Journal of Finance & Accounting, Uniwersytet Mikolaja Kopernika, vol. 6(3), pages 47-61.
    13. Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
    14. Cerdá, Emilio & López-Otero, Xiral & Quiroga, Sonia & Soliño, Mario, 2024. "Willingness to pay for renewables: Insights from a meta-analysis of choice experiments," Energy Economics, Elsevier, vol. 130(C).
    15. Aliasghar Ahmad Kiadaliri & Johan Jarl & Georgios Gavriilidis & Ulf-G Gerdtham, 2013. "Alcohol Drinking Cessation and the Risk of Laryngeal and Pharyngeal Cancers: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-10, March.
    16. Jenny T van der Steen & Cornelis A van den Bogert & Mirjam C van Soest-Poortvliet & Soulmaz Fazeli Farsani & René H J Otten & Gerben ter Riet & Lex M Bouter, 2018. "Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-15, February.
    17. Brian P Walcott & Sameer A Sheth & Brian V Nahed & Jean-Valery Coumans, 2012. "Conflict of Interest in Spine Research Reporting," PLOS ONE, Public Library of Science, vol. 7(8), pages 1-4, August.
    18. Christian Harlos & Tim C. Edgell & Johan Hollander, 2017. "No evidence of publication bias in climate change science," Climatic Change, Springer, vol. 140(3), pages 375-385, February.
    19. Labandeira, Xavier & Labeaga, José M. & Linares, Pedro & López-Otero, Xiral, 2020. "The impacts of energy efficiency policies: Meta-analysis," Energy Policy, Elsevier, vol. 147(C).
    20. Roger M. Harbord & Ross J. Harris & Jonathan A. C. Sterne, 2009. "Updated tests for small-study effects in meta-analyses," Stata Journal, StataCorp LP, vol. 9(2), pages 197-210, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0141468. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.