IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v53y2024i2s0048733323002238.html
   My bibliography  Save this article

External review letters in academic promotion and tenure decisions are reflective of reviewer characteristics

Author

Listed:
  • Madera, Juan M.
  • Spitzmueller, Christiane
  • Yu, Heyao
  • Edema-Sillo, Ebenezer
  • Clarke, Mark S.F.

Abstract

We examine validity and bias in external review letters (ERLs) in academic settings. ERLs play a critical role in promotion and tenure (P&T) decisions across the globe, ending careers in some cases while allowing other scientists' careers to flourish. We coded and analyzed 995 ERLs submitted by letter writers at various institutions as part of the P&T portfolios of 195 candidates at an R1 university. We examined their relationship with P&T committees' percent of positive votes by department, college, and university committees. We investigated how ERL linguistic features, letter writer characteristics, and candidate characteristics (productivity and gender) relate to P&T decision-making. Results show writer characteristics are more strongly related than candidate characteristics to ERL linguistic features associated with P&T decisions at the department and university levels. We develop recommendations for policymakers, including changes in the use of ERLs for the P&T process.

Suggested Citation

  • Madera, Juan M. & Spitzmueller, Christiane & Yu, Heyao & Edema-Sillo, Ebenezer & Clarke, Mark S.F., 2024. "External review letters in academic promotion and tenure decisions are reflective of reviewer characteristics," Research Policy, Elsevier, vol. 53(2).
    • Handle: RePEc:eee:respol:v:53:y:2024:i:2:s0048733323002238
    DOI: 10.1016/j.respol.2023.104939
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733323002238
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2023.104939?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall & Kayvan Kousha, 2021. "Researchers’ attitudes towards the h-index on Twitter 2007–2020: criticism and acceptance," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5361-5368, June.
    2. Isabel Fernandez-Mateo & Roberto M. Fernandez, 2016. "Bending the Pipeline? Executive Search and Gender Inequality in Hiring for Top Management Jobs," Management Science, INFORMS, vol. 62(12), pages 3636-3655, December.
    3. J. E. Hirsch, 2010. "An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 741-754, December.
    4. Buunk, Abraham P. & Gibbons, Frederick X., 2007. "Social comparison: The end of a theory and the emergence of a field," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 3-21, January.
    5. Judge, Timothy A. & Higgins, Chad A., 1998. "Affective Disposition and the Letter of Reference," Organizational Behavior and Human Decision Processes, Elsevier, vol. 75(3), pages 207-221, September.
    6. Wang, Feng & Liu, Xuefeng & Fang, Eric (Er), 2015. "User Reviews Variance, Critic Reviews Variance, and Product Sales: An Exploration of Customer Breadth and Depth Effects," Journal of Retailing, Elsevier, vol. 91(3), pages 372-389.
    7. Greenberg, Jerald & Ashton-James, Claire E. & Ashkanasy, Neal M., 2007. "Social comparison processes in organizations," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 22-41, January.
    8. Stephen J. Sauer & Melissa C. Thomas-Hunt & Patrick A. Morris, 2010. "Too Good to Be True? The Unintended Signaling Effects of Educational Prestige on External Expectations of Team Performance," Organization Science, INFORMS, vol. 21(5), pages 1108-1120, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matt Grimes, 2010. "Strategic Sensemaking within Funding Relationships: The Effects of Performance Measurement on Organizational Identity in the Social Sector," Entrepreneurship Theory and Practice, , vol. 34(4), pages 763-783, July.
    2. Miaomiao Li & Xiaofeng Xu & Ho Kwong Kwan, 2023. "The antecedents and consequences of workplace envy: A meta-analytic review," Asia Pacific Journal of Management, Springer, vol. 40(1), pages 1-35, March.
    3. Ken Cheng & Limin Guo & Jinlian Luo, 2023. "The more you exploit, the more expedient I will be: A moral disengagement and Chinese traditionality examination of exploitative leadership and employee expediency," Asia Pacific Journal of Management, Springer, vol. 40(1), pages 151-167, March.
    4. Breidenthal, Amy P. & Liu, Dong & Bai, Yuntao & Mao, Yina, 2020. "The dark side of creativity: Coworker envy and ostracism as a response to employee creativity," Organizational Behavior and Human Decision Processes, Elsevier, vol. 161(C), pages 242-254.
    5. Goodman, Paul S. & Haisley, Emily, 2007. "Social comparison processes in an organizational context: New directions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(1), pages 109-125, January.
    6. Hernández-Ortega, Blanca, 2020. "When the performance comes into play: The influence of positive online consumer reviews on individuals' post-consumption responses," Journal of Business Research, Elsevier, vol. 113(C), pages 422-435.
    7. Matthew J. Quade & Rebecca L. Greenbaum & Mary B. Mawritz, 2019. "“If Only My Coworker Was More Ethical”: When Ethical and Performance Comparisons Lead to Negative Emotions, Social Undermining, and Ostracism," Journal of Business Ethics, Springer, vol. 159(2), pages 567-586, October.
    8. Zi Wang & Guiquan Li, 2018. "You don’t actually want to get closer to the star: How LMX leads to workplace ostracism," Frontiers of Business Research in China, Springer, vol. 12(1), pages 1-13, December.
    9. Natalie J. Shin & Jonathan C. Ziegert & Miriam Muethel, 2022. "The Detrimental Effects of Ethical Incongruence in Teams: An Interactionist Perspective of Ethical Fit on Relationship Conflict and Information Sharing," Journal of Business Ethics, Springer, vol. 179(1), pages 259-272, August.
    10. Antonio Fernandez-Cano & Inés M. Fernández-Guerrero, 2017. "A multivariate model for evaluating emergency medicine journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 991-1003, February.
    11. Yingjin Song & Ruiyi Li & Guanyi Chen & Beibei Yan & Lei Zhong & Yuxin Wang & Yihang Li & Jinlei Li & Yingxiu Zhang, 2021. "Bibliometric Analysis of Current Status on Bioremediation of Petroleum Contaminated Soils during 2000–2019," IJERPH, MDPI, vol. 18(16), pages 1-20, August.
    12. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    13. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    14. Ash Mohammad Abbas, 2011. "Weighted indices for evaluating the quality of research with multiple authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 107-131, July.
    15. Jiang Wu, 2013. "Geographical knowledge diffusion and spatial diversity citation rank," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 181-201, January.
    16. Christian Maier & Sven Laumer & Tim Weitzel, 2022. "A Dark Side of Telework: A Social Comparison-Based Study from the Perspective of Office Workers," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(6), pages 793-811, December.
    17. Goerke, Laszlo & Pannenberg, Markus, 2015. "Direct evidence for income comparisons and subjective well-being across reference groups," Economics Letters, Elsevier, vol. 137(C), pages 95-101.
    18. Liu, Jiawei & Niederdeppe, Jeff, 2024. "Effects of communicating health disparities using social comparison framing: A comprehensive review," Social Science & Medicine, Elsevier, vol. 348(C).
    19. Claudine Gartenberg & Julie Wulf, 2017. "Pay Harmony? Social Comparison and Performance Compensation in Multibusiness Firms," Organization Science, INFORMS, vol. 28(1), pages 39-55, February.
    20. Minnema, Alec & Bijmolt, Tammo H.A. & Gensler, Sonja & Wiesel, Thorsten, 2016. "To Keep or Not to Keep: Effects of Online Customer Reviews on Product Returns," Journal of Retailing, Elsevier, vol. 92(3), pages 253-267.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:53:y:2024:i:2:s0048733323002238. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.