IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v46y2017i5p911-924.html
   My bibliography  Save this article

Gender effects in research evaluation

Author

Listed:
  • Jappelli, Tullio
  • Nappi, Carmela Anna
  • Torrini, Roberto

Abstract

The paper contributes to the literature on gender gap in research investigating whether there is a gender gap in research evaluation. We use detailed data on 180,000 research papers evaluated during the Italian national research assessment (VQR 2004–2010) conducted by the Agency for the Evaluation of Universities and Research Institutes (Anvur). The data are merged with information on individual researchers and characteristics of referees. The most important empirical finding is that there is a significant gender gap in research evaluation. The gap is reduced once we control for researchers’ characteristics, such as age and academic rank, but is almost unaffected by the characteristics of the research output (monographs, journal articles, book chapters, etc.), co-authorships, international collaborations. Childbearing and maternity leaves do not account for the remaining gap in research evaluation. The evaluation method (peer review or bibliometric analysis) and the referee mix (whether men or women) do not disadvantage women. Analysis of a random sample of papers evaluated using bibliometric indicators and peer review reveals that bibliometric evaluation proves to be more favourable to women than peer review evaluation.

Suggested Citation

  • Jappelli, Tullio & Nappi, Carmela Anna & Torrini, Roberto, 2017. "Gender effects in research evaluation," Research Policy, Elsevier, vol. 46(5), pages 911-924.
    • Handle: RePEc:eee:respol:v:46:y:2017:i:5:p:911-924
    DOI: 10.1016/j.respol.2017.03.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733317300458
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2017.03.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Beaudry, Catherine & Larivière, Vincent, 2016. "Which gender gap? Factors affecting researchers’ scientific impact in science and medicine," Research Policy, Elsevier, vol. 45(9), pages 1790-1817.
    2. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    3. Lex Borghans & Bart H. H. Golsteyn & James J. Heckman & Huub Meijers, 2009. "Gender Differences in Risk Aversion and Ambiguity Aversion," Journal of the European Economic Association, MIT Press, vol. 7(2-3), pages 649-658, 04-05.
    4. Krapf, Matthias & Ursprung, Heinrich W. & Zimmermann, Christian, 2017. "Parenthood and productivity of highly skilled labor: Evidence from the groves of academe," Journal of Economic Behavior & Organization, Elsevier, vol. 140(C), pages 147-175.
    5. Maria De Paola & Michela Ponzo & Vincenzo Scoppa, 2015. "Gender Differences in Attitudes Towards Competition: Evidence from the Italian Scientific Qualification," CSEF Working Papers 391, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy.
    6. Vincent Larivière & Éric Archambault & Yves Gingras & Étienne Vignola‐Gagné, 2006. "The place of serials in referencing practices: Comparing natural sciences and engineering with social sciences and humanities," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(8), pages 997-1004, June.
    7. Del Boca, Daniela, 2015. "Child Care Arrangements and Labor Supply," IDB Publications (Working Papers) 6812, Inter-American Development Bank.
    8. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems," Journal of Informetrics, Elsevier, vol. 6(2), pages 155-168.
    9. Maliniak, Daniel & Powers, Ryan & Walter, Barbara F., 2013. "The Gender Citation Gap in International Relations," International Organization, Cambridge University Press, vol. 67(4), pages 889-922, October.
    10. Maria De Paola & Vincenzo Scoppa, 2015. "Gender Discrimination and Evaluators’ Gender: Evidence from Italian Academia," Economica, London School of Economics and Political Science, vol. 82(325), pages 162-188, January.
    11. Natalia Zinovyeva & Manuel F. Bagues, 2010. "Does gender matter for academic promotion? Evidence from a randomized natural experiment," Working Papers 2010-15, FEDEA.
    12. Brooks, Chris & Fenton, Evelyn M. & Walker, James T., 2014. "Gender and the evaluation of research," Research Policy, Elsevier, vol. 43(6), pages 990-1001.
    13. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    14. Geuna, Aldo & Piolatto, Matteo, 2016. "Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)," Research Policy, Elsevier, vol. 45(1), pages 260-271.
    15. Broder, Ivy E, 1993. "Review of NSF Economics Proposals: Gender and Institutional Patterns," American Economic Review, American Economic Association, vol. 83(4), pages 964-970, September.
    16. Alessio Ancaiani & Alberto F. Anfossi & Anna Barbara & Sergio Benedetto & Brigida Blasi & Valentina Carletti & Tindaro Cicero & Alberto Ciolfi & Filippo Costa & Giovanna Colizza & Marco Costantini & F, 2015. "Evaluating scientific research in Italy: The 2004–10 research evaluation exercise," Research Evaluation, Oxford University Press, vol. 24(3), pages 242-255.
    17. Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    18. Barbara Good & Niki Vermeulen & Brigitte Tiefenthaler & Erik Arnold, 2015. "Counting quality? The Czech performance-based research funding system," Research Evaluation, Oxford University Press, vol. 24(2), pages 91-105.
    19. Clemens B. Fell & Cornelius J. König, 2016. "Is there a gender difference in scientific collaboration? A scientometric examination of co-authorships among industrial–organizational psychologists," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 113-141, July.
    20. Wiji Arulampalam & Alison L. Booth & Mark L. Bryan, 2007. "Is There a Glass Ceiling over Europe? Exploring the Gender Pay Gap across the Wage Distribution," ILR Review, Cornell University, ILR School, vol. 60(2), pages 163-186, January.
    21. Jevin D West & Jennifer Jacquet & Molly M King & Shelley J Correll & Carl T Bergstrom, 2013. "The Role of Gender in Scholarly Authorship," PLOS ONE, Public Library of Science, vol. 8(7), pages 1-6, July.
    22. Donna K. Ginther & Shulamit Kahn, 2009. "Does Science Promote Women? Evidence from Academia 1973-2001," NBER Chapters, in: Science and Engineering Careers in the United States: An Analysis of Markets and Employment, pages 163-194, National Bureau of Economic Research, Inc.
    23. Frandsen, Tove Faber & Jacobsen, Rasmus Højbjerg & Wallin, Johan A. & Brixen, Kim & Ousager, Jakob, 2015. "Gender differences in scientific performance: A bibliometric matching analysis of Danish health sciences Graduates," Journal of Informetrics, Elsevier, vol. 9(4), pages 1007-1017.
    24. Lex Borghans & Bart H.H. Golsteyn & James J. Heckman & Huub Meijers, 2009. "Gender Differences in Risk Aversion and Ambiguity," Working Papers 200903, Geary Institute, University College Dublin.
    25. Daniela Del Boca, 2015. "Child Care Arrangements and Labor Supply," IDB Publications (Working Papers) 88074, Inter-American Development Bank.
    26. Manuel Bagues & Mauro Sylos-Labini & Natalia Zinovyeva, 2014. "Do gender quotas pass the test ? Evidence from academic evaluations in Italy," LEM Papers Series 2014/14, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    27. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    28. Elba Mauleón & María Bordons, 2006. "Productivity, impact and publication habits by gender in the area of Materials Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 199-218, January.
    29. John M. McDowell & Larry D. Singell & Mark Stater, 2006. "Two to Tango? Gender Differences in the Decisions to Publish and Coauthor," Economic Inquiry, Western Economic Association International, vol. 44(1), pages 153-168, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mike Thelwall, 2020. "Female citation impact superiority 1996–2018 in six out of seven English‐speaking nations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(8), pages 979-990, August.
    2. Corsi, Marcella & D’Ippoliti, Carlo & Zacchia, Giulia, 2019. "Diversity of backgrounds and ideas: The case of research evaluation in economics," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    3. Cristina Rodríguez-Faneca & Alexander Maz-Machado & David Gutiérrez-Rubio & Cristina Pedrosa-Jesús, 2022. "Presence of women on the editorial boards of the language and linguistics journals in Spain," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4237-4249, July.
    4. Thelwall, Mike & Kousha, Kayvan & Stuart, Emma & Makita, Meiko & Abdoli, Mahshid & Wilson, Paul & Levitt, Jonathan, 2023. "Do bibliometrics introduce gender, institutional or interdisciplinary biases into research evaluations?," Research Policy, Elsevier, vol. 52(8).
    5. Mingting Kou & Yi Zhang & Yu Zhang & Kaihua Chen & Jiancheng Guan & Senmao Xia, 2020. "Does gender structure influence R&D efficiency? A regional perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 477-501, January.
    6. Horbach, Serge P.J.M. & Schneider, Jesper W. & Sainte-Marie, Maxime, 2022. "Ungendered writing: Writing styles are unlikely to account for gender differences in funding rates in the natural and technical sciences," Journal of Informetrics, Elsevier, vol. 16(4).
    7. Maria De Paola & Roberto Nisticò & Vincenzo Scoppa, 2021. "Academic Careers And Fertility Decisions," Working Papers 202101, Università della Calabria, Dipartimento di Economia, Statistica e Finanza "Giovanni Anania" - DESF.
    8. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    9. Tove Faber Frandsen & Rasmus Højbjerg Jacobsen & Jakob Ousager, 2020. "Gender gaps in scientific performance: a longitudinal matching study of health sciences researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1511-1527, August.
    10. Filandri, Marianna & Pasqua, Silvia, 2019. "Gender discrimination in academic careers in Italy," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201921, University of Turin.
    11. Pär Mårtensson & Uno Fors & Emelie Fröberg & Udo Zander & Gunnar H Nilsson, 2019. "Quality of Research Practice – An interdisciplinary face validity evaluation of a quality model," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    12. Auschra, Carolin & Bartosch, Julia & Lohmeyer, Nora, 2022. "Differences in female representation in leading management and organization journals: Establishing a benchmark," Research Policy, Elsevier, vol. 51(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tullio Jappelli & Carmela Anna Nappi & Roberto Torrini, 2015. "Research Quality and Gender Gap in Research Assessment," CSEF Working Papers 418, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy.
    2. Zacchia, Giulia, 2016. "Segregation or homologation? Gender differences in recent Italian economic thought," MPRA Paper 72279, University Library of Munich, Germany.
    3. Manuel Bagues & Mauro Sylos-Labini & Natalia Zinovyeva, 2017. "Does the Gender Composition of Scientific Committees Matter?," American Economic Review, American Economic Association, vol. 107(4), pages 1207-1238, April.
    4. Tahereh Dehdarirad & Anna Villarroya & Maite Barrios, 2015. "Research on women in science and higher education: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 795-812, June.
    5. Paul Heidhues & Botond KH{o}szegi & Philipp Strack, 2019. "Overconfidence and Prejudice," Papers 1909.08497, arXiv.org.
    6. D. Checchi & S. Cicognani & N. Kulic, 2015. "Gender quotas or girls networks? Towards an understanding of recruitment in the research profession in Italy," Working Papers wp1047, Dipartimento Scienze Economiche, Universita' di Bologna.
    7. Marek Kwiek & Wojciech Roszka, 2022. "Are female scientists less inclined to publish alone? The gender solo research gap," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1697-1735, April.
    8. Bransch, Felix & Kvasnicka, Michael, 2022. "Male Gatekeepers: Gender Bias in the Publishing Process?," Journal of Economic Behavior & Organization, Elsevier, vol. 202(C), pages 714-732.
    9. Booth, Alison L., 2009. "Gender and competition," Labour Economics, Elsevier, vol. 16(6), pages 599-606, December.
    10. Kwiek, Marek & Roszka, Wojciech, 2021. "Gender-based homophily in research: A large-scale study of man-woman collaboration," Journal of Informetrics, Elsevier, vol. 15(3).
    11. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2016. "Gender bias in academic recruitment," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 119-141, January.
    12. Beaudry, Catherine & Larivière, Vincent, 2016. "Which gender gap? Factors affecting researchers’ scientific impact in science and medicine," Research Policy, Elsevier, vol. 45(9), pages 1790-1817.
    13. Becchetti, Leonardo & Degli Antoni, Giacomo & Ottone, Stefania & Solferino, Nazaria, 2013. "Allocation criteria under task performance: The gendered preference for protection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 45(C), pages 96-111.
    14. Balafoutas, Loukas & Sutter, Matthias, 2019. "How uncertainty and ambiguity in tournaments affect gender differences in competitive behavior," European Economic Review, Elsevier, vol. 118(C), pages 1-13.
    15. Michalis Drouvelis & Julian C. Jamison, 2015. "Selecting public goods institutions: Who likes to punish and reward?," Southern Economic Journal, John Wiley & Sons, vol. 82(2), pages 501-534, October.
    16. Czibor, Eszter & Claussen, Jörg & van Praag, Mirjam, 2019. "Women in a men’s world: Risk taking in an online card game community," Journal of Economic Behavior & Organization, Elsevier, vol. 158(C), pages 62-89.
    17. Giliberto Capano & Benedetto Lepori, 2024. "Designing policies that could work: understanding the interaction between policy design spaces and organizational responses in public sector," Policy Sciences, Springer;Society of Policy Sciences, vol. 57(1), pages 53-82, March.
    18. Marianne Bertrand & Esther Duflo, 2016. "Field Experiments on Discrimination," NBER Working Papers 22014, National Bureau of Economic Research, Inc.
    19. Laura Hospido & Luc Laeven & Ana Lamo, 2022. "The Gender Promotion Gap: Evidence from Central Banking," The Review of Economics and Statistics, MIT Press, vol. 104(5), pages 981-996, December.
    20. Andrea Blasco & Olivia S. Jung & Karim R. Lakhani & Michael Menietti, 2016. "Motivating Effort In Contributing to Public Goods Inside Organizations: Field Experimental Evidence," NBER Working Papers 22189, National Bureau of Economic Research, Inc.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:46:y:2017:i:5:p:911-924. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.