IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v48y2000i1d10.1023_a1005636503358.html
   My bibliography  Save this article

The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea

Author

Listed:
  • Mushin Lee

    (Korea Advanced Institute of Science and Technology (KAIST))

  • Kiyong Om

    (Electronics and Telecommunications Research Institute (ETRI))

  • Joon Koh

    (Korea Advanced Institute of Science and Technology (KAIST))

Abstract

This article compares empirically the major factors affecting blinded and sighted reviewers in the selection of research proposals to be funded in a "scientifically small" country. Fisher's Z-test shows that the applicant characteristics (rank of undergraduate school where the applicant studied, professional age of the applicant, and academic recognition of the applicant) are the major factors leading to the significantly different evaluation scores between blinded and sighted reviewers. This means that "open" evaluation of research proposals is obviously biased. Policy implications of the findings and future research directions are discussed.

Suggested Citation

  • Mushin Lee & Kiyong Om & Joon Koh, 2000. "The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(1), pages 99-116, June.
    • Handle: RePEc:spr:scient:v:48:y:2000:i:1:d:10.1023_a:1005636503358
    DOI: 10.1023/A:1005636503358
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1023/A:1005636503358
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1023/A:1005636503358?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. M. Lee & K. Om & J. Koh, 1999. "Blind review of research proposals in Korea: Its effectiveness and factors affecting applicant detection," Scientometrics, Springer;Akadémiai Kiadó, vol. 45(1), pages 17-31, May.
    2. Lee, Mushin & Son, Byoungho & Om, Kiyong, 1996. "Evaluation of national R&D projects in Korea," Research Policy, Elsevier, vol. 25(5), pages 805-818, August.
    3. Chapman, I. D. & Farina, C., 1983. "Peer review and national need," Research Policy, Elsevier, vol. 12(6), pages 317-327, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Banal-Estañol, Albert & Macho-Stadler, Inés & Pérez-Castrillo, David, 2019. "Evaluation in research funding agencies: Are structurally diverse teams biased against?," Research Policy, Elsevier, vol. 48(7), pages 1823-1840.
    2. Alexandre Rodrigues Oliveira & Carlos Fernando Mello, 2016. "Importance and susceptibility of scientific productivity indicators: two sides of the same coin," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 697-722, November.
    3. Dennis L Murray & Douglas Morris & Claude Lavoie & Peter R Leavitt & Hugh MacIsaac & Michael E J Masson & Marc-Andre Villard, 2016. "Bias in Research Grant Evaluation Has Dire Consequences for Small Universities," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-19, June.
    4. Hendy Abdoul & Christophe Perrey & Florence Tubach & Philippe Amiel & Isabelle Durand-Zaleski & Corinne Alberti, 2012. "Non-Financial Conflicts of Interest in Academic Grant Evaluation: A Qualitative Study of Multiple Stakeholders in France," PLOS ONE, Public Library of Science, vol. 7(4), pages 1-10, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. M. Lee & K. Om & J. Koh, 1999. "Blind review of research proposals in Korea: Its effectiveness and factors affecting applicant detection," Scientometrics, Springer;Akadémiai Kiadó, vol. 45(1), pages 17-31, May.
    2. Kiwon Lee & Suchul Lee, 2023. "Enhancing R&D Performance Management: A Case of R&D Projects in South Korea," Sustainability, MDPI, vol. 15(15), pages 1-14, July.
    3. Sara Amoroso & Simone Vannuccini, 2019. "Teaming up with Large R&D Investors: Good or Bad for Knowledge Production and Diffusion?," SPRU Working Paper Series 2019-20, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Katarína Cechlárová & Tamás Fleiner & Eva Potpinková, 2014. "Assigning evaluators to research grant applications: the case of Slovak Research and Development Agency," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 495-506, May.
    5. Takahiro NISHI, 2015. "Corporate diversification and board composition in Japanese electronics corporations," International Journal of Business and Management, International Institute of Social and Economic Sciences, vol. 3(2), pages 27-44, May.
    6. Sharon Poczter, 2017. "Rethinking the government as innovator: Evidence from Asian firms," Asia Pacific Journal of Management, Springer, vol. 34(2), pages 367-397, June.
    7. Sandström, Ulf & Van den Besselaar, Peter, 2018. "Funding, evaluation, and the performance of national research systems," Journal of Informetrics, Elsevier, vol. 12(1), pages 365-384.
    8. Noh, Heeyong & Seo, Ju-Hwan & Sun Yoo, Hyoung & Lee, Sungjoo, 2018. "How to improve a technology evaluation model: A data-driven approach," Technovation, Elsevier, vol. 72, pages 1-12.
    9. Sakakibara, Mariko & Cho, Dong-Sung, 2002. "Cooperative R&D in Japan and Korea: a comparison of industrial policy," Research Policy, Elsevier, vol. 31(5), pages 673-692, July.
    10. Lee, Hakyeon & Park, Yongtae & Choi, Hoogon, 2009. "Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach," European Journal of Operational Research, Elsevier, vol. 196(3), pages 847-855, August.
    11. Kyoungmi Lee & Sunglok Choi & Jae-Suk Yang, 2021. "Can expensive research equipment boost research and development performances?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7715-7742, September.
    12. Bae, Sung Joo & Lee, Hyeonsuh, 2020. "The role of government in fostering collaborative R&D projects: Empirical evidence from South Korea," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    13. Leonardo Sastoque Pinilla & Raúl Llorente Rodríguez & Nerea Toledo Gandarias & Luis Norberto López de Lacalle & Mahboobeh Ramezani Farokhad, 2019. "TRLs 5–7 Advanced Manufacturing Centres, Practical Model to Boost Technology Transfer in Manufacturing," Sustainability, MDPI, vol. 11(18), pages 1-14, September.
    14. Dalpe, Robert & Anderson, Frances, 1995. "National priorities in academic research-strategic research and contracts in renewable energies," Research Policy, Elsevier, vol. 24(4), pages 563-581, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:48:y:2000:i:1:d:10.1023_a:1005636503358. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.