IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v113y2017i1d10.1007_s11192-017-2522-8.html
   My bibliography  Save this article

The emergence of a field: a network analysis of research on peer review

Author

Listed:
  • Vladimir Batagelj

    (Institute of Mathematics, Physics and Mechanics
    University of Primorska)

  • Anuška Ferligoj

    (University of Ljubljana)

  • Flaminio Squazzoni

    (University of Brescia)

Abstract

This article provides a quantitative analysis of peer review as an emerging field of research by revealing patterns and connections between authors, fields and journals from 1950 to 2016. By collecting all available sources from Web of Science, we built a dataset that included approximately 23,000 indexed records and reconstructed collaboration and citation networks over time. This allowed us to trace the emergence and evolution of this field of research by identifying relevant authors, publications and journals and revealing important development stages. Results showed that while the term “peer review” itself was relatively unknown before 1970 (“referee” was more frequently used), publications on peer review significantly grew especially after 1990. We found that the field was marked by three development stages: (1) before 1982, in which most influential studies were made by social scientists; (2) from 1983 to 2002, in which research was dominated by biomedical journals, and (3) from 2003 to 2016, in which specialised journals on science studies, such as Scientometrics, gained momentum frequently publishing research on peer review and so becoming the most influential outlets. The evolution of citation networks revealed a body of 47 publications that form the main path of the field, i.e., cited sources in all the most influential publications. They could be viewed as the main corpus of knowledge for any newcomer in the field.

Suggested Citation

  • Vladimir Batagelj & Anuška Ferligoj & Flaminio Squazzoni, 2017. "The emergence of a field: a network analysis of research on peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 503-532, October.
    • Handle: RePEc:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2522-8
    DOI: 10.1007/s11192-017-2522-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2522-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-017-2522-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John S. Liu & Louis Y.Y. Lu, 2012. "An integrated approach for main path analysis: Development of the Hirsch index as an example," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(3), pages 528-542, March.
    2. Halevi, Gali & Moed, Henk & Bar-Ilan, Judit, 2017. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the Literature," Journal of Informetrics, Elsevier, vol. 11(3), pages 823-834.
    3. Nees Jan Eck & Ludo Waltman, 2010. "Software survey: VOSviewer, a computer program for bibliometric mapping," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 523-538, August.
    4. John S. Liu & Louis Y.Y. Lu, 2012. "An integrated approach for main path analysis: Development of the Hirsch index as an example," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(3), pages 528-542, March.
    5. Loet Leydesdorff & Petra Ahrweiler, 2014. "In search of a network theory of innovations: Relations, positions, and perspectives," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(11), pages 2359-2374, November.
    6. Flaminio Squazzoni & Francisco Grimaldo & Ana Marušić, 2017. "Publishing: Journals could share peer-review data," Nature, Nature, vol. 546(7658), pages 352-352, June.
    7. Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
    8. Marianne Gauffriau & Peder Olesen Larsen & Isabelle Maye & Anne Roulin-Perriard & Markus Ins, 2007. "Publication, cooperation and productivity measures in scientific research," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(2), pages 175-214, November.
    9. van Eck, Nees Jan & Waltman, Ludo, 2014. "CitNetExplorer: A new software tool for analyzing and visualizing citation networks," Journal of Informetrics, Elsevier, vol. 8(4), pages 802-823.
    10. Niccolò Casnici & Francisco Grimaldo & Nigel Gilbert & Flaminio Squazzoni, 2017. "Attitudes of referees in a multidisciplinary journal: An empirical analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(7), pages 1763-1771, July.
    11. Eugene Garfield & A. I. Pudovkin & V. S. Istomin, 2003. "Why do we need algorithmic historiography?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(5), pages 400-412, March.
    12. Vladimir Batagelj & Monika Cerinšek, 2013. "On bibliographic networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 845-864, September.
    13. Flaminio Squazzoni & Károly Takács, 2011. "Social Simulation That 'Peers into Peer Review'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 14(4), pages 1-3.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bravo, Giangiacomo & Farjam, Mike & Grimaldo Moreno, Francisco & Birukou, Aliaksandr & Squazzoni, Flaminio, 2018. "Hidden connections: Network effects on editorial decisions in four computer science journals," Journal of Informetrics, Elsevier, vol. 12(1), pages 101-112.
    2. Abdelghani Maddi, 2018. "Analyse scientométrique de la crise économique," CEPN Working Papers 2018-08, Centre d'Economie de l'Université de Paris Nord.
    3. Batagelj, Vladimir & Maltseva, Daria, 2020. "Temporal bibliographic networks," Journal of Informetrics, Elsevier, vol. 14(1).
    4. Daria Maltseva & Vladimir Batagelj, 2020. "Towards a systematic description of the field using keywords analysis: main topics in social networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 357-382, April.
    5. John S. Liu & Louis Y. Y. Lu & Mei Hsiu-Ching Ho, 2019. "A few notes on main path analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 379-391, April.
    6. Abdelghani Maddi, 2018. "Analyse scientométrique de la crise économique : Courants de pensée, auteurs influents et thématiques," CEPN Working Papers hal-01922256, HAL.
    7. Flavia Filippin, 2021. "Do main paths reflect technological trajectories? Applying main path analysis to the semiconductor manufacturing industry," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6443-6477, August.
    8. Abdelghani Maddi, 2018. "Analyse scientométrique de la crise économique : Courants de pensée, auteurs influents et thématiques," Working Papers hal-01922256, HAL.
    9. Shuo Xu & Liyuan Hao & Xin An & Hongshen Pang & Ting Li, 2020. "Review on emerging research topics with key-route main path analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 607-624, January.
    10. Guillermo Armando Ronda-Pupo & Thong Pham, 2018. "The evolutions of the rich get richer and the fit get richer phenomena in scholarly networks: the case of the strategic management journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 363-383, July.
    11. Wenqing Wu & Haixu Xi & Chengzhi Zhang, 2024. "Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4109-4135, July.
    12. Petersen, Alexander M., 2019. "Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE," Journal of Informetrics, Elsevier, vol. 13(4).
    13. Daria Maltseva & Vladimir Batagelj, 2021. "Journals publishing social network analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3593-3620, April.
    14. Daria Maltseva & Vladimir Batagelj, 2020. "iMetrics: the development of the discipline with many names," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 313-359, October.
    15. Francesco Pasimeni, 2020. "The Origin of the Sharing Economy Meets the Legacy of Fractional Ownership," SPRU Working Paper Series 2020-19, SPRU - Science Policy Research Unit, University of Sussex Business School.
    16. Daria Maltseva & Vladimir Batagelj, 2019. "Social network analysis as a field of invasions: bibliographic approach to study SNA development," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 1085-1128, November.
    17. Hans-Dieter Daniel, 2019. "Lutz Bornmann: Recipient of the 2019 Derek John de Solla Price Medal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1235-1238, December.
    18. Aliakbar Akbaritabar & Vincent Antonio Traag & Alberto Caimo & Flaminio Squazzoni, 2020. "Italian sociologists: a community of disconnected groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2361-2382, September.
    19. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    20. Daria Maltseva & Vladimir Batagelj, 2022. "Collaboration between authors in the field of social network analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3437-3470, June.
    21. Akbaritabar, Aliakbar & Stephen, Dimity & Squazzoni, Flaminio, 2022. "A study of referencing changes in preprint-publication pairs across multiple fields," Journal of Informetrics, Elsevier, vol. 16(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magdalena Olczyk, 2016. "A systematic retrieval of international competitiveness literature: a bibliometric study," Eurasian Economic Review, Springer;Eurasia Business and Economics Society, vol. 6(3), pages 429-457, December.
    2. Niccol? Comerio & Patrizia Tettamanzi, 2019. "Systematic literature network analysis in accounting: A first application on integrated reporting research," FINANCIAL REPORTING, FrancoAngeli Editore, vol. 2019(2), pages 73-95.
    3. Kashin Sugishita & Yasuo Asakura, 2021. "Vulnerability studies in the fields of transportation and complex networks: a citation network analysis," Public Transport, Springer, vol. 13(1), pages 1-34, March.
    4. Loet Leydesdorff & Jordan A. Comins & Aaron A. Sorensen & Lutz Bornmann & Iina Hellsten, 2016. "Cited references and Medical Subject Headings (MeSH) as two different knowledge representations: clustering and mappings at the paper level," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2077-2091, December.
    5. Daria Maltseva & Vladimir Batagelj, 2020. "iMetrics: the development of the discipline with many names," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 313-359, October.
    6. Xiaorui Jiang & Junjun Liu, 2023. "Extracting the evolutionary backbone of scientific domains: The semantic main path network analysis approach based on citation context analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 546-569, May.
    7. Elie, Luc & Granier, Caroline & Rigot, Sandra, 2021. "The different types of renewable energy finance: A Bibliometric analysis," Energy Economics, Elsevier, vol. 93(C).
    8. Bruno Miranda Henrique & Vinicius Amorim Sobreiro & Herbert Kimura, 2018. "Building direct citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 817-832, May.
    9. Francesco Pasimeni, 2020. "The Origin of the Sharing Economy Meets the Legacy of Fractional Ownership," SPRU Working Paper Series 2020-19, SPRU - Science Policy Research Unit, University of Sussex Business School.
    10. Chen, Kaihua & Zhang, Yi & Fu, Xiaolan, 2019. "International research collaboration: An emerging domain of innovation studies?," Research Policy, Elsevier, vol. 48(1), pages 149-168.
    11. Hiran H. Lathabai & Thara Prabhakaran & Manoj Changat, 2017. "Contextual productivity assessment of authors and journals: a network scientometric approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 711-737, February.
    12. Massimo Franceschet & Giovanni Colavizza, 2020. "Quantifying the higher-order influence of scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 951-963, November.
    13. Kim, Erin H.J. & Jeong, Yoo Kyung & Kim, YongHwan & Song, Min, 2022. "Exploring scientific trajectories of a large-scale dataset using topic-integrated path extraction," Journal of Informetrics, Elsevier, vol. 16(1).
    14. Alessandra Centinaio, 2024. "How gender diversity in boards affects disclosure? A literature review," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 31(3), pages 2352-2382, May.
    15. McLevey, John & McIlroy-Young, Reid, 2017. "Introducing metaknowledge: Software for computational research in information science, network analysis, and science of science," Journal of Informetrics, Elsevier, vol. 11(1), pages 176-197.
    16. Dejing Kong & Jianzhong Yang & Lingfeng Li, 2020. "Early identification of technological convergence in numerical control machine tool: a deep learning approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 1983-2009, December.
    17. Zoltán Lakner & Brigitta Plasek & Gyula Kasza & Anna Kiss & Sándor Soós & Ágoston Temesi, 2021. "Towards Understanding the Food Consumer Behavior–Food Safety–Sustainability Triangle: A Bibliometric Approach," Sustainability, MDPI, vol. 13(21), pages 1-23, November.
    18. Alessandri, Enrico, 2023. "Identifying technological trajectories in the mining sector using patent citation networks," Resources Policy, Elsevier, vol. 80(C).
    19. Thor, Andreas & Marx, Werner & Leydesdorff, Loet & Bornmann, Lutz, 2016. "Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization," Journal of Informetrics, Elsevier, vol. 10(2), pages 503-515.
    20. Mehdi Toloo & Rouhollah Khodabandelou & Amar Oukil, 2022. "A Comprehensive Bibliometric Analysis of Fractional Programming (1965–2020)," Mathematics, MDPI, vol. 10(11), pages 1-21, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2522-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.