IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v116y2018i2d10.1007_s11192-018-2763-1.html
   My bibliography  Save this article

The impact of conference ranking systems in computer science: a comparative regression analysis

Author

Listed:
  • Xiancheng Li

    (Beihang University
    Beihang University)

  • Wenge Rong

    (Beihang University
    Beihang University)

  • Haoran Shi

    (Beihang University)

  • Jie Tang

    (Tsinghua University)

  • Zhang Xiong

    (Beihang University
    Beihang University)

Abstract

Nowadays, conference publications have gained importance both quantitatively and qualitatively. People are seeking ways to distinguish the quality and impact of different conferences. Some bibliometrics like conference impact factor have been proposed to assess them. Meanwhile, associations in some countries have implemented several projects to build conference ranking systems that classify conferences based on certain quality measures. The Computing Research and Education Association of Australasia and China Computer Federation lists are two such well-known and widely used conference rating systems. They can serve as a guide when researchers need to publish their work. At the same time, they can influence researchers’ publication decisions. In this paper, we try to find out how publication patterns in different countries have been influenced by these two lists as well as by some other factors. A random-effect Negative Binominal Regression Model is used to identify the level of the impact caused by different factors.

Suggested Citation

  • Xiancheng Li & Wenge Rong & Haoran Shi & Jie Tang & Zhang Xiong, 2018. "The impact of conference ranking systems in computer science: a comparative regression analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 879-907, August.
    • Handle: RePEc:spr:scient:v:116:y:2018:i:2:d:10.1007_s11192-018-2763-1
    DOI: 10.1007/s11192-018-2763-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-018-2763-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-018-2763-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anne-Wil Harzing & Satu Alakangas, 2017. "Microsoft Academic: is the phoenix getting wings?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 371-383, January.
    2. Helmut A. Abt & Eugene Garfield, 2002. "Is the relationship between numbers of references and paper lengths the same for all sciences?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(13), pages 1106-1112, November.
    3. Frondel, Manuel & Vance, Colin, 2010. "Fixed, random, or something in between? A variant of Hausman's specification test for panel data estimators," Economics Letters, Elsevier, vol. 107(3), pages 327-329, June.
    4. Fairclough, Ruth & Thelwall, Mike, 2015. "More precise methods for national research citation impact comparisons," Journal of Informetrics, Elsevier, vol. 9(4), pages 895-906.
    5. Jiancheng Guan & Xia Gao, 2008. "Comparison and evaluation of Chinese research performance in the field of bioinformatics," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 357-379, May.
    6. Waister Silva Martins & Marcos André Gonçalves & Alberto H. F. Laender & Nivio Ziviani, 2010. "Assessing the quality of scientific conferences based on bibliographic citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 133-155, April.
    7. Simone Diniz Junqueira Barbosa & Milene Selbach Silveira & Isabela Gasparini, 2017. "What publications metadata tell us about the evolution of a scientific community: the case of the Brazilian human–computer interaction conference series," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 275-300, January.
    8. In-Uck Park & Mike W. Peacey & Marcus R. Munafò, 2014. "Modelling the effects of subjective and objective decision making in scientific peer review," Nature, Nature, vol. 506(7486), pages 93-96, February.
    9. Peep Küngas & Siim Karus & Svitlana Vakulenko & Marlon Dumas & Cristhian Parra & Fabio Casati, 2013. "Reverse-engineering conference rankings: what does it take to make a reputable conference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 651-665, August.
    10. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    11. repec:zbw:rwirep:0160 is not listed on IDEAS
    12. Perlin, Marcelo S. & Santos, André A.P. & Imasato, Takeyoshi & Borenstein, Denis & Da Silva, Sergio, 2017. "The Brazilian scientific output published in journals: A study based on a large CV database," Journal of Informetrics, Elsevier, vol. 11(1), pages 18-31.
    13. Michel Zitt, 2012. "The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 485-503, August.
    14. Jiancheng Guan & Nan Ma, 2007. "A bibliometric study of China’s semiconductor literature compared with other major asian countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 70(1), pages 107-124, January.
    15. George Vrettas & Mark Sanderson, 2015. "Conferences versus journals in computer science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2674-2684, December.
    16. Costas, Rodrigo & Bordons, María, 2007. "The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level," Journal of Informetrics, Elsevier, vol. 1(3), pages 193-203.
    17. Ying He & Jiancheng Guan, 2008. "Contribution of Chinese publications in computer science: A case study on LNCS," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(3), pages 519-534, June.
    18. Anne-Wil Harzing & Satu Alakangas, 2017. "Microsoft Academic is one year old: the Phoenix is ready to leave the nest," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1887-1894, September.
    19. Ali Gazni & Cassidy R. Sugimoto & Fereshteh Didegah, 2012. "Mapping world scientific collaboration: Authors, institutions, and countries," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 323-335, February.
    20. Harzing, Anne-Wil & Giroud, Axèle, 2014. "The competitive advantage of nations: An application to academia," Journal of Informetrics, Elsevier, vol. 8(1), pages 29-42.
    21. Yifan Qian & Wenge Rong & Nan Jiang & Jie Tang & Zhang Xiong, 2017. "Citation regression analysis of computer science publications in different ranking categories and subfields," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1351-1374, March.
    22. Ali Gazni & Cassidy R. Sugimoto & Fereshteh Didegah, 2012. "Mapping world scientific collaboration: Authors, institutions, and countries," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 323-335, February.
    23. Yinian Gu, 2002. "An exploratory study of Malaysian publication productivity in computer science and information technology," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(12), pages 974-986.
    24. Suresh Kumar & K. C. Garg, 2005. "Scientometrics of computer science research in India and China," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 121-132, August.
    25. Sven E. Hug & Michael Ochsner & Martin P. Brändle, 2017. "Citation analysis with microsoft academic," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 371-378, April.
    26. Chen, Zifeng & Guan, Jiancheng, 2010. "The impact of small world on innovation: An empirical study of 16 countries," Journal of Informetrics, Elsevier, vol. 4(1), pages 97-106.
    27. Anne-Wil Harzing, 2016. "Microsoft Academic (Search): a Phoenix arisen from the ashes?," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1637-1647, September.
    28. Jiancheng Guan & Nan Ma, 2004. "A comparative study of research performance in computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(3), pages 339-359, November.
    29. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    30. Loizides, Orestis-Stavros & Koutsakis, Polychronis, 2017. "On evaluating the quality of a computer science/computer engineering conference," Journal of Informetrics, Elsevier, vol. 11(2), pages 541-552.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Danielle H. Lee, 2019. "Predictive power of conference-related factors on citation rates of conference papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 281-304, January.
    2. Marcelo Mendoza, 2021. "Differences in Citation Patterns across Areas, Article Types and Age Groups of Researchers," Publications, MDPI, vol. 9(4), pages 1-23, October.
    3. Saarela, Mirka & Kärkkäinen, Tommi, 2020. "Can we automate expert-based journal rankings? Analysis of the Finnish publication indicator," Journal of Informetrics, Elsevier, vol. 14(2).
    4. Omar Mubin & Fady Alnajjar & Abdullah Shamail & Suleman Shahid & Simeon Simoff, 2021. "The new norm: Computer Science conferences respond to COVID-19," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1813-1827, February.
    5. Evelyn Eika & Frode Eika Sandnes, 2022. "Starstruck by journal prestige and citation counts? On students’ bias and perceptions of trustworthiness according to clues in publication references," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6363-6390, November.
    6. Meho, Lokman I., 2019. "Using Scopus’s CiteScore for assessing the quality of computer science conferences," Journal of Informetrics, Elsevier, vol. 13(1), pages 419-433.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Meho, Lokman I., 2019. "Using Scopus’s CiteScore for assessing the quality of computer science conferences," Journal of Informetrics, Elsevier, vol. 13(1), pages 419-433.
    4. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    5. Anne-Wil Harzing, 2019. "Two new kids on the block: How do Crossref and Dimensions compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 341-349, July.
    6. Danielle H. Lee, 2019. "Predictive power of conference-related factors on citation rates of conference papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 281-304, January.
    7. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    8. Mike Thelwall, 2018. "Does Microsoft Academic find early citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 325-334, January.
    9. Michael Thelwall, 2018. "Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 913-928, May.
    10. B. M. Gupta & Avinash Kshitij & Charu Verma, 2011. "Mapping of Indian computer science research output, 1999–2008," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 261-283, February.
    11. Sven E. Hug & Martin P. Brändle, 2017. "The coverage of Microsoft Academic: analyzing the publication output of a university," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1551-1571, December.
    12. Robin Haunschild & Sven E. Hug & Martin P. Brändle & Lutz Bornmann, 2018. "The number of linked references of publications in Microsoft Academic in comparison with the Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 367-370, January.
    13. Camil Demetrescu & Irene Finocchi & Andrea Ribichini & Marco Schaerf, 2022. "On computer science research and its temporal evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4913-4938, August.
    14. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    15. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    16. María Bordons & Borja González-Albo & Javier Aparicio & Luz Moreno, 2015. "The influence of R&D intensity of countries on the impact of international collaborative research: evidence from Spain," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1385-1400, February.
    17. Dalibor Fiala & Gabriel Tutoky, 2017. "Computer Science Papers in Web of Science: A Bibliometric Analysis," Publications, MDPI, vol. 5(4), pages 1-16, September.
    18. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    19. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    20. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 871-906, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:116:y:2018:i:2:d:10.1007_s11192-018-2763-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.