IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v10y2016i4p1005-1022.html
   My bibliography  Save this article

Is this conference a top-tier? ConfAssist: An assistive conflict resolution framework for conference categorization

Author

Listed:
  • Singh, Mayank
  • Chakraborty, Tanmoy
  • Mukherjee, Animesh
  • Goyal, Pawan

Abstract

Classifying publication venues into top-tier or non-top-tier is quite subjective and can be debatable at times. In this paper, we propose ConfAssist, a novel assisting framework for conference categorization that aims to address the limitations in the existing systems and portals for venue classification. We start with the hypothesis that top-tier conferences are much more stable than other conferences and the inherent dynamics of these groups differs to a very large extent. We identify various features related to the stability of conferences that might help us separate a top-tier conference from the rest of the lot. While there are many clear cases where expert agreement can be almost immediately achieved as to whether a conference is a top-tier or not, there are equally many cases that can result in a conflict even among the experts. ConfAssist tries to serve as an aid in such cases by increasing the confidence of the experts in their decision. An analysis of 110 conferences from 22 sub-fields of computer science clearly favors our hypothesis as the top-tier conferences are found to exhibit much less fluctuations in the stability related features than the non-top-tier ones. We evaluate our hypothesis using systems based on conference categorization. For the evaluation, we conducted human judgment survey with 28 domain experts. The results are impressive with 85.18% classification accuracy. We also compare the dynamics of the newly started conferences with the older conferences to identify the initial signals of popularity. The system is applicable to any conference with atleast 5 years of publication history.

Suggested Citation

  • Singh, Mayank & Chakraborty, Tanmoy & Mukherjee, Animesh & Goyal, Pawan, 2016. "Is this conference a top-tier? ConfAssist: An assistive conflict resolution framework for conference categorization," Journal of Informetrics, Elsevier, vol. 10(4), pages 1005-1022.
    • Handle: RePEc:eee:infome:v:10:y:2016:i:4:p:1005-1022
    DOI: 10.1016/j.joi.2016.08.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157715301255
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2016.08.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dunaiski, Marcel & Visser, Willem & Geldenhuys, Jaco, 2016. "Evaluating paper and author ranking algorithms using impact and contribution awards," Journal of Informetrics, Elsevier, vol. 10(2), pages 392-407.
    2. Ismael Rafols & Martin Meyer, 2010. "Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 263-287, February.
    3. Alan L. Porter & Alex S. Cohen & J. David Roessner & Marty Perreault, 2007. "Measuring researcher interdisciplinarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 117-147, July.
    4. Leydesdorff, Loet & Rafols, Ismael, 2011. "Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations," Journal of Informetrics, Elsevier, vol. 5(1), pages 87-100.
    5. Huang, Ding-wei, 2016. "Positive correlation between quality and quantity in academic journals," Journal of Informetrics, Elsevier, vol. 10(2), pages 329-335.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Arthur Lackner & Said Fathalla & Mojtaba Nayyeri & Andreas Behrend & Rainer Manthey & Sören Auer & Jens Lehmann & Sahar Vahdati, 2021. "Analysing the evolution of computer science events leveraging a scholarly knowledge graph: a scientometrics study of top-ranked events in the past decade," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8129-8151, September.
    2. Alhoori, Hamed & Furuta, Richard, 2017. "Recommendation of scholarly venues based on dynamic user interests," Journal of Informetrics, Elsevier, vol. 11(2), pages 553-563.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    2. Wolfgang Glänzel & Koenraad Debackere, 2022. "Various aspects of interdisciplinarity in research and how to quantify and measure those," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5551-5569, September.
    3. Zhichao Ba & Yujie Cao & Jin Mao & Gang Li, 2019. "A hierarchical approach to analyzing knowledge integration between two fields—a case study on medical informatics and computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1455-1486, June.
    4. Alfonso Ávila-Robinson & Cristian Mejia & Shintaro Sengoku, 2021. "Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7477-7502, September.
    5. Zhao, Yi & Liu, Lifan & Zhang, Chengzhi, 2022. "Is coronavirus-related research becoming more interdisciplinary? A perspective of co-occurrence analysis and diversity measure of scientific articles," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    6. Xuefeng Wang & Zhinan Wang & Ying Huang & Yun Chen & Yi Zhang & Huichao Ren & Rongrong Li & Jinhui Pang, 2017. "Measuring interdisciplinarity of a research system: detecting distinction between publication categories and citation categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 2023-2039, June.
    7. Jorge Mannana-Rodriguez & Elea Giménez-Toledo, 2018. "Specialization and multidisciplinarity of scholarly book publishers: differences between Spanish University Presses and other scholarly publishers," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 19-30, January.
    8. Lin Zhang & Beibei Sun & Zaida Chinchilla-Rodríguez & Lixin Chen & Ying Huang, 2018. "Interdisciplinarity and collaboration: on the relationship between disciplinary diversity in departmental affiliations and reference lists," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 271-291, October.
    9. Shogo Katoh & Rick (H.L.) Aalbers & Shintaro Sengoku, 2021. "Effects and Interactions of Researcher’s Motivation and Personality in Promoting Interdisciplinary and Transdisciplinary Research," Sustainability, MDPI, vol. 13(22), pages 1-19, November.
    10. Shunshun Shi & Wenyu Zhang & Shuai Zhang & Jie Chen, 2018. "Does prestige dimension influence the interdisciplinary performance of scientific entities in knowledge flow? Evidence from the e-government field," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 1237-1264, November.
    11. Ruimin Ma & Erjia Yan, 2016. "Uncovering inter-specialty knowledge communication using author citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 839-854, November.
    12. Ilya V. Ponomarev & Brian K. Lawton & Duane E. Williams & Joshua D. Schnell, 2014. "Breakthrough paper indicator 2.0: can geographical diversity and interdisciplinarity improve the accuracy of outstanding papers prediction?," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(3), pages 755-765, September.
    13. Jingjing Ren & Fang Wang & Minglu Li, 2023. "Dynamics and characteristics of interdisciplinary research in scientific breakthroughs: case studies of Nobel-winning research in the past 120 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4383-4419, August.
    14. Kavitha Karunan & Hiran H. Lathabai & Thara Prabhakaran, 2017. "Discovering interdisciplinary interactions between two research fields using citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 335-367, October.
    15. Sándor Soós & George Kampis, 2012. "Beyond the basemap of science: mapping multiple structures in research portfolios: evidence from Hungary," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 869-891, December.
    16. Shengli Deng & Sudi Xia, 2020. "Mapping the interdisciplinarity in information behavior research: a quantitative study using diversity measure and co-occurrence analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 489-513, July.
    17. Jorge Mañana Rodríguez, 2017. "Disciplinarity and interdisciplinarity in citation and reference dimensions: knowledge importation and exportation taxonomy of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 617-642, February.
    18. Leydesdorff, Loet & Wagner, Caroline S. & Bornmann, Lutz, 2019. "Interdisciplinarity as diversity in citation patterns among journals: Rao-Stirling diversity, relative variety, and the Gini coefficient," Journal of Informetrics, Elsevier, vol. 13(1), pages 255-269.
    19. Loet Leydesdorff & Caroline S. Wagner & Lutz Bornmann, 2018. "Betweenness and diversity in journal citation networks as measures of interdisciplinarity—A tribute to Eugene Garfield," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 567-592, February.
    20. Stephen Carley & Alan L. Porter, 2012. "A forward diversity index," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 407-427, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:10:y:2016:i:4:p:1005-1022. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.