IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i9d10.1007_s11192-024-05090-4.html
   My bibliography  Save this article

Evaluating the wisdom of scholar crowds from the perspective of knowledge diffusion

Author

Listed:
  • Le Song

    (South China University of Technology)

  • Guilong Zhu

    (South China University of Technology)

  • Xiao Yin

    (South China University of Technology)

Abstract

‘The wisdom of crowds’ theory has received widespread attention and application. For scholars, the wisdom of crowds is of great significance in revealing the operating mechanism of the scientific community. However, scholar crowds are jointly affected by scientific cognition and coordination, which are different from general human crowds. ‘The wisdom of crowds’ theory poses significant challenges in terms of directly explaining and evaluating the wisdom generation among scholars. Considering that knowledge diffusion is an important way to generate scientific cognition and coordination, this work proposed ‘the wisdom of scholar crowds’ and evaluates it from the perspective of knowledge diffusion. First, scholar-paper and scholar-topic two-layer networks were constructed, achieving a holistic representation of scientific coordination and cognition in the network structure dimension. Second, the topic consistency among scholars was identified using the two-layer networks, and a knowledge diffusion evaluation model based on topic consistency was designed to evaluate the scale and threshold of the wisdom generation of scholar crowds. Finally, combined with 3,838,048 paper data, this work revealed that the cohesion and bridging of network structure contribute to the wisdom generation of scholar crowds. By comparing with the commonly used evaluation methods, this study shows that the generating difficulty of the wisdom of scholar crowds will be underestimated without topic consistency. This work provides a new perspective for expanding the ‘wisdom of crowds’ theory and a novel method for evaluating knowledge diffusion and the wisdom of scholar crowds.

Suggested Citation

  • Le Song & Guilong Zhu & Xiao Yin, 2024. "Evaluating the wisdom of scholar crowds from the perspective of knowledge diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(9), pages 5103-5139, September.
  • Handle: RePEc:spr:scient:v:129:y:2024:i:9:d:10.1007_s11192-024-05090-4
    DOI: 10.1007/s11192-024-05090-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05090-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-024-05090-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Albert E. Mannes, 2009. "Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision," Management Science, INFORMS, vol. 55(8), pages 1267-1279, August.
    2. De Paola, Maria & Gioia, Francesca & Scoppa, Vincenzo, 2019. "Free-riding and knowledge spillovers in teams: The role of social ties," European Economic Review, Elsevier, vol. 112(C), pages 74-90.
    3. Wei Wang & Shuo Yu & Teshome Megersa Bekele & Xiangjie Kong & Feng Xia, 2017. "Scientific collaboration patterns vary with scholars’ academic ages," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 329-343, July.
    4. Bart Thijs & Lin Zhang & Wolfgang Glänzel, 2015. "Bibliographic coupling and hierarchical clustering for the validation and improvement of subject-classification schemes," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1453-1467, December.
    5. Ying Huang & Wolfgang Glänzel & Lin Zhang, 2021. "Tracing the development of mapping knowledge domains," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6201-6224, July.
    6. Melika Mosleh & Saeed Roshani & Mario Coccia, 2022. "Scientific laws of research funding to support citations and diffusion of knowledge in life science," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1931-1951, April.
    7. Jingda Ding & Dehui Du, 2023. "A study of the correlation between publication delays and measurement indicators of journal articles in the social network environment—based on online data in PLOS," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1711-1743, March.
    8. Deichmann, Dirk & Moser, Christine & Birkholz, Julie M. & Nerghes, Adina & Groenewegen, Peter & Wang, Shenghui, 2020. "Ideas with impact: How connectivity shapes idea diffusion," Research Policy, Elsevier, vol. 49(1).
    9. Lei Wang & Bart Thijs & Wolfgang Glänzel, 2015. "Characteristics of international collaboration in sport sciences publications and its influence on citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(2), pages 843-862, November.
    10. Song, Le & Ma, Yinghong, 2022. "Evaluating tacit knowledge diffusion with algebra matrix algorithm based social networks," Applied Mathematics and Computation, Elsevier, vol. 428(C).
    11. Pavel Atanasov & Phillip Rescober & Eric Stone & Samuel A. Swift & Emile Servan-Schreiber & Philip Tetlock & Lyle Ungar & Barbara Mellers, 2017. "Distilling the Wisdom of Crowds: Prediction Markets vs. Prediction Polls," Management Science, INFORMS, vol. 63(3), pages 691-706, March.
    12. Diana Purwitasari & Chastine Fatichah & Surya Sumpeno & Christian Steglich & Mauridhi Hery Purnomo, 2020. "Identifying collaboration dynamics of bipartite author-topic networks with the influences of interest changes," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1407-1443, March.
    13. Chen, Ssu-Han & Huang, Mu-Hsuan & Chen, Dar-Zen, 2012. "Identifying and visualizing technology evolution: A case study of smart grid technology," Technological Forecasting and Social Change, Elsevier, vol. 79(6), pages 1099-1110.
    14. Liu, Yuxian & Rousseau, Ronald & Guns, Raf, 2013. "A layered framework to study collaboration as a form of knowledge sharing and diffusion," Journal of Informetrics, Elsevier, vol. 7(3), pages 651-664.
    15. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    16. Lin, Yiling & Evans, James A. & Wu, Lingfei, 2022. "New directions in science emerge from disconnection and discord," Journal of Informetrics, Elsevier, vol. 16(1).
    17. Tian, Yunpei & Li, Gang & Mao, Jin, 2023. "Predicting the evolution of scientific communities by interpretable machine learning approaches," Journal of Informetrics, Elsevier, vol. 17(2).
    18. Shan Jiang & Hsinchun Chen, 2019. "Examining patterns of scientific knowledge diffusion based on knowledge cyber infrastructure: a multi-dimensional network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1599-1617, December.
    19. Kiss, Istvan Z. & Broom, Mark & Craze, Paul G. & Rafols, Ismael, 2010. "Can epidemic models describe the diffusion of topics across disciplines?," Journal of Informetrics, Elsevier, vol. 4(1), pages 74-82.
    20. Jasjit Singh, 2005. "Collaborative Networks as Determinants of Knowledge Diffusion Patterns," Management Science, INFORMS, vol. 51(5), pages 756-770, May.
    21. Curtis Atkisson & Piotr J. G'orski & Matthew O. Jackson & Janusz A. Ho{l}yst & Raissa M. D'Souza, 2019. "Why understanding multiplex social network structuring processes will help us better understand the evolution of human behavior," Papers 1903.11183, arXiv.org, revised May 2020.
    22. Olle Persson & Wolfgang Glänzel & Rickard Danell, 2004. "Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 421-432, August.
    23. Lu Liu & Nima Dehmamy & Jillian Chown & C. Lee Giles & Dashun Wang, 2021. "Understanding the onset of hot streaks across artistic, cultural, and scientific careers," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    24. Lambiotte, R. & Panzarasa, P., 2009. "Communities, knowledge creation, and information diffusion," Journal of Informetrics, Elsevier, vol. 3(3), pages 180-190.
    25. Vincenz Frey & Arnout van de Rijt, 2021. "Social Influence Undermines the Wisdom of the Crowd in Sequential Decision Making," Management Science, INFORMS, vol. 67(7), pages 4273-4286, July.
    26. Franzoni, Chiara & Sauermann, Henry, 2014. "Crowd science: The organization of scientific research in open collaborative projects," Research Policy, Elsevier, vol. 43(1), pages 1-20.
    27. Saeed Osat & Ali Faqeeh & Filippo Radicchi, 2017. "Optimal percolation on multiplex networks," Nature Communications, Nature, vol. 8(1), pages 1-7, December.
    28. Zenghui Yue & Haiyun Xu & Guoting Yuan & Yan Qi, 2022. "Modeling knowledge diffusion in the disciplinary citation network based on differential dynamics," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7593-7613, December.
    29. Wang, Xiaoguang & He, Jing & Huang, Han & Wang, Hongyu, 2022. "MatrixSim: A new method for detecting the evolution paths of research topics," Journal of Informetrics, Elsevier, vol. 16(4).
    30. Mei Hsiu-Ching Ho & John S. Liu, 2013. "The motivations for knowledge transfer across borders: the diffusion of data envelopment analysis (DEA) methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 397-421, January.
    31. Zhi Da & Xing Huang, 2020. "Harnessing the Wisdom of Crowds," Management Science, INFORMS, vol. 66(5), pages 1847-1867, May.
    32. M. Pelacho & G. Ruiz & F. Sanz & A. Tarancón & J. Clemente-Gallardo, 2021. "Analysis of the evolution and collaboration networks of citizen science scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 225-257, January.
    33. Lutz Bornmann & Werner Marx, 2014. "The wisdom of citing scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(6), pages 1288-1292, June.
    34. Ba, Zhichao & Liang, Zhentao, 2021. "A novel approach to measuring science-technology linkage: From the perspective of knowledge network coupling," Journal of Informetrics, Elsevier, vol. 15(3).
    35. Jochen Gläser & Wolfgang Glänzel & Andrea Scharnhorst, 2017. "Same data—different results? Towards a comparative approach to the identification of thematic structures in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 981-998, May.
    36. Nicolaisen, Jeppe & Frandsen, Tove Faber, 2012. "Consensus formation in science modeled by aggregated bibliographic coupling," Journal of Informetrics, Elsevier, vol. 6(2), pages 276-284.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yue, Zenghui & Xu, Haiyun & Yuan, Guoting & Pang, Hongshen, 2019. "Modeling study of knowledge diffusion in scientific collaboration networks based on differential dynamics: A case study in graphene field," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 524(C), pages 375-391.
    2. Joshua Aaron Becker & Douglas Guilbeault & Edward Bishop Smith, 2022. "The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy," Management Science, INFORMS, vol. 68(5), pages 3949-3965, May.
    3. Joshua Becker & Douglas Guilbeault & Ned Smith, 2021. "The Crowd Classification Problem: Social Dynamics of Binary Choice Accuracy," Papers 2104.11300, arXiv.org.
    4. Tang, Ming & Liao, Huchang, 2024. "Group efficiency and individual fairness tradeoff in making wise decisions," Omega, Elsevier, vol. 124(C).
    5. Song, Le & Ma, Yinghong, 2022. "Evaluating tacit knowledge diffusion with algebra matrix algorithm based social networks," Applied Mathematics and Computation, Elsevier, vol. 428(C).
    6. Guo, Liying & Wang, Yang & Li, Meiling, 2024. "Exploration, exploitation and funding success: Evidence from junior scientists supported by the Chinese Young Scientists Fund," Journal of Informetrics, Elsevier, vol. 18(2).
    7. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    8. Li, Jingjing & Zhang, Yumei & Man, Jiayu & Zhou, Yun & Wu, Xiaojun, 2017. "SISL and SIRL: Two knowledge dissemination models with leader nodes on cooperative learning networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 468(C), pages 740-749.
    9. Lin Zhang & Beibei Sun & Fei Shu & Ying Huang, 2022. "Comparing paper level classifications across different methods and systems: an investigation of Nature publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7633-7651, December.
    10. Chi, Yuxue & Tang, Xianyi & Liu, Yijun, 2022. "Exploring the “awakening effect” in knowledge diffusion: a case study of publications in the library and information science domain," Journal of Informetrics, Elsevier, vol. 16(4).
    11. Xiaojun Hu & Ronald Rousseau & Jin Chen, 2012. "Structural indicators in citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 451-460, May.
    12. Keppo, Jussi & Satopää, Ville A., 2024. "Bayesian herd detection for dynamic data," International Journal of Forecasting, Elsevier, vol. 40(1), pages 285-301.
    13. Mingyang Wang & Jiaqi Zhang & Shijia Jiao & Tianyu Zhang, 2019. "Evaluating the impact of citations of articles based on knowledge flow patterns hidden in the citations," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-19, November.
    14. Joshua Becker & Abdullah Almaatouq & EmH{o}ke-'Agnes Horv'at, 2020. "Network Structures of Collective Intelligence: The Contingent Benefits of Group Discussion," Papers 2009.07202, arXiv.org, revised Mar 2021.
    15. Hric, Darko & Kaski, Kimmo & Kivelä, Mikko, 2018. "Stochastic block model reveals maps of citation patterns and their evolution in time," Journal of Informetrics, Elsevier, vol. 12(3), pages 757-783.
    16. Xuan Liu & Shan Jiang & Hsinchun Chen & Catherine A. Larson & Mihail C. Roco, 2015. "Modeling knowledge diffusion in scientific innovation networks: an institutional comparison between China and US with illustration for nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1953-1984, December.
    17. Bárbara S. Lancho-Barrantes & Vicente P. Guerrero-Bote & Félix Moya-Anegón, 2013. "Citation increments between collaborating countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 817-831, March.
    18. Satopää, Ville A. & Salikhov, Marat & Tetlock, Philip E. & Mellers, Barbara, 2023. "Decomposing the effects of crowd-wisdom aggregators: The bias–information–noise (BIN) model," International Journal of Forecasting, Elsevier, vol. 39(1), pages 470-485.
    19. Wang, Chun-Chieh & Lin, Jia-Tian & Chen, Dar-Zen & Lo, Szu-Chia, 2023. "A New Look at National Diversity of Inventor Teams within Organizations," Journal of Informetrics, Elsevier, vol. 17(1).
    20. Liao, Shi-Gen & Yi, Shu-Ping, 2021. "Modeling and analyzing knowledge transmission process considering free-riding behavior of knowledge acquisition: A waterborne disease approach," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 569(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:9:d:10.1007_s11192-024-05090-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.