IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v14y2020i2s1751157718301901.html
   My bibliography  Save this article

Application of machine learning techniques to assess the trends and alignment of the funded research output

Author

Listed:
  • Ebadi, Ashkan
  • Tremblay, Stéphane
  • Goutte, Cyril
  • Schiffauerova, Andrea

Abstract

Research and development activities are regarded as one of the most influencing factors of the future of a country. Large investments in research can yield a tremendous outcome in terms of a country’s overall wealth and strength. However, public financial resources of countries are often limited which calls for a wise and targeted investment. Scientific publications are considered as one of the main outputs of research investment. Although the general trend of scientific publications is increasing, a detailed analysis is required to monitor the research trends and assess whether they are in line with the top research priorities of the country. Such focused monitoring can shed light on scientific activities evolution as well as the formation of new research areas, thus helping governments to adjust priorities, if required. But monitoring the output of the funded research manually is not only very expensive and difficult, it is also subjective. Using structural topic models, in this paper we evaluated the trends in academic research performed by federally funded Canadian researchers during the time-frame of 2000–2018, covering more than 140,000 research publications. The proposed approach makes it possible to objectively and systematically monitor research projects, or any other set of documents related to research activities such as funding proposals, at large-scale. Our results confirm the accordance between the performed federally funded research projects and the top research priorities of Canada.

Suggested Citation

  • Ebadi, Ashkan & Tremblay, Stéphane & Goutte, Cyril & Schiffauerova, Andrea, 2020. "Application of machine learning techniques to assess the trends and alignment of the funded research output," Journal of Informetrics, Elsevier, vol. 14(2).
    • Handle: RePEc:eee:infome:v:14:y:2020:i:2:s1751157718301901
    DOI: 10.1016/j.joi.2020.101018
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157718301901
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2020.101018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kulczycki, Emanuel & Korzeń, Marcin & Korytkowski, Przemysław, 2017. "Toward an excellence-based research funding system: Evidence from Poland," Journal of Informetrics, Elsevier, vol. 11(1), pages 282-298.
    2. An Zeng & Zhesi Shen & Jianlin Zhou & Ying Fan & Zengru Di & Yougui Wang & H. Eugene Stanley & Shlomo Havlin, 2019. "Increasing trend of scientists to switch between topics," Nature Communications, Nature, vol. 10(1), pages 1-11, December.
    3. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    4. Grajzl, Peter & Murrell, Peter, 2019. "Toward understanding 17th century English culture: A structural topic model of Francis Bacon's ideas," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 111-135.
    5. Cassidy R. Sugimoto & Daifeng Li & Terrell G. Russell & S. Craig Finlay & Ying Ding, 2011. "The shifting sands of disciplinary development: Analyzing North American Library and Information Science dissertations using latent Dirichlet allocation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(1), pages 185-204, January.
    6. Cassidy R. Sugimoto & Daifeng Li & Terrell G. Russell & S. Craig Finlay & Ying Ding, 2011. "The shifting sands of disciplinary development: Analyzing North American Library and Information Science dissertations using latent Dirichlet allocation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(1), pages 185-204, January.
    7. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    8. Ebadi, Ashkan & Schiffauerova, Andrea, 2015. "How to become an important player in scientific collaboration networks?," Journal of Informetrics, Elsevier, vol. 9(4), pages 809-825.
    9. Ashkan Ebadi & Andrea Schiffauerova, 2015. "How to Receive More Funding for Your Research? Get Connected to the Right People!," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-19, July.
    10. Ashkan Ebadi & Andrea Schiffauerova, 2016. "How to boost scientific production? A statistical analysis of research funding and other influencing factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1093-1116, March.
    11. Hoyeop Lee & Jueun Kwak & Min Song & Chang Ouk Kim, 2015. "Coherence analysis of research and education using topic modeling," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1119-1137, February.
    12. Margaret Roberts & Brandon Stewart & Tingley, Dustin, 2014. "stm: R Package for Structural Topic Models," Working Paper 176291, Harvard University OpenScholar.
    13. Lucas, Christopher & Nielsen, Richard A. & Roberts, Margaret E. & Stewart, Brandon M. & Storer, Alex & Tingley, Dustin, 2015. "Computer-Assisted Text Analysis for Comparative Politics," Political Analysis, Cambridge University Press, vol. 23(2), pages 254-277, April.
    14. Erjia Yan, 2014. "Topic-based Pagerank: toward a topic-level scientific evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 407-437, August.
    15. Margaret E. Roberts & Brandon M. Stewart & Dustin Tingley & Christopher Lucas & Jetson Leder‐Luis & Shana Kushner Gadarian & Bethany Albertson & David G. Rand, 2014. "Structural Topic Models for Open‐Ended Survey Responses," American Journal of Political Science, John Wiley & Sons, vol. 58(4), pages 1064-1082, October.
    16. van den Besselaar, Peter & Heyman, Ulf & Sandström, Ulf, 2017. "Perverse effects of output-based research funding? Butler’s Australian case revisited," Journal of Informetrics, Elsevier, vol. 11(3), pages 905-918.
    17. Zhang, Yi & Zhang, Guangquan & Chen, Hongshu & Porter, Alan L. & Zhu, Donghua & Lu, Jie, 2016. "Topic analysis and forecasting for science, technology and innovation: Methodology with a case study focusing on big data research," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 179-191.
    18. Ashkan Ebadi & Andrea Schiffauerova, 2016. "iSEER: an intelligent automatic computer system for scientific evaluation of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 477-498, May.
    19. Bagozzi, Benjamin E. & Berliner, Daniel, 2018. "The Politics of Scrutiny in Human Rights Monitoring: Evidence from Structural Topic Models of US State Department Human Rights Reports," Political Science Research and Methods, Cambridge University Press, vol. 6(4), pages 661-677, October.
    20. Ubfal, Diego & Maffioli, Alessandro, 2011. "The impact of funding on research collaboration: Evidence from a developing country," Research Policy, Elsevier, vol. 40(9), pages 1269-1279.
    21. Jean‐Christophe Doré & Tiiu Ojasoo, 2001. "How to analyze publication time trends by correspondence factor analysis: Analysis of publications by 48 countries in 18 disciplines over 12 years," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(9), pages 763-769.
    22. Diego Ubfal & Alessandro Maffioli, 2010. "The Impact of Funding on Research Collaboration: Evidence from Argentina," SPD Working Papers 1006, Inter-American Development Bank, Office of Strategic Planning and Development Effectiveness (SPD).
    23. Hyun-Lim Yang & Tai-Woo Chang & Yerim Choi, 2018. "Exploring the Research Trend of Smart Factory with Topic Modeling," Sustainability, MDPI, vol. 10(8), pages 1-15, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lijie Feng & Kehui Liu & Jinfeng Wang & Kuo-Yi Lin & Ke Zhang & Luyao Zhang, 2022. "Identifying Promising Technologies of Electric Vehicles from the Perspective of Market and Technical Attributes," Energies, MDPI, vol. 15(20), pages 1-22, October.
    2. Fernandez Martinez, Roberto & Lostado Lorza, Ruben & Santos Delgado, Ana Alexandra & Piedra, Nelson, 2021. "Use of classification trees and rule-based models to optimize the funding assignment to research projects: A case study of UTPL," Journal of Informetrics, Elsevier, vol. 15(1).
    3. Ashkan Ebadi & Pengcheng Xi & Stéphane Tremblay & Bruce Spencer & Raman Pall & Alexander Wong, 2021. "Understanding the temporal evolution of COVID-19 research through machine learning and natural language processing," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 725-739, January.
    4. Benjamin M. Knisely & Holly H. Pavliscsak, 2023. "Research proposal content extraction using natural language processing and semi-supervised clustering: A demonstration and comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 3197-3224, May.
    5. Soroush Taheri & Sadegh Aliakbary, 2022. "Research trend prediction in computer science publications: a deep neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 849-869, February.
    6. Hajibabaei, Anahita & Schiffauerova, Andrea & Ebadi, Ashkan, 2022. "Gender-specific patterns in the artificial intelligence scientific ecosystem," Journal of Informetrics, Elsevier, vol. 16(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.
    2. Peter Grajzl & Peter Murrell, 2021. "Characterizing a legal–intellectual culture: Bacon, Coke, and seventeenth-century England," Cliometrica, Journal of Historical Economics and Econometric History, Association Française de Cliométrie (AFC), vol. 15(1), pages 43-88, January.
    3. Anahita Hajibabaei & Andrea Schiffauerova & Ashkan Ebadi, 2023. "Women and key positions in scientific collaboration networks: analyzing central scientists’ profiles in the artificial intelligence ecosystem through a gender lens," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1219-1240, February.
    4. Edward Kerby & Alexander Moradi & Hanjo Odendaal, 2022. "African time travellers: what can we learn from 500 years of written accounts?," Oxford Economic and Social History Working Papers _201, University of Oxford, Department of Economics.
    5. Star X. Zhao & Shuang Yu & Alice M. Tan & Xin Xu & Haiyan Yu, 2016. "Global pattern of science funding in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(1), pages 463-479, October.
    6. Belén Álvarez-Bornstein & Adrián A. Díaz-Faes & María Bordons, 2019. "What characterises funded biomedical research? Evidence from a basic and a clinical domain," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 805-825, May.
    7. Grajzl, Peter & Murrell, Peter, 2021. "A machine-learning history of English caselaw and legal ideas prior to the Industrial Revolution I: generating and interpreting the estimates," Journal of Institutional Economics, Cambridge University Press, vol. 17(1), pages 1-19, February.
    8. Dehler-Holland, Joris & Schumacher, Kira & Fichtner, Wolf, 2021. "Topic Modeling Uncovers Shifts in Media Framing of the German Renewable Energy Act," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 2(1).
    9. Ferrara, Federico M. & Masciandaro, Donato & Moschella, Manuela & Romelli, Davide, 2022. "Political voice on monetary policy: Evidence from the parliamentary hearings of the European Central Bank," European Journal of Political Economy, Elsevier, vol. 74(C).
    10. Álvarez-Bornstein, Belén & Bordons, María, 2021. "Is funding related to higher research impact? Exploring its relationship and the mediating role of collaboration in several disciplines," Journal of Informetrics, Elsevier, vol. 15(1).
    11. Andreas Rehs, 2020. "A structural topic model approach to scientific reorientation of economics and chemistry after German reunification," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1229-1251, November.
    12. Dehler-Holland, Joris & Okoh, Marvin & Keles, Dogan, 2022. "Assessing technology legitimacy with topic models and sentiment analysis – The case of wind power in Germany," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    13. Ulrich Fritsche & Johannes Puckelwald, 2018. "Deciphering Professional Forecasters’ Stories - Analyzing a Corpus of Textual Predictions for the German Economy," Macroeconomics and Finance Series 201804, University of Hamburg, Department of Socioeconomics.
    14. Hager, Anselm & Hilbig, Hanno, 2020. "Does Public Opinion Affect Political Speech?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 64(4), pages 921-937.
    15. Seraphine F. Maerz & Carsten Q. Schneider, 2020. "Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 517-545, April.
    16. Peter Grajzl & Cindy Irby, 2019. "Reflections on study abroad: a computational linguistics approach," Journal of Computational Social Science, Springer, vol. 2(2), pages 151-181, July.
    17. Xieling Chen & Juan Chen & Gary Cheng & Tao Gong, 2020. "Topics and trends in artificial intelligence assisted human brain research," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-27, April.
    18. Jing Tu, 2019. "What connections lead to good scientific performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 587-604, February.
    19. Mengjiao Qi & An Zeng & Menghui Li & Ying Fan & Zengru Di, 2017. "Standing on the shoulders of giants: the effect of outstanding scientists on young collaborators’ careers," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1839-1850, June.
    20. Nikoleta E. Glynatsi & Vincent A. Knight, 2021. "A bibliometric study of research topics, collaboration, and centrality in the iterated prisoner’s dilemma," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-12, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:14:y:2020:i:2:s1751157718301901. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.