IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v72y2007i2d10.1007_s11192-007-1696-x.html
   My bibliography  Save this article

Profiling citation impact: A new methodology

Author

Listed:
  • Jonathan Adams

    (Evidence Ltd)

  • Karen Gurney

    (Evidence Ltd)

  • Stuart Marshall

    (Evidence Ltd)

Abstract

A methodology for creating bibliometric impact profiles is described. The advantages of such profiles as a management tool to supplement the reporting power of traditional average impact metrics are discussed. The impact profile for the UK as a whole reveals the extent to which the median and modal UK impact values differ from and are significantly below average impact. Only one-third of UK output for 1995-2004 is above world average impact although the UK’s average world-normalised impact is 1.24. Time-categorised impact profiles are used to test hypotheses about changing impact and confirm that the increase in average UK impact is due to real improvement rather than a reduction in low impact outputs. The impact profile methodology has been applied across disciplines as well as years and is shown to work well in all subject categories. It reveals substantial variations in performance between disciplines. The value of calculating the profile median and mode as well as the average impact are demonstrated. Finally, the methodology is applied to a specific data-set to compare the impact profile of the elite Laboratory of Molecular Biology (Cambridge) with the relevant UK average. This demonstrates an application of the methodology by identifying where the institute’s exceptional performance is located. The value of impact profiles lies in their role as an interpretive aid for non-specialists, not as a technical transformation of the data for scientometricians.

Suggested Citation

  • Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
    • Handle: RePEc:spr:scient:v:72:y:2007:i:2:d:10.1007_s11192-007-1696-x
    DOI: 10.1007/s11192-007-1696-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-007-1696-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-007-1696-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Henry Small, 2004. "Why authors think their papers are highly cited," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 305-316, August.
    2. Loet Leydesdorff & Stephen Bensman, 2006. "Classification and powerlaws: The logarithmic transformation," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(11), pages 1470-1486, September.
    3. Michel Zitt & Suzy Ramanana-Rahary & Elise Bassecoulard, 2005. "Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 373-401, April.
    4. Jonathan Adams, 1998. "Benchmarking international research," Nature, Nature, vol. 396(6712), pages 615-618, December.
    5. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    6. David A. King, 2004. "The scientific impact of nations," Nature, Nature, vol. 430(6997), pages 311-316, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cova, Tânia F.G.G. & Jarmelo, Susana & Formosinho, Sebastião J. & de Melo, J. Sérgio Seixas & Pais, Alberto A.C.C., 2015. "Unsupervised characterization of research institutions with task-force estimation," Journal of Informetrics, Elsevier, vol. 9(1), pages 59-68.
    2. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & Nicolas Robinson-García & Daniel Torres-Salinas, 2013. "Benchmarking research performance at the university level with information theoretic measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 435-452, April.
    3. David A. Pendlebury & Jonathan Adams, 2012. "Comments on a critique of the Thomson Reuters journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 395-401, August.
    4. Michel Zitt, 2012. "The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 485-503, August.
    5. Jonathan Adams & Jo Johnson & Jonathan Grant, 2022. "The rise of UK–China research collaboration: Trends, opportunities and challenges [The West Should Start Sending Its Scientists to China]," Science and Public Policy, Oxford University Press, vol. 49(1), pages 132-147.
    6. Potter, Ross W.K. & Szomszor, Martin & Adams, Jonathan, 2020. "Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type," Journal of Informetrics, Elsevier, vol. 14(4).
    7. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    8. Wolfgang Glänzel & Bart Thijs & Koenraad Debackere, 2014. "The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 939-952, November.
    9. Oldac, Yusuf Ikbal & Yang, Lili, 2023. "Regionalisation and agency in science space: A historical bibliometric analysis of ASEAN science," International Journal of Educational Development, Elsevier, vol. 97(C).
    10. Juan Gorraiz & Ursula Ulrych & Wolfgang Glänzel & Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2022. "Measuring the excellence contribution at the journal level: an alternative to Garfield’s impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7229-7251, December.
    11. Juan Gorraiz & Ralph Reimann & Christian Gumpenberger, 2012. "Key factors and considerations in the assessment of international collaboration: a case study for Austria and six countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 417-433, May.
    12. Juan Gorraiz & Christian Gumpenberger, 2015. "A flexible bibliometric approach for the assessment of professorial appointments," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1699-1719, December.
    13. Rodrigo Costas & Sarah de Rijcke & Noortje Marres, 2021. "“Heterogeneous couplings”: Operationalizing network perspectives to study science‐society interactions through social media metrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(5), pages 595-610, May.
    14. Zhou, Ping & Zhong, Yongfeng, 2012. "The citation-based indicator and combined impact indicator—New options for measuring impact," Journal of Informetrics, Elsevier, vol. 6(4), pages 631-638.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    2. Kovac, Dejan & Scrbec, Nikol & Podobnik, Boris, 2018. "Does it payoff to research economics—A tale of citation, knowledge and economic growth in transition countries," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 505(C), pages 293-305.
    3. Tamara Krajna & Jelka Petrak, 2019. "Croatian Highly Cited Papers," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 17(3-B), pages 684-696.
    4. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    5. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2014. "Assessing national strengths and weaknesses in research fields," Journal of Informetrics, Elsevier, vol. 8(3), pages 766-775.
    6. Albarrán, Pedro & Crespo, Juan A. & Ortuño, Ignacio, 2009. "A comparison of the scientific performance of the U. S. and the European Union at the turn of the XXI century," UC3M Working papers. Economics we095534, Universidad Carlos III de Madrid. Departamento de Economía.
    7. Qiuju Zhou & Ronald Rousseau & Liying Yang & Ting Yue & Guoliang Yang, 2012. "A general framework for describing diversity within systems and similarity between systems with applications in informetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 787-812, December.
    8. Neus Herranz & Javier Ruiz-Castillo, 2013. "The end of the “European Paradox”," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 453-464, April.
    9. Gonzalez-Brambila, Claudia & Jenkins, Mauricio & Lloret, Antonio, 2016. "Challenges for scholarly business research in Latin America," Journal of Business Research, Elsevier, vol. 69(2), pages 383-387.
    10. Jonathan Adams & Karen Gurney & Louise Jackson, 2008. "Calibrating the zoom — a test of Zitt’s hypothesis," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(1), pages 81-95, April.
    11. Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2010. "A comparison of the scientific performance of the U.S. and the European union at the turn of the 21st century," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 329-344, October.
    12. Frenken, Koen & Hardeman, Sjoerd & Hoekman, Jarno, 2009. "Spatial scientometrics: Towards a cumulative research program," Journal of Informetrics, Elsevier, vol. 3(3), pages 222-232.
    13. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "A methodology to compute the territorial productivity of scientists: The case of Italy," Journal of Informetrics, Elsevier, vol. 9(4), pages 675-685.
    14. Leonardo Reyes-Gonzalez & Claudia N. Gonzalez-Brambila & Francisco Veloso, 2016. "Using co-authorship and citation analysis to identify research groups: a new way to assess performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1171-1191, September.
    15. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    16. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    17. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2020. "A novel methodology to assess the scientific standing of nations at field level," Journal of Informetrics, Elsevier, vol. 14(1).
    18. Mariani, Manuel Sebastian & Medo, Matúš & Zhang, Yi-Cheng, 2016. "Identification of milestone papers through time-balanced network centrality," Journal of Informetrics, Elsevier, vol. 10(4), pages 1207-1223.
    19. Pin-Hua Lin & Jong-Rong Chen & Chih-Hai Yang, 2014. "Academic research resources and academic quality: a cross-country analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 109-123, October.
    20. Almeida, J.A.S. & Pais, A.A.C.C. & Formosinho, S.J., 2009. "Science indicators and science patterns in Europe," Journal of Informetrics, Elsevier, vol. 3(2), pages 134-142.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:72:y:2007:i:2:d:10.1007_s11192-007-1696-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.