IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0244407.html
   My bibliography  Save this article

A science impact framework to measure impact beyond journal metrics

Author

Listed:
  • Mary D Ari
  • John Iskander
  • John Araujo
  • Christine Casey
  • John Kools
  • Bin Chen
  • Robert Swain
  • Miriam Kelly
  • Tanja Popovic

Abstract

Measuring the impact of public health science or research is important especially when it comes to health outcomes. Achieving the desired health outcomes take time and may be influenced by several contributors, making attribution of credit to any one entity or effort problematic. Here we offer a science impact framework (SIF) for tracing and linking public health science to events and/or actions with recognized impact beyond journal metrics. The SIF was modeled on the Institute of Medicine’s (IOM) Degrees of Impact Thermometer, but differs in that SIF is not incremental, not chronological, and has expanded scope. The SIF recognizes five domains of influence: disseminating science, creating awareness, catalyzing action, effecting change and shaping the future (scope differs from IOM). For public health, the goal is to achieve one or more specific health outcomes. What is unique about this framework is that the focus is not just on the projected impact or outcome but rather the effects that are occurring in real time with the recognition that the measurement field is complex, and it takes time for the ultimate outcome to occur. The SIF is flexible and can be tailored to measure the impact of any scientific effort: from complex initiatives to individual publications. The SIF may be used to measure impact prospectively of an ongoing or new body of work (e.g., research, guidelines and recommendations, or technology) and retrospectively of completed and disseminated work, through linking of events using indicators that are known and have been used for measuring impact. Additionally, linking events offers an approach to both tell our story and also acknowledge other players in the chain of events. The value added by science can easily be relayed to the scientific community, policy makers and the public.

Suggested Citation

  • Mary D Ari & John Iskander & John Araujo & Christine Casey & John Kools & Bin Chen & Robert Swain & Miriam Kelly & Tanja Popovic, 2020. "A science impact framework to measure impact beyond journal metrics," PLOS ONE, Public Library of Science, vol. 15(12), pages 1-15, December.
    • Handle: RePEc:plo:pone00:0244407
    DOI: 10.1371/journal.pone.0244407
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0244407
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0244407&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0244407?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Flanders, W.D. & O'Brien, T.R., 1989. "Inappropriate comparisons of incidence and prevalence in epidemiologic research," American Journal of Public Health, American Public Health Association, vol. 79(9), pages 1301-1303.
    2. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    3. Alston, Julian M. & Pardey, Philip G., 2001. "Attribution and other problems in assessing the returns to agricultural R&D," Agricultural Economics, Blackwell, vol. 25(2-3), pages 141-152, September.
    4. Haileyesus Getahun & Wanitchaya Kittikraisak & Charles M Heilig & Elizabeth L Corbett & Helen Ayles & Kevin P Cain & Alison D Grant & Gavin J Churchyard & Michael Kimerling & Sarita Shah & Stephen D L, 2011. "Development of a Standardized Screening Rule for Tuberculosis in People Living with HIV in Resource-Constrained Settings: Individual Participant Data Meta-analysis of Observational Studies," PLOS Medicine, Public Library of Science, vol. 8(1), pages 1-14, January.
    5. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barbara McGillivray & Mathias Astell, 2019. "The relationship between usage and citations in an open access mega-journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 817-838, November.
    2. Walters, William H., 2017. "Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?," Journal of Informetrics, Elsevier, vol. 11(3), pages 730-744.
    3. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    4. Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
    5. Huffman, Wallace E. & Evenson, Robert E., 2003. "New Econometric Evidence On Agricultural Total Factor Productivity Determinants: Impact Of Funding Sources," Working Papers 18201, Iowa State University, Department of Economics.
    6. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    7. Ana Teresa Santos & Sandro Mendonça, 2022. "Do papers (really) match journals’ “aims and scope”? A computational assessment of innovation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7449-7470, December.
    8. Davis, K. & Nkonya, E. & Kato, E. & Mekonnen, D.A. & Odendo, M. & Miiro, R. & Nkuba, J., 2012. "Impact of Farmer Field Schools on Agricultural Productivity and Poverty in East Africa," World Development, Elsevier, vol. 40(2), pages 402-413.
    9. Ortega, José Luis, 2018. "The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX," Journal of Informetrics, Elsevier, vol. 12(3), pages 579-589.
    10. Alejandro Plastina & Lilyan Fulginiti, 2012. "Rates of return to public agricultural research in 48 US states," Journal of Productivity Analysis, Springer, vol. 37(2), pages 95-113, April.
    11. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    12. Katarzyna Cheba, 2015. "The Influence Of Clusters On Economic Development. A Comparative Analysis Of Cluster Policy In The European Union And Japan," Oeconomia Copernicana, Institute of Economic Research, vol. 6(3), pages 73-88, September.
    13. Cameron Neylon & Shirley Wu, 2009. "Article-Level Metrics and the Evolution of Scientific Impact," PLOS Biology, Public Library of Science, vol. 7(11), pages 1-6, November.
    14. B Ian Hutchins & Xin Yuan & James M Anderson & George M Santangelo, 2016. "Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-25, September.
    15. Peter Weißhuhn & Katharina Helming & Johanna Ferretti, 2018. "Research impact assessment in agriculture—A review of approaches and impact areas," Research Evaluation, Oxford University Press, vol. 27(1), pages 36-42.
    16. Mullen, J. D., 2002. "Farm Management In The 21st Century," 2002 Conference (46th), February 13-15, 2002, Canberra, Australia 174072, Australian Agricultural and Resource Economics Society.
    17. Wadjamsse Djezou, 2016. "Land Tenure Security and Deforestation: A case Study of Forest land conversion to Perennial crops in Côte d'Ivoire," Economics Bulletin, AccessEcon, vol. 36(1), pages 173-186.
    18. Xie, Qing & Zhang, Xinyuan & Song, Min, 2021. "A network embedding-based scholar assessment indicator considering four facets: Research topic, author credit allocation, field-normalized journal impact, and published time," Journal of Informetrics, Elsevier, vol. 15(4).
    19. Philip G. Pardey & Julian M. Alston & Connie Chan-Kang & Eduardo C. Magalhães & Stephen A. Vosti, 2006. "International and Institutional R&D Spillovers: Attribution of Benefits among Sources for Brazil's New Crop Varieties," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 88(1), pages 104-123.
    20. Bert Lenaerts & Yann de Mey & Matty Demont, 2018. "Global impact of accelerated plant breeding: Evidence from a meta-analysis on rice breeding," PLOS ONE, Public Library of Science, vol. 13(6), pages 1-21, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0244407. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.