IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v114y2018i3d10.1007_s11192-017-2585-6.html
   My bibliography  Save this article

Assessing the health research’s social impact: a systematic review

Author

Listed:
  • Matteo Pedrini

    (Università Cattolica del Sacro Cuore)

  • Valentina Langella

    (Università Cattolica del Sacro Cuore)

  • Mario Alberto Battaglia

    (Università di Siena)

  • Paola Zaratin

    (Fondazione Italiana Sclerosi Multipla)

Abstract

In recent year, a growing attention is dedicated to the assessment of research’s social impact. While prior research has often dealt with results of research, the last decade has begun to generate knowledge on the assessment of health research’s social impact. However, this knowledge is scattered across different disciplines, research communities, and journals. Therefore, this paper analyzes the heterogeneous picture research has drawn within the past years with a focus on the health research’s social impact on different stakeholders through an interdisciplinary, systematic review. By consulting major research databases, we have analyzed 53 key journal articles bibliographically and thematically. We argued that the adoption of a multi-stakeholder could be an evolution of the existing methods used to assess impact of research. After presenting a model to assess the health research’s social impact with a multi stakeholder perspective, we suggest the implementation in the research process of three practice: a multi-stakeholder workshop on research agenda; a multi stakeholder supervisory board; a multi-stakeholder review process.

Suggested Citation

  • Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    • Handle: RePEc:spr:scient:v:114:y:2018:i:3:d:10.1007_s11192-017-2585-6
    DOI: 10.1007/s11192-017-2585-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2585-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-017-2585-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Annette Boaz & Siobhan Fitzpatrick & Ben Shaw, 2009. "Assessing the impact of research on policy: A literature review," Science and Public Policy, Oxford University Press, vol. 36(4), pages 255-270, May.
    2. Paul Nightingale & Alister Scott, 2007. "Peer review and the relevance gap: Ten suggestions for policy-makers," Science and Public Policy, Oxford University Press, vol. 34(8), pages 543-553, October.
    3. Czarnitzki, Dirk & Lopes-Bento, Cindy, 2013. "Value for money? New microeconometric evidence on public R&D grants in Flanders," Research Policy, Elsevier, vol. 42(1), pages 76-89.
    4. Fernando A. B. Colugnati & Sergio Firpo & Paula F. Drummond Castro & Juan E. Sepulveda & Sergio L. M. Salles-Filho, 2014. "A propensity score approach in the impact evaluation on scientific production in Brazilian biodiversity research: the BIOTA Program," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 85-107, October.
    5. N. Sombatsompop & T. Markpin & W. Yochai & M. Saechiew, 2005. "An evaluation of research performance for different subject categories using Impact Factor Point Average (IFPA) index: Thailand case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 65(3), pages 293-305, December.
    6. Figueredo, Aurelio Jose & Sechrest, Lee, 2001. "Approaches used in conducting health outcomes and effectiveness research," Evaluation and Program Planning, Elsevier, vol. 24(1), pages 41-59, February.
    7. J Britt Holbrook & Robert Frodeman, 2011. "Peer review and the ex ante assessment of societal impacts," Research Evaluation, Oxford University Press, vol. 20(3), pages 239-246, September.
    8. Paule Leduc, 1994. "Evaluation in the social sciences: the strategic context," Research Evaluation, Oxford University Press, vol. 4(1), pages 2-5, April.
    9. Fiorenzo Franceschini & Domenico Maisano & Luca Mastrogiacomo, 2015. "Research quality evaluation: comparing citation counts considering bibliometric database errors," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(1), pages 155-165, January.
    10. Erik Ernø-Kjølhede & Finn Hansson, 2011. "Measuring research performance during a changing relationship between science and society," Research Evaluation, Oxford University Press, vol. 20(2), pages 131-143, June.
    11. Henk F. Moed & Gali Halevi, 2015. "Multidimensional assessment of scholarly research impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 1988-2002, October.
    12. J. Guinea & E. Sela & A. J. Gómez-Núñez & T. Mangwende & A. Ambali & N. Ngum & H. Jaramillo & J. M. Gallego & A. Patiño & C. Latorre & S. Srivanichakorn & B. Thepthien, 2015. "Impact oriented monitoring: A new methodology for monitoring and evaluation of international public health research projects," Research Evaluation, Oxford University Press, vol. 24(2), pages 131-145.
    13. Bruno S. Frey & Katja Rost, 2010. "Do rankings reflect research quality?," Journal of Applied Economics, Universidad del CEMA, vol. 13, pages 1-38, May.
    14. Howard Newby, 1994. "The challenge for social science: a new role in public policy-making," Research Evaluation, Oxford University Press, vol. 4(1), pages 6-11, April.
    15. Frederic S. Lee, 2007. "The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 31(2), pages 309-325, March.
    16. Bo Göransson & Rasigan Maharajh & Ulrich Schmoch, 2009. "New activities of universities in transfer and extension: multiple requirements and manifold solutions," Science and Public Policy, Oxford University Press, vol. 36(2), pages 157-164, March.
    17. Dannenberg, A.L. & Bhatia, R. & Cole, B.L. & Dora, C. & Fielding, J.E. & Kraft, K. & McClymont-Peace, D. & Mindell, J. & Onyekere, C. & Roberts, J.A. & Ross, C.L. & Rutt, C.D. & Scott-Samuel, A. & Til, 2006. "Growing the field of health impact assessment in the United States: An agenda for research and practice," American Journal of Public Health, American Public Health Association, vol. 96(2), pages 262-270.
    18. Bianca Potì & Giovanni Cerulli, 2011. "Evaluation of firm R&D and innovation support: new indicators and the ex-ante prediction of ex-post additionality," Research Evaluation, Oxford University Press, vol. 20(1), pages 19-29, March.
    19. Katharine Barker, 2007. "The UK Research Assessment Exercise: the evolution of a national research evaluation system," Research Evaluation, Oxford University Press, vol. 16(1), pages 3-12, March.
    20. Jordi Molas-Gallart & Puay Tang, 2011. "Tracing ‘productive interactions’ to identify social impacts: an example from the social sciences," Research Evaluation, Oxford University Press, vol. 20(3), pages 219-226, September.
    21. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    22. Ekboir, Javier, 2003. "Why impact analysis should not be used for research evaluation and what the alternatives are," Agricultural Systems, Elsevier, vol. 78(2), pages 166-184, November.
    23. Lutz Bornmann & Werner Marx, 2014. "How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 211-219, January.
    24. Maredia, Mywish K. & Byerlee, Derek, 2000. "Efficiency of research investments in the presence of international spillovers: wheat research in developing countries," Agricultural Economics, Blackwell, vol. 22(1), pages 1-16, January.
    25. David Roessner, 2000. "Quantitative and qualitative methods and measures in the evaluation of research," Research Evaluation, Oxford University Press, vol. 9(2), pages 125-132, August.
    26. Claire Donovan, 2007. "The qualitative future of research evaluation," Science and Public Policy, Oxford University Press, vol. 34(8), pages 585-597, October.
    27. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    28. Martin Buxton & Steve Hanney & Tim Packwood & Simon Roberts & Penny Youll, 2000. "Getting Reearch into Practice: Assessing Benefits from Department of Health and National Health Service Research & Development," Public Money & Management, Taylor & Francis Journals, vol. 20(4), pages 29-34, October.
    29. David Pontille & Didier Torny, 2010. "The controversial policies of journal ratings: evaluating social sciences and humanities," Research Evaluation, Oxford University Press, vol. 19(5), pages 347-360, December.
    30. Christina H. Drew & Kristianna G. Pettibone & Fallis Owen Finch & Douglas Giles & Paul Jordan, 2016. "Automated Research Impact Assessment: a new bibliometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 987-1005, March.
    31. Peter van den Besselaar & Loet Leydesdorff, 2009. "Past performance, peer review and project selection: a case study in the social and behavioral sciences," Research Evaluation, Oxford University Press, vol. 18(4), pages 273-288, October.
    32. Stefan P L de Jong & Pleun van Arensbergen & Floortje Daemen & Barend van der Meulen & Peter van den Besselaar, 2011. "Evaluation of research in context: an approach and two cases," Research Evaluation, Oxford University Press, vol. 20(1), pages 61-72, March.
    33. Chris Henshall, 2011. "The impact of Payback research: developing and using evidence in policy," Research Evaluation, Oxford University Press, vol. 20(3), pages 257-258, September.
    34. Ghislaine Tremblay & Sandra Zohar & Juliana Bravo & Peter Potsepp & Meg Barker, 2010. "The Canada Foundation for Innovation's outcome measurement study: a pioneering approach to research evaluation," Research Evaluation, Oxford University Press, vol. 19(5), pages 333-345, December.
    35. John D Brewer, 2011. "The impact of impact," Research Evaluation, Oxford University Press, vol. 20(3), pages 255-256, September.
    36. Roberto Ippoliti & Greta Falavigna, 2014. "Public Health Institutions, Clinical Research and Protection System of Patients’ Rights: An Impact Evaluation of Public Policy," Public Organization Review, Springer, vol. 14(2), pages 109-125, June.
    37. Barend van der Meulen & Arie Rip, 2000. "Evaluation of societal quality of public sector research in the Netherlands," Research Evaluation, Oxford University Press, vol. 9(1), pages 11-25, April.
    38. Paula Adam & Maite Solans-Domènech & Joan M. V. Pons & Marta Aymerich & Silvina Berra & Imma Guillamon & Emilia Sánchez & Gaietà Permanyer-Miralda, 2012. "Assessment of the impact of a clinical and health services research call in Catalonia," Research Evaluation, Oxford University Press, vol. 21(4), pages 319-328, October.
    39. Ben R Martin, 2011. "The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster?," Research Evaluation, Oxford University Press, vol. 20(3), pages 247-254, September.
    40. Sarah Bell & Ben Shaw & Annette Boaz, 2011. "Real-world approaches to assessing the impact of environmental research on policy," Research Evaluation, Oxford University Press, vol. 20(3), pages 227-237, September.
    41. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    42. Luukkonen, Terttu, 1998. "The difficulties in assessing the impact of EU framework programmes," Research Policy, Elsevier, vol. 27(6), pages 599-610, September.
    43. Jack Spaapen & Leonie van Drooge, 2011. "Introducing ‘productive interactions’ in social impact assessment," Research Evaluation, Oxford University Press, vol. 20(3), pages 211-218, September.
    44. Evans, Annette & Strezov, Vladimir & Evans, Tim J., 2009. "Assessment of sustainability indicators for renewable energy technologies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(5), pages 1082-1088, June.
    45. Roula Inglesi-Lotz & Anastassios Pouris, 2011. "Scientometric impact assessment of a research policy instrument: the case of rating researchers on scientific outputs in South Africa," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 747-760, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Oskar Jonsson & Susanne Iwarsson, 2022. "Scrutinizing the collaboration criterion in research: how do policy ambitions play out in proposals and assessments?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4675-4696, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    2. Andrea Bonaccorsi & Filippo Chiarello & Gualtiero Fantoni, 2021. "Impact for whom? Mapping the users of public research with lexicon-based text mining," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1745-1774, February.
    3. Lutz Bornmann & Werner Marx, 2014. "How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 211-219, January.
    4. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    5. Paul Benneworth, 2015. "Between certainty and comprehensiveness in evaluating the societal impact of humanities research," CHEPS Working Papers 201502, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    6. Joly, P.B. & Gaunand, A. & Colinet, L. & Larédo, P. & Lemarié, S. & Matt, M., 2015. "ASIRPA: a comprehensive theory-based approach to assessing the societal impacts of a research organization," Working Papers 2015-04, Grenoble Applied Economics Laboratory (GAEL).
    7. Gaunand, A. & Hocdé, A. & Lemarié, S. & Matt, M. & Turckheim, E.de, 2015. "How does public agricultural research impact society? A characterization of various patterns," Research Policy, Elsevier, vol. 44(4), pages 849-861.
    8. Dotti, Nicola Francesco & Walczyk, Julia, 2022. "What is the societal impact of university research? A policy-oriented review to map approaches, identify monitoring methods and success factors," Evaluation and Program Planning, Elsevier, vol. 95(C).
    9. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    10. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    11. Reetta Muhonen & Paul Benneworth & Julia Olmos-Peñuela, 2018. "From productive interactions to impact pathways," CHEPS Working Papers 201802, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    12. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    13. Alessandro Margherita & Gianluca Elia & Claudio Petti, 2022. "What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives," Sustainability, MDPI, vol. 14(5), pages 1-18, March.
    14. Eva María de la Torre & Fernando Casani & Carmen Pérez Esparrells, 2021. "Measuring universities’ engagement: a revision of the European research projects and the actual use of the so-called ‘third mission’ indicators," Revista de Estudios Regionales, Universidades Públicas de Andalucía, vol. 1, pages 97-128.
    15. A. Gaunand & L. Colinet & P.-B. Joly & M. Matt, 2022. "Counting what really counts? Assessing the political impact of science," The Journal of Technology Transfer, Springer, vol. 47(3), pages 699-721, June.
    16. Irwin Feller, 2022. "Assessing the societal impact of publicly funded research," The Journal of Technology Transfer, Springer, vol. 47(3), pages 632-650, June.
    17. Pierre Benoit Joly & Laurence Colinet & Ariane Gaunand & Stephane Lemarié & Mireille Matt, 2016. "Agricultural research impact assessment: issues, methods and challenges," Working Papers hal-01431457, HAL.
    18. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    19. de Jong, Stefan P.L. & Wardenaar, Tjerk & Horlings, Edwin, 2016. "Exploring the promises of transdisciplinary research: A quantitative study of two climate research programmes," Research Policy, Elsevier, vol. 45(7), pages 1397-1409.
    20. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:114:y:2018:i:3:d:10.1007_s11192-017-2585-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.