IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v30y2021i4p458-469..html
   My bibliography  Save this article

Looking for evidence of research impact and use: A qualitative study of an Australian research-policy system
[Public Social Policy Development and Implementation: A Case Study of the Ghana National Health Insurance Scheme]

Author

Listed:
  • Robyn S. Newson
  • Lucie Rychetnik
  • Lesley King
  • Andrew J. Milat
  • Adrian E. Bauman

Abstract

Current assessments of research impact have been criticized for capturing what can be easily counted not what actually counts. To empirically examine this issue, we approached measuring research impact from two directions, tracing forwards from research and backwards from policy, within a defined research-policy system (childhood obesity prevention research and policy in New South Wales, Australia from 2000 to 2015). The forward tracing research impact assessment component traced a sample of 148 local research projects forward to examine their policy impacts. Of the projects considered, 16% had an impact on local policy and for a further 19%, decision-makers were aware of the research, but there was no evidence it influenced policy decisions. The backward tracing component of the study included an analysis of research use across three policy initiatives. It provided a more nuanced understanding of the relative influence of research on policy. Both direct uses of specific research and indirect uses of research incorporated as broader bodies of knowledge were evident. Measuring research impact from both directions captured the diverse ways that research was used in decision-making. Our findings illustrate complexities in the assessment process and in real-life policymaking trajectories. They highlight the role that timing of assessment plays in perception of impacts and difficulties attributing longer-term impacts to specific research. This study supports the use of models where politics and complex system dynamics shape knowledge and its influence on decision-making, rather than research being the primary driver for policy change.

Suggested Citation

  • Robyn S. Newson & Lucie Rychetnik & Lesley King & Andrew J. Milat & Adrian E. Bauman, 2021. "Looking for evidence of research impact and use: A qualitative study of an Australian research-policy system [Public Social Policy Development and Implementation: A Case Study of the Ghana National," Research Evaluation, Oxford University Press, vol. 30(4), pages 458-469.
    • Handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:458-469.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvab017
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shelley Bowen & Anthony B Zwi, 2005. "Pathways to “Evidence-Informed” Policy and Practice: A Framework for Action," PLOS Medicine, Public Library of Science, vol. 2(7), pages 1-1, May.
    2. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefan P L de Jong & Corina Balaban & Maria Nedeva, 2022. "From ‘productive interactions’ to ‘enabling conditions’: The role of organizations in generating societal impact of academic research [One Size Does Not Fit All! New Perspectives on the University ," Science and Public Policy, Oxford University Press, vol. 49(4), pages 643-645.
    2. Rosa Kuipers-Dirven & Matthijs Janssen & Jarno Hoekman, 2023. "Assessing university policies for enhancing societal impact of academic research: A multicriteria mapping approach," Research Evaluation, Oxford University Press, vol. 32(2), pages 371-383.
    3. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.
    4. Adrian Rauchfleisch & Mike S Schäfer & Dario Siegen, 2021. "Beyond the ivory tower: Measuring and explaining academic engagement with journalists, politicians and industry representatives among Swiss professorss," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-20, May.
    5. Ole Henning Sørensen & Jakob Bjørner & Andreas Holtermann & Johnny Dyreborg & Jorid Birkelund Sørli & Jesper Kristiansen & Steffen Bohni Nielsen, 2022. "Measuring societal impact of research—Developing and validating an impact instrument for occupational health and safety," Research Evaluation, Oxford University Press, vol. 31(1), pages 118-131.
    6. Oskar Marg & Lena Theiler, 2024. "Effects of transdisciplinary research on scientific knowledge and reflexivity," Research Evaluation, Oxford University Press, vol. 32(4), pages 635-647.
    7. Thomas, Duncan Andrew & Ramos-Vielba, Irene, 2022. "Reframing study of research(er) funding towards configurations and trails," SocArXiv uty2v, Center for Open Science.
    8. Junwen Luo & Lai Ma & Kalpana Shankar, 2021. "Does the inclusion of non-academic reviewers make any difference for grant impact panels? [Understanding the Long Term Impact of the Framework Programme, European Policy Evaluation Consortium (EPEC," Science and Public Policy, Oxford University Press, vol. 48(6), pages 763-775.
    9. Suominen, Arho & Kauppinen, Henni & Hyytinen, Kirsi, 2021. "‘Gold’, ‘Ribbon’ or ‘Puzzle’: What motivates researchers to work in Research and Technology Organizations," Technological Forecasting and Social Change, Elsevier, vol. 170(C).
    10. Zhang, Lin & Sivertsen, Gunnar & Du, Huiying & HUANG, Ying & Glänzel, Wolfgang, 2021. "Gender differences in the aims and impacts of research," SocArXiv 9n347, Center for Open Science.
    11. Wesley M. Cohen & Henry Sauermann & Paula Stephan, 2020. "Not in the Job Description: The Commercial Activities of Academic Scientists and Engineers," Management Science, INFORMS, vol. 66(9), pages 4108-4117, September.
    12. Pandey, Poonam & Pansera, Mario, 2020. "Bringing Laxmi and Saraswati together: Nano-scientists and academic entrepreneurship in India," Technology in Society, Elsevier, vol. 63(C).
    13. Gunnar Sivertsen & Ingeborg Meijer, 2020. "Normal versus extraordinary societal impact: how to understand, evaluate, and improve research activities in their relations to society?," Research Evaluation, Oxford University Press, vol. 29(1), pages 66-70.
    14. Igors Skute, 2019. "Opening the black box of academic entrepreneurship: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 237-265, July.
    15. Kwadwo Atta-Owusu & Rune Dahl Fitjar, 2022. "What motivates academics for external engagement? Exploring the effects of motivational drivers and organizational fairness [The Nature of Academic Entrepreneurship in the UK: Widening the Focus on," Science and Public Policy, Oxford University Press, vol. 49(2), pages 201-218.
    16. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    17. Joshua B Cohen, 2022. "Institutionalizing public engagement in research and innovation: Toward the construction of institutional entrepreneurial collectives [Limits of Decentered Governance in Science-society Policies]," Science and Public Policy, Oxford University Press, vol. 49(5), pages 673-685.
    18. Janzwood, Scott, 2021. "R&D priority-setting for global catastrophic risks: The case of the NASA planetary defense mission," Research Policy, Elsevier, vol. 50(6).
    19. Bastian Rake & Pablo D’Este & Maureen McKelvey, 2021. "Exploring network dynamics in science: the formation of ties to knowledge translators in clinical research," Journal of Evolutionary Economics, Springer, vol. 31(5), pages 1433-1464, November.
    20. Amara, Nabil & Olmos-Peñuela, Julia & Fernández-de-Lucio, Ignacio, 2019. "Overcoming the “lost before translation” problem: An exploratory study," Research Policy, Elsevier, vol. 48(1), pages 22-36.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:458-469.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.