IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v28y2019i2p123-135..html
   My bibliography  Save this article

Making a difference in the real world? A meta-analysis of the quality of use-oriented research using the Research Quality Plus approach

Author

Listed:
  • Robert K D McLean
  • Kunal Sen

Abstract

High-quality, use-oriented, and well-communicated research can improve social outcomes in low- and middle-income countries and, by doing so, accelerate development progress. We provide a meta-analysis of research supported by Canada’s International Development Research Centre. We use a large and unique data set that comprises 170 research studies undertaken over the period 2010–2015. The research examined spans multiple disciplines of the social and natural sciences and was conducted across the globe, with the majority in Africa, Asia, Latin America, the Caribbean, and the Middle East. The evaluative framework we use—Research Quality Plus, RQ+—incorporates argumentation espoused in the Leiden Manifesto. As such, this article presents a case study of doing research evaluation differently and what the results can look like for research policymakers. Our analysis suggests that contrary to conventional wisdom, there is no clear trade-off between the rigor and the utility of research and that research capacity-strengthening effort is positively correlated with the scientific merit of a project. We conclude that those located closest to a development challenge are generally those best positioned to innovate a solution. The results present novel evidence for those supporting, using, and doing research for development.

Suggested Citation

  • Robert K D McLean & Kunal Sen, 2019. "Making a difference in the real world? A meta-analysis of the quality of use-oriented research using the Research Quality Plus approach," Research Evaluation, Oxford University Press, vol. 28(2), pages 123-135.
    • Handle: RePEc:oup:rseval:v:28:y:2019:i:2:p:123-135.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvy026
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    2. Cash, David & Clark, William & Alcock, Frank & Dickson, Nancy & Eckley, Noelle & Jager, Jill, 2002. "Salience, Credibility, Legitimacy and Boundaries: Linking Research, Assessment and Decision Making," Working Paper Series rwp02-046, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stina Hansson & Merritt Polk, 2018. "Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact," Research Evaluation, Oxford University Press, vol. 27(2), pages 132-144.
    2. Stina Hansson & Merritt Polk, 2019. "Comments to Belcher et al. 2018’s critique of Hansson and Polk 2018," Research Evaluation, Oxford University Press, vol. 28(2), pages 202-205.
    3. Keith B. Matthews & Ansel Renner & Kirsty L. Blackstock & Kerry A. Waylen & Dave G. Miller & Doug H. Wardell-Johnson & Alba Juarez-Bourke & Juan Cadillo-Benalcazar & Joep F. Schyns & Mario Giampietro, 2021. "Old Wine in New Bottles: Exploiting Data from the EU’s Farm Accountancy Data Network for Pan-EU Sustainability Assessments of Agricultural Production Systems," Sustainability, MDPI, vol. 13(18), pages 1-26, September.
    4. Jonathan P. Doh & Lorraine Eden & Anne S. Tsui & Srilata Zaheer, 2023. "Developing international business scholarship for global societal impact," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 54(5), pages 757-767, July.
    5. Kristjanson, Patti & Reid, Robin & Dickson, Nancy & Clark, William C. & Vishnubhotla, Prasad & Romney, Dannie & Bezkorowajnyj, Peter & Said, Mohammed & Kaelo, Dickson & Makui, Ogeli & Nkedianye, David, 2008. "Linking International Agricultural Research Knowledge with Action for Sustainable Poverty Alleviation: What Works?," Working Paper Series rwp08-045, Harvard University, John F. Kennedy School of Government.
    6. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    7. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    8. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    9. Juha-Pekka Lauronen, 2022. "Tension in Interpretations of the Social Impact of the Social Sciences: Walking a Tightrope Between Divergent Conceptualizations of Research Utilization," SAGE Open, , vol. 12(2), pages 21582440221, April.
    10. Beatriz Barros & Ana Fernández-Zubieta & Raul Fidalgo-Merino & Francisco Triguero, 2018. "Scientific knowledge percolation process and social impact: A case study on the biotechnology and microbiology perceptions on Twitter," Science and Public Policy, Oxford University Press, vol. 45(6), pages 804-814.
    11. van der Waal, Mark B. & Feddema, Jelle J. & van de Burgwal, Linda H.M., 2023. "Mapping the broad societal impact of patents," Technovation, Elsevier, vol. 128(C).
    12. Castelnovo, Paolo & Florio, Massimo & Forte, Stefano & Rossi, Lucio & Sirtori, Emanuela, 2018. "The economic impact of technological procurement for large-scale research infrastructures: Evidence from the Large Hadron Collider at CERN," Research Policy, Elsevier, vol. 47(9), pages 1853-1867.
    13. Hallie Eakin & Victor Magaña & Joel Smith & José Moreno & José Martínez & Osvaldo Landavazo, 2007. "A stakeholder driven process to reduce vulnerability to climate change in Hermosillo, Sonora, Mexico," Mitigation and Adaptation Strategies for Global Change, Springer, vol. 12(5), pages 935-955, June.
    14. Rolf, Werner & Diehl, Katharina & Zasada, Ingo & Wiggering, Hubert, 2020. "Integrating farmland in urban green infrastructure planning. An evidence synthesis for informed policymaking," Land Use Policy, Elsevier, vol. 99(C).
    15. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    16. Zeigermann, Ulrike & Böcher, Michael, 2020. "Challenges for bridging the gap between knowledge and governance in sustainability policy – The case of OECD ‘Focal Points’ for Policy Coherence for Development," Forest Policy and Economics, Elsevier, vol. 114(C).
    17. Molly Anderson, 2015. "The role of knowledge in building food security resilience across food system domains," Journal of Environmental Studies and Sciences, Springer;Association of Environmental Studies and Sciences, vol. 5(4), pages 543-559, December.
    18. Heikki Tuomenvirta & Hilppa Gregow & Atte Harjanne & Sanna Luhtala & Antti Mäkelä & Karoliina Pilli-Sihvola & Sirkku Juhola & Mikael Hildén & Pirjo Peltonen-Sainio & Ilkka T. Miettinen & Mikko Halonen, 2019. "Identifying Policy Actions Supporting Weather-Related Risk Management and Climate Change Adaptation in Finland," Sustainability, MDPI, vol. 11(13), pages 1-15, July.
    19. John Harlow & Erik Johnston & Eric Hekler & Zoë Yeh, 2018. "Fostering Sustainability Transitions by Designing for the Convergence of Policy Windows and Transition Arenas," Sustainability, MDPI, vol. 10(9), pages 1-19, August.
    20. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:28:y:2019:i:2:p:123-135.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.