IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v28y2019i2p123-135..html
   My bibliography  Save this article

Making a difference in the real world? A meta-analysis of the quality of use-oriented research using the Research Quality Plus approach

Author

Listed:
  • Robert K D McLean
  • Kunal Sen

Abstract

High-quality, use-oriented, and well-communicated research can improve social outcomes in low- and middle-income countries and, by doing so, accelerate development progress. We provide a meta-analysis of research supported by Canada’s International Development Research Centre. We use a large and unique data set that comprises 170 research studies undertaken over the period 2010–2015. The research examined spans multiple disciplines of the social and natural sciences and was conducted across the globe, with the majority in Africa, Asia, Latin America, the Caribbean, and the Middle East. The evaluative framework we use—Research Quality Plus, RQ+—incorporates argumentation espoused in the Leiden Manifesto. As such, this article presents a case study of doing research evaluation differently and what the results can look like for research policymakers. Our analysis suggests that contrary to conventional wisdom, there is no clear trade-off between the rigor and the utility of research and that research capacity-strengthening effort is positively correlated with the scientific merit of a project. We conclude that those located closest to a development challenge are generally those best positioned to innovate a solution. The results present novel evidence for those supporting, using, and doing research for development.

Suggested Citation

  • Robert K D McLean & Kunal Sen, 2019. "Making a difference in the real world? A meta-analysis of the quality of use-oriented research using the Research Quality Plus approach," Research Evaluation, Oxford University Press, vol. 28(2), pages 123-135.
    • Handle: RePEc:oup:rseval:v:28:y:2019:i:2:p:123-135.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvy026
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    2. Cash, David & Clark, William & Alcock, Frank & Dickson, Nancy & Eckley, Noelle & Jager, Jill, 2002. "Salience, Credibility, Legitimacy and Boundaries: Linking Research, Assessment and Decision Making," Working Paper Series rwp02-046, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stina Hansson & Merritt Polk, 2019. "Comments to Belcher et al. 2018’s critique of Hansson and Polk 2018," Research Evaluation, Oxford University Press, vol. 28(2), pages 202-205.
    2. Stina Hansson & Merritt Polk, 2018. "Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact," Research Evaluation, Oxford University Press, vol. 27(2), pages 132-144.
    3. Jonathan P. Doh & Lorraine Eden & Anne S. Tsui & Srilata Zaheer, 2023. "Developing international business scholarship for global societal impact," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 54(5), pages 757-767, July.
    4. Kristjanson, Patti & Reid, Robin & Dickson, Nancy & Clark, William C. & Vishnubhotla, Prasad & Romney, Dannie & Bezkorowajnyj, Peter & Said, Mohammed & Kaelo, Dickson & Makui, Ogeli & Nkedianye, David, 2008. "Linking International Agricultural Research Knowledge with Action for Sustainable Poverty Alleviation: What Works?," Working Paper Series rwp08-045, Harvard University, John F. Kennedy School of Government.
    5. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    6. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.
    7. Beatriz Barros & Ana Fernández-Zubieta & Raul Fidalgo-Merino & Francisco Triguero, 2018. "Scientific knowledge percolation process and social impact: A case study on the biotechnology and microbiology perceptions on Twitter," Science and Public Policy, Oxford University Press, vol. 45(6), pages 804-814.
    8. van der Waal, Mark B. & Feddema, Jelle J. & van de Burgwal, Linda H.M., 2023. "Mapping the broad societal impact of patents," Technovation, Elsevier, vol. 128(C).
    9. Castelnovo, Paolo & Florio, Massimo & Forte, Stefano & Rossi, Lucio & Sirtori, Emanuela, 2018. "The economic impact of technological procurement for large-scale research infrastructures: Evidence from the Large Hadron Collider at CERN," Research Policy, Elsevier, vol. 47(9), pages 1853-1867.
    10. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    11. Zeigermann, Ulrike & Böcher, Michael, 2020. "Challenges for bridging the gap between knowledge and governance in sustainability policy – The case of OECD ‘Focal Points’ for Policy Coherence for Development," Forest Policy and Economics, Elsevier, vol. 114(C).
    12. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.
    13. Bullock Graham, 2015. "Signaling the credibility of private actors as public agents: transparency, independence, and expertise in environmental evaluations of products and companies," Business and Politics, De Gruyter, vol. 17(2), pages 177-219, August.
    14. Gillian L. Galford & Julie Nash & Alan K. Betts & Sam Carlson & Sarah Ford & Ann Hoogenboom & Deborah Markowitz & Andrew Nash & Elizabeth Palchak & Sarah Pears & Kristen L. Underwood, 2016. "Bridging the climate information gap: a framework for engaging knowledge brokers and decision makers in state climate assessments," Climatic Change, Springer, vol. 138(3), pages 383-395, October.
    15. Dragana Bojovic & Andria Nicodemou & Asun Lera St.Clair & Isadora Christel & Francisco J. Doblas-Reyes, 2022. "Exploring the landscape of seasonal forecast provision by Global Producing Centres," Climatic Change, Springer, vol. 172(1), pages 1-23, May.
    16. Sébastien Chailleux, 2020. "Making the subsurface political: How enhanced oil recovery techniques reshaped the energy transition," Environment and Planning C, , vol. 38(4), pages 733-750, June.
    17. Shahzad, Murtuza & Alhoori, Hamed & Freedman, Reva & Rahman, Shaikh Abdul, 2022. "Quantifying the online long-term interest in research," Journal of Informetrics, Elsevier, vol. 16(2).
    18. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    19. Alba Viana Lora & Marta Gemma Nel-lo Andreu, 2020. "Alternative Metrics for Assessing the Social Impact of Tourism Research," Sustainability, MDPI, vol. 12(10), pages 1-12, May.
    20. Ole Henning Sørensen & Jakob Bjørner & Andreas Holtermann & Johnny Dyreborg & Jorid Birkelund Sørli & Jesper Kristiansen & Steffen Bohni Nielsen, 2022. "Measuring societal impact of research—Developing and validating an impact instrument for occupational health and safety," Research Evaluation, Oxford University Press, vol. 31(1), pages 118-131.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:28:y:2019:i:2:p:123-135.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.