IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v48y2024i3p410-426.html
   My bibliography  Save this article

Transferability of Lessons From Program Evaluations: Iron Laws, Hiding Hands and the Evidence Ecosystem

Author

Listed:
  • Tom Ling

Abstract

Assessing the transferability of lessons from social research or evaluation continues to raise challenges. Efforts to identify transferable lessons can be based on two different forms of argumentation. The first draws upon statistics and causal inferences. The second involves constructing a reasoned case based on weighing up different data collected along the causal chain from designing to delivery. Both approaches benefit from designing research based upon existing evidence and ensuring that the descriptions of the programme, context, and intended beneficiaries are sufficiently rich. Identifying transferable lessons should not be thought of as a one-off event but involves contributing to the iterative and learning of a scientific community. To understand the circumstances under which findings can be confidently transferred, we need to understand: (1) How far and why outcomes of interest have multiple, interacting and fluctuating causes. (2) The program design and implementation capacity. (3) Prior knowledge and causal landscapes (and how far these are included in the theory of change). (4) New and relevant knowledge; what can we learn in our ‘disputatious community of truth seekers’.

Suggested Citation

  • Tom Ling, 2024. "Transferability of Lessons From Program Evaluations: Iron Laws, Hiding Hands and the Evidence Ecosystem," Evaluation Review, , vol. 48(3), pages 410-426, June.
    • Handle: RePEc:sae:evarev:v:48:y:2024:i:3:p:410-426
    DOI: 10.1177/0193841X241228332
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X241228332
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X241228332?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Littell, Julia H., 2005. "Lessons from a systematic review of effects of multisystemic therapy," Children and Youth Services Review, Elsevier, vol. 27(4), pages 445-463, April.
    2. Bent Flyvbjerg, 2014. "What You Should Know About Megaprojects, and Why: An Overview," Papers 1409.0003, arXiv.org.
    3. Karl E. Weick & Kathleen M. Sutcliffe & David Obstfeld, 2005. "Organizing and the Process of Sensemaking," Organization Science, INFORMS, vol. 16(4), pages 409-421, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giovanni Esposito & Andrea Terlizzi, 2023. "Governing wickedness in megaprojects: discursive and institutional perspectives," Policy and Society, Darryl S. Jarvis and M. Ramesh, vol. 42(2), pages 131-147.
    2. Diane Coyle & Marianne Sensier, 2020. "The imperial treasury: appraisal methodology and regional economic performance in the UK," Regional Studies, Taylor & Francis Journals, vol. 54(3), pages 283-295, March.
    3. Schreiner, Lena & Madlener, Reinhard, 2022. "Investing in power grid infrastructure as a flexibility option: A DSGE assessment for Germany," Energy Economics, Elsevier, vol. 107(C).
    4. Carlos Martin-Rios, 2016. "Innovative management control systems in knowledge work: a middle manager perspective," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 27(2), pages 181-204, May.
    5. Verena Brinks, 2016. "Situated affect and collective meaning: A community perspective on processes of value creation and commercialization in enthusiast-driven fields," Environment and Planning A, , vol. 48(6), pages 1152-1169, June.
    6. Stefan Gröschl & Patricia Gabaldón & Tobias Hahn, 2019. "The Co-evolution of Leaders’ Cognitive Complexity and Corporate Sustainability: The Case of the CEO of Puma," Journal of Business Ethics, Springer, vol. 155(3), pages 741-762, March.
    7. Elena Antonacopoulou, 2018. "Energising critique in action and in learning: The GNOSIS 4R Framework," Action Learning: Research and Practice, Taylor & Francis Journals, vol. 15(2), pages 102-125, May.
    8. Guiette, Alain & Vandenbempt, Koen, 2017. "Change managerialism and micro-processes of sensemaking during change implementation," Scandinavian Journal of Management, Elsevier, vol. 33(2), pages 65-81.
    9. Martina Linnenluecke & Andrew Griffiths & Peter Mumby, 2015. "Executives’ engagement with climate science and perceived need for business adaptation to climate change," Climatic Change, Springer, vol. 131(2), pages 321-333, July.
    10. Alexander Budzier & Bent Flyvbjerg & Andi Garavaglia & Andreas Leed, 2019. "Quantitative Cost and Schedule Risk Analysis of Nuclear Waste Storage," Papers 1901.11123, arXiv.org.
    11. Per Engelseth & Richard Glavee-Geo & Artur Janusz & Enoch Niboi, 2020. "The Emergent Nature of Networked Sustainable Procurement," Sustainability, MDPI, vol. 13(1), pages 1-18, December.
    12. Femke Hilverda & Margôt Kuttschreuter, 2018. "Online Information Sharing About Risks: The Case of Organic Food," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1904-1920, September.
    13. Jeffery S. McMullen & Dimo Dimov, 2013. "Time and the Entrepreneurial Journey: The Problems and Promise of Studying Entrepreneurship as a Process," Journal of Management Studies, Wiley Blackwell, vol. 50(8), pages 1481-1512, December.
    14. Emil Evenhuis, 2017. "Institutional change in cities and regions: a path dependency approach," Cambridge Journal of Regions, Economy and Society, Cambridge Political Economy Society, vol. 10(3), pages 509-526.
    15. Francesco Di Maddaloni & Roya Derakhshan, 2019. "A Leap from Negative to Positive Bond. A Step towards Project Sustainability," Administrative Sciences, MDPI, vol. 9(2), pages 1-19, June.
    16. Bent Flyvbjerg & Alexander Budzier & M. D. Christodoulou & M. Zottoli, 2024. "Uniqueness Bias: Why It Matters, How to Curb It," Papers 2408.07710, arXiv.org.
    17. Minkkinen, Matti, 2019. "The anatomy of plausible futures in policy processes: Comparing the cases of data protection and comprehensive security," Technological Forecasting and Social Change, Elsevier, vol. 143(C), pages 172-180.
    18. Stea, Diego & Foss, Nicolai J. & Christensen, Peter Holdt, 2015. "Physical separation in the workplace: Separation cues, separation awareness, and employee motivation," European Management Journal, Elsevier, vol. 33(6), pages 462-471.
    19. Heather Rosoff & Robert Siko & Richard John & William J. Burns, 2013. "Should I stay or should I go? An experimental study of health and economic government policies following a severe biological agent release," Environment Systems and Decisions, Springer, vol. 33(1), pages 121-137, March.
    20. Tiina J. Peltola & Hanna Tiirinki, 2020. "More Than Numbers: Discourses of Health Care Quality in Finland," SAGE Open, , vol. 10(4), pages 21582440209, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:48:y:2024:i:3:p:410-426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.