IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v48y2024i3p410-426.html
   My bibliography  Save this article

Transferability of Lessons From Program Evaluations: Iron Laws, Hiding Hands and the Evidence Ecosystem

Author

Listed:
  • Tom Ling

Abstract

Assessing the transferability of lessons from social research or evaluation continues to raise challenges. Efforts to identify transferable lessons can be based on two different forms of argumentation. The first draws upon statistics and causal inferences. The second involves constructing a reasoned case based on weighing up different data collected along the causal chain from designing to delivery. Both approaches benefit from designing research based upon existing evidence and ensuring that the descriptions of the programme, context, and intended beneficiaries are sufficiently rich. Identifying transferable lessons should not be thought of as a one-off event but involves contributing to the iterative and learning of a scientific community. To understand the circumstances under which findings can be confidently transferred, we need to understand: (1) How far and why outcomes of interest have multiple, interacting and fluctuating causes. (2) The program design and implementation capacity. (3) Prior knowledge and causal landscapes (and how far these are included in the theory of change). (4) New and relevant knowledge; what can we learn in our ‘disputatious community of truth seekers’.

Suggested Citation

  • Tom Ling, 2024. "Transferability of Lessons From Program Evaluations: Iron Laws, Hiding Hands and the Evidence Ecosystem," Evaluation Review, , vol. 48(3), pages 410-426, June.
    • Handle: RePEc:sae:evarev:v:48:y:2024:i:3:p:410-426
    DOI: 10.1177/0193841X241228332
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X241228332
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X241228332?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Littell, Julia H., 2005. "Lessons from a systematic review of effects of multisystemic therapy," Children and Youth Services Review, Elsevier, vol. 27(4), pages 445-463, April.
    2. Bent Flyvbjerg, 2014. "What You Should Know About Megaprojects, and Why: An Overview," Papers 1409.0003, arXiv.org.
    3. Karl E. Weick & Kathleen M. Sutcliffe & David Obstfeld, 2005. "Organizing and the Process of Sensemaking," Organization Science, INFORMS, vol. 16(4), pages 409-421, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giovanni Esposito & Andrea Terlizzi, 2023. "Governing wickedness in megaprojects: discursive and institutional perspectives," Policy and Society, Darryl S. Jarvis and M. Ramesh, vol. 42(2), pages 131-147.
    2. Diane Coyle & Marianne Sensier, 2020. "The imperial treasury: appraisal methodology and regional economic performance in the UK," Regional Studies, Taylor & Francis Journals, vol. 54(3), pages 283-295, March.
    3. Carlos Martin-Rios, 2016. "Innovative management control systems in knowledge work: a middle manager perspective," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 27(2), pages 181-204, May.
    4. Elena Antonacopoulou, 2018. "Energising critique in action and in learning: The GNOSIS 4R Framework," Action Learning: Research and Practice, Taylor & Francis Journals, vol. 15(2), pages 102-125, May.
    5. Guiette, Alain & Vandenbempt, Koen, 2017. "Change managerialism and micro-processes of sensemaking during change implementation," Scandinavian Journal of Management, Elsevier, vol. 33(2), pages 65-81.
    6. Martina Linnenluecke & Andrew Griffiths & Peter Mumby, 2015. "Executives’ engagement with climate science and perceived need for business adaptation to climate change," Climatic Change, Springer, vol. 131(2), pages 321-333, July.
    7. Alexander Budzier & Bent Flyvbjerg & Andi Garavaglia & Andreas Leed, 2019. "Quantitative Cost and Schedule Risk Analysis of Nuclear Waste Storage," Papers 1901.11123, arXiv.org.
    8. Jeffery S. McMullen & Dimo Dimov, 2013. "Time and the Entrepreneurial Journey: The Problems and Promise of Studying Entrepreneurship as a Process," Journal of Management Studies, Wiley Blackwell, vol. 50(8), pages 1481-1512, December.
    9. Emil Evenhuis, 2017. "Institutional change in cities and regions: a path dependency approach," Cambridge Journal of Regions, Economy and Society, Cambridge Political Economy Society, vol. 10(3), pages 509-526.
    10. Bent Flyvbjerg & Alexander Budzier & M. D. Christodoulou & M. Zottoli, 2024. "Uniqueness Bias: Why It Matters, How to Curb It," Papers 2408.07710, arXiv.org.
    11. Stea, Diego & Foss, Nicolai J. & Christensen, Peter Holdt, 2015. "Physical separation in the workplace: Separation cues, separation awareness, and employee motivation," European Management Journal, Elsevier, vol. 33(6), pages 462-471.
    12. Tiina J. Peltola & Hanna Tiirinki, 2020. "More Than Numbers: Discourses of Health Care Quality in Finland," SAGE Open, , vol. 10(4), pages 21582440209, December.
    13. Rydén, Pernille & Ringberg, Torsten & Wilke, Ricky, 2015. "How Managers' Shared Mental Models of Business–Customer Interactions Create Different Sensemaking of Social Media," Journal of Interactive Marketing, Elsevier, vol. 31(C), pages 1-16.
    14. Amal Aouadi & Sylvain Marsat, 2018. "Do ESG Controversies Matter for Firm Value? Evidence from International Data," Journal of Business Ethics, Springer, vol. 151(4), pages 1027-1047, September.
    15. Love, Peter E.D. & Ika, Lavagnon A. & Ahiaga-Dagbui, Dominic D., 2019. "On de-bunking ‘fake news’ in a post truth era: Why does the Planning Fallacy explanation for cost overruns fall short?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 126(C), pages 397-408.
    16. Joseph McManus, 2021. "Emotions and Ethical Decision Making at Work: Organizational Norms, Emotional Dogs, and the Rational Tales They Tell Themselves and Others," Journal of Business Ethics, Springer, vol. 169(1), pages 153-168, February.
    17. Tongyu Meng & Jamie Newth & Christine Woods, 2022. "Ethical Sensemaking in Impact Investing: Reasons and Motives in the Chinese Renewable Energy Sector," Journal of Business Ethics, Springer, vol. 179(4), pages 1091-1117, September.
    18. Shutian Zhou & Guofang Zhai & Yijun Shi, 2018. "What Drives the Rise of Metro Developments in China? Evidence from Nantong," Sustainability, MDPI, vol. 10(8), pages 1-20, August.
    19. Sandra Waddock, 2019. "Shaping the Shift: Shamanic Leadership, Memes, and Transformation," Journal of Business Ethics, Springer, vol. 155(4), pages 931-939, April.
    20. Elfenbein, Hillary Anger, 2007. "Emotion in Organizations: A Review in Stages," Institute for Research on Labor and Employment, Working Paper Series qt2bn0n9mv, Institute of Industrial Relations, UC Berkeley.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:48:y:2024:i:3:p:410-426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.