IDEAS home Printed from https://ideas.repec.org/a/gam/jijerp/v14y2017i9p1007-d110727.html
   My bibliography  Save this article

Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda

Author

Listed:
  • Dimitri Renmans

    (Institute of Development Policy and Management (IOB), University of Antwerp, Antwerp, 2000, Belgium
    Department of Public Health, Institute of Tropical Medicine, Antwerp, 2000, Belgium)

  • Nathalie Holvoet

    (Institute of Development Policy and Management (IOB), University of Antwerp, Antwerp, 2000, Belgium)

  • Bart Criel

    (Department of Public Health, Institute of Tropical Medicine, Antwerp, 2000, Belgium)

Abstract

Increased attention on “complexity” in health systems evaluation has resulted in many different methodological responses. Theory-driven evaluations and systems thinking are two such responses that aim for better understanding of the mechanisms underlying given outcomes. Here, we studied the implementation of a performance-based financing intervention by the Belgian Technical Cooperation in Western Uganda to illustrate a methodological strategy of combining these two approaches. We utilized a systems dynamics tool called causal loop diagramming (CLD) to generate hypotheses feeding into a theory-driven evaluation. Semi-structured interviews were conducted with 30 health workers from two districts (Kasese and Kyenjojo) and with 16 key informants. After CLD, we identified three relevant hypotheses: “success to the successful”, “growth and underinvestment”, and “supervision conundrum”. The first hypothesis leads to increasing improvements in performance, as better performance leads to more incentives, which in turn leads to better performance. The latter two hypotheses point to potential bottlenecks. Thus, the proposed methodological strategy was a useful tool for identifying hypotheses that can inform a theory-driven evaluation. The hypotheses are represented in a comprehensible way while highlighting the underlying assumptions, and are more easily falsifiable than hypotheses identified without using CLD.

Suggested Citation

  • Dimitri Renmans & Nathalie Holvoet & Bart Criel, 2017. "Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda," IJERPH, MDPI, vol. 14(9), pages 1-20, September.
  • Handle: RePEc:gam:jijerp:v:14:y:2017:i:9:p:1007-:d:110727
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1660-4601/14/9/1007/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1660-4601/14/9/1007/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Franco, Lynne Miller & Bennett, Sara & Kanfer, Ruth, 2002. "Health sector reform and public sector health worker motivation: a conceptual framework," Social Science & Medicine, Elsevier, vol. 54(8), pages 1255-1266, April.
    2. Mayumana, Iddy & Borghi, Jo & Anselmi, Laura & Mamdani, Masuma & Lange, Siri, 2017. "Effects of Payment for Performance on accountability mechanisms: Evidence from Pwani, Tanzania," Social Science & Medicine, Elsevier, vol. 179(C), pages 61-73.
    3. Sarriot, Eric & Morrow, Melanie & Langston, Anne & Weiss, Jennifer & Landegger, Justine & Tsuma, Laban, 2015. "A causal loop analysis of the sustainability of integrated community case management in Rwanda," Social Science & Medicine, Elsevier, vol. 131(C), pages 147-155.
    4. Government of India, 2017. "National Health Policy 2017," Working Papers id:11664, eSocialSciences.
    5. Lannes, Laurence, 2015. "Improving health worker performance: The patient-perspective from a PBF program in Rwanda," Social Science & Medicine, Elsevier, vol. 138(C), pages 1-11.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Singh, Neha S. & Kovacs, Roxanne J. & Cassidy, Rachel & Kristensen, Søren R. & Borghi, Josephine & Brown, Garrett W., 2021. "A realist review to assess for whom, under what conditions and how pay for performance programmes work in low- and middle-income countries," Social Science & Medicine, Elsevier, vol. 270(C).
    2. Tajul Masron & Mduduzi Biyase & Talent Zwane & Thomas Udimal & Frederich Kirsten, 2023. "Ecological footprint and population health outcomes: an analysis of E7 countries," Economics Working Papers edwrg-07-2023, College of Business and Economics, University of Johannesburg, South Africa, revised 2023.
    3. G. Kent Fellows & Daniel J. Dutton & Aidan Hollis, 2018. "Making Sure Orphan Drugs Don’t Get Left Behind," SPP Communique, The School of Public Policy, University of Calgary, vol. 10(6), August.
    4. Shawhan, Daniel L. & Picciano, Paul D., 2019. "Costs and benefits of saving unprofitable generators: A simulation case study for US coal and nuclear power plants," Energy Policy, Elsevier, vol. 124(C), pages 383-400.
    5. Christopher H. Herbst & Monique Vledder & Karen Campbell & Mirja Sjöblom & Agnes Soucat, 2011. "The Human Resources for Health Crisis in Zambia : An Outcome of Health Worker Entry, Exit, and Performance within the National Health Labor Market," World Bank Publications - Books, The World Bank Group, number 5938.
    6. Richard Isralowitz & Mor Yehudai & Daichi Sugawara & Akihiro Masuyama & Shai-li Romem Porat & Adi Dagan & Alexander Reznik, 2022. "Economic Impact on Health and Well-Being: Comparative Study of Israeli and Japanese University “Help” Profession Students," Social Sciences, MDPI, vol. 11(12), pages 1-11, November.
    7. Shu Yan & Lizi Pan & Yan Lu & Juan Chen & Ting Zhang & Dongzi Xu & Zhaolian Ouyang, 2023. "Towards Sustainable Drug Supply in China: A Bibliometric Analysis of Drug Reform Policies," Sustainability, MDPI, vol. 15(13), pages 1-20, June.
    8. Liang-Chung Huang & Wu-Fu Chung & Shih-Wei Liu & Jau-Ching Wu & Li-Fu Chen & Yu-Chun Chen, 2019. "Characteristics of Non-Emergent Visits in Emergency Departments: Profiles and Longitudinal Pattern Changes in Taiwan, 2000–2010," IJERPH, MDPI, vol. 16(11), pages 1-16, June.
    9. Bozena Wielgoszewska & Alex Bryson & Monica Costa-Dias & Francesca Foliano & Heather Joshi & David Wilkinson, 2021. "Exploring the Reasons for Labour Market Gender Inequality a Year into the Covid-19 Pandemic: Evidence from the UK Cohort Studies," DoQSS Working Papers 21-23, Quantitative Social Science - UCL Social Research Institute, University College London.
    10. Shi, Wunan & Wouters, Olivier J. & Liu, Gordon & Mossialos, Elias & Yang, Xiuyun, 2020. "Association between provincial income levels and drug prices in China over the period 2010–2017," Social Science & Medicine, Elsevier, vol. 263(C).
    11. Anshul Kastor & Sanjay K Mohanty, 2018. "Disease-specific out-of-pocket and catastrophic health expenditure on hospitalization in India: Do Indian households face distress health financing?," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-18, May.
    12. Lewis, Maureen & Pettersson, Gunilla, 2009. "Governance in health care delivery : raising performance," Policy Research Working Paper Series 5074, The World Bank.
    13. Francesco Bogliacino & Rafael Charris & Camilo Gómez & Felipe Montealegre & Cristiano Codagnone, 2021. "Expert endorsement and the legitimacy of public policy. Evidence from Covid19 mitigation strategies," Journal of Risk Research, Taylor & Francis Journals, vol. 24(3-4), pages 394-415, April.
    14. Katie Adamek & Sarah Bardin & So O'Neil & Dara Lee Luca, "undated". "Accelerating Teen Pregnancy Prevention in Phillips County, Arkansas and Coahoma County, Mississippi," Mathematica Policy Research Reports 7fda86cbb1bf4303a84f466e8, Mathematica Policy Research.
    15. Umeobi, Ngozi Christy & Akam, Godwin Uche & Okeke, M. C, 2023. "Financial Resources And Performance Of Manufacturing Firms In The South-East, Nigeria," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(1), pages 629-641, January.
    16. Miho Sato & Deogratias Maufi & Upendo John Mwingira & Melkidezek T Leshabari & Mayumi Ohnishi & Sumihisa Honda, 2017. "Measuring three aspects of motivation among health workers at primary level health facilities in rural Tanzania," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-17, May.
    17. Kellen J. Karimi & Aijaz Ahmad & Adriano Duse & Mutuku Mwanthi & Richard Ayah, 2022. "Prevalence of Antibiotic Use and Disposal at Household Level in Informal Settlements of Kisumu, Kenya," IJERPH, MDPI, vol. 20(1), pages 1-12, December.
    18. ADEBOLA, Olukemi Grace, 2020. "Universal Health Coverage In Nigeria And Its Determinants: The Case Of National Health Insurance Scheme," Academic Review of Humanities and Social Sciences, Bursa Teknik Üniversitesi, vol. 3(1), pages 97-111.
    19. Signorelli, C. & Odone, A. & Oradini-Alacreu, A. & Pelissero, G., 2020. "Universal Health Coverage in Italy: lights and shades of the Italian National Health Service which celebrated its 40th anniversary," Health Policy, Elsevier, vol. 124(1), pages 69-74.
    20. Pankaj Bahuguna & Indranil Mukhopadhyay & Akashdeep Singh Chauhan & Saroj Kumar Rana & Sakthivel Selvaraj & Shankar Prinja, 2018. "Sub-national health accounts: Experience from Punjab State in India," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-17, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijerp:v:14:y:2017:i:9:p:1007-:d:110727. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.