IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v36y2013i1p1-14.html
   My bibliography  Save this article

Field trials of a novel toolkit for evaluating ‘intangible’ values-related dimensions of projects

Author

Listed:
  • Burford, Gemma
  • Velasco, Ismael
  • Janoušková, Svatava
  • Zahradnik, Martin
  • Hak, Tomas
  • Podger, Dimity
  • Piggot, Georgia
  • Harder, Marie K.

Abstract

A novel toolkit has been developed, using an original approach to develop its components, for the purpose of evaluating ‘soft’ outcomes and processes that have previously been generally considered ‘intangible’: those which are specifically values based. This represents a step-wise, significant, change in provision for the assessment of values-based achievements that are of absolutely key importance to most civil society organisations (CSOs) and values-based businesses, and fills a known gap in evaluation practice. In this paper, we demonstrate the significance and rigour of the toolkit by presenting an evaluation of it in three diverse scenarios where different CSOs use it to co-evaluate locally relevant outcomes and processes to obtain results which are both meaningful to them and potentially comparable across organisations. A key strength of the toolkit is its original use of a prior generated, peer-elicited ‘menu’ of values-based indicators which provides a framework for user CSOs to localise. Principles of participatory, process-based and utilisation-focused evaluation are embedded in this toolkit and shown to be critical to its success, achieving high face-validity and wide applicability. The emerging contribution of this next-generation evaluation tool to other fields, such as environmental values, development and environmental sustainable development, shared values, business, education and organisational change is outlined.

Suggested Citation

  • Burford, Gemma & Velasco, Ismael & Janoušková, Svatava & Zahradnik, Martin & Hak, Tomas & Podger, Dimity & Piggot, Georgia & Harder, Marie K., 2013. "Field trials of a novel toolkit for evaluating ‘intangible’ values-related dimensions of projects," Evaluation and Program Planning, Elsevier, vol. 36(1), pages 1-14.
    • Handle: RePEc:eee:epplan:v:36:y:2013:i:1:p:1-14
    DOI: 10.1016/j.evalprogplan.2012.04.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718912000444
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2012.04.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bouffard, Jeffrey A. & Taxman, Faye S. & Silverman, Rebecca, 2003. "Improving process evaluations of correctional programs by using a comprehensive evaluation methodology," Evaluation and Program Planning, Elsevier, vol. 26(2), pages 149-161, May.
    2. Heimlich, Joe E., 2010. "Environmental education evaluation: Reinterpreting education as a strategy for meeting mission," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 180-185, May.
    3. Odendaal, Willem A. & Marais, Sandra & Munro, Salla & van Niekerk, Ashley, 2008. "When the trivial becomes meaningful: Reflections on a process evaluation of a home visitation programme in South Africa," Evaluation and Program Planning, Elsevier, vol. 31(2), pages 209-216, May.
    4. Ebrahim, Alnoor, 2003. "Accountability In Practice: Mechanisms for NGOs," World Development, Elsevier, vol. 31(5), pages 813-829, May.
    5. Edwards, Michael & Hulme, David, 1996. "Too close for comfort? the impact of official aid on nongovernmental organizations," World Development, Elsevier, vol. 24(6), pages 961-973, June.
    6. Flowers, Alice B., 2010. "Blazing an evaluation pathway: Lessons learned from applying utilization-focused evaluation to a conservation education program," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 165-171, May.
    7. Holte-McKenzie, Merydth & Forde, Sarah & Theobald, Sally, 2006. "Development of a participatory monitoring and evaluation strategy," Evaluation and Program Planning, Elsevier, vol. 29(4), pages 365-376, November.
    8. Crohn, Kara & Birnbaum, Matthew, 2010. "Environmental education evaluation: Time to reflect, time for change," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 155-158, May.
    9. Hogard, Elaine, 2008. "Purpose and method for the evaluation of interpersonal process in health and social care," Evaluation and Program Planning, Elsevier, vol. 31(1), pages 34-40, February.
    10. Turner, R. Kerry & Paavola, Jouni & Cooper, Philip & Farber, Stephen & Jessamy, Valma & Georgiou, Stavros, 2003. "Valuing nature: lessons learned and future research directions," Ecological Economics, Elsevier, vol. 46(3), pages 493-510, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Campbell, Rebecca & Townsend, Stephanie M. & Shaw, Jessica & Karim, Nidal & Markowitz, Jenifer, 2015. "Can a workbook work? Examining whether a practitioner evaluation toolkit can promote instrumental use," Evaluation and Program Planning, Elsevier, vol. 52(C), pages 107-117.
    2. Gemma Burford & Elona Hoover & Ismael Velasco & Svatava Janoušková & Alicia Jimenez & Georgia Piggot & Dimity Podger & Marie K. Harder, 2013. "Bringing the “Missing Pillar” into Sustainable Development Goals: Towards Intersubjective Values-Based Indicators," Sustainability, MDPI, vol. 5(7), pages 1-25, July.
    3. Daigneault, Pierre-Marc, 2014. "Taking stock of four decades of quantitative research on stakeholder participation and evaluation use: A systematic map," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 171-181.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Urquía-Grande, Elena & Estébanez, Raquel Pérez & Alcaraz-Quiles, Francisco José, 2022. "Impact of Non-Profit Organizations’ Accountability: Empirical evidence from the democratic Republic of Congo," World Development Perspectives, Elsevier, vol. 28(C).
    2. Roberts, Susan M. & Jones III, John Paul & Frohling, Oliver, 2005. "NGOs and the globalization of managerialism: A research framework," World Development, Elsevier, vol. 33(11), pages 1845-1864, November.
    3. Brass, Jennifer N., 2012. "Why Do NGOs Go Where They Go? Evidence from Kenya," World Development, Elsevier, vol. 40(2), pages 387-401.
    4. Acharya, Arnab & Martínez-Álvarez, Melisa, 2012. "Aid Effectiveness in the Health Sector," WIDER Working Paper Series 069, World Institute for Development Economic Research (UNU-WIDER).
    5. Dewi, Miranti Kartika & Manochin, Melina & Belal, Ataur, 2021. "Towards a conceptual framework of beneficiary accountability by NGOs: An Indonesian case study," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 80(C).
    6. Crohn, Kara & Birnbaum, Matthew, 2010. "Environmental education evaluation: Time to reflect, time for change," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 155-158, May.
    7. Dorothea Baur & Hans Schmitz, 2012. "Corporations and NGOs: When Accountability Leads to Co-optation," Journal of Business Ethics, Springer, vol. 106(1), pages 9-21, March.
    8. Duval, Anne-Marie & Gendron, Yves & Roux-Dufort, Christophe, 2015. "Exhibiting nongovernmental organizations: Reifying the performance discourse through framing power," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 29(C), pages 31-53.
    9. Raphael Zumofen, 2016. "Public accountability: a summary analysis [Accountability publique - Une analyse synthétique]," Post-Print hal-03623871, HAL.
    10. Hyman Zyl & Frederik Claeyé, 2019. "Up and Down, and Inside Out: Where do We Stand on NGO Accountability?," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 31(3), pages 604-619, July.
    11. Banks, Nicola & Hulme, David & Edwards, Michael, 2015. "NGOs, States, and Donors Revisited: Still Too Close for Comfort?," World Development, Elsevier, vol. 66(C), pages 707-718.
    12. Brendan O'Dwyer & Jeffrey Unerman, 2010. "Enhancing the role of accountability in promoting the rights of beneficiaries of development NGOs," Accounting and Business Research, Taylor & Francis Journals, vol. 40(5), pages 451-471.
    13. Gani ALDASHEV & Cecilia NAVARRA, 2018. "Development Ngos: Basic Facts," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 89(1), pages 125-155, March.
    14. Lore Wellens & Marc Jegers, 2017. "Beneficiaries’ participation in development organizations through local partners: A case study in Southern Africa," Development Policy Review, Overseas Development Institute, vol. 35, pages 196-213, October.
    15. Taylor, Dennis & Tharapos, Meredith & Sidaway, Shannon, 2014. "Downward accountability for a natural disaster recovery effort: Evidence and issues from Australia's Black Saturday," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 25(7), pages 633-651.
    16. repec:bla:annpce:v:89:y:2018:i:1:p:125-155 is not listed on IDEAS
    17. Kopel, Michael & Marini, Marco A., 2022. "Mandatory disclosure of managerial contracts in NGOs," Journal of Economic Behavior & Organization, Elsevier, vol. 199(C), pages 65-85.
    18. Yasmin, Sofia & Ghafran, Chaudhry, 2019. "The problematics of accountability: Internal responses to external pressures in exposed organisations," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 64(C).
    19. Alisa Moldavanova & Tamaki Onishi & Stefan Toepler, 2023. "Civil society and democratization: The role of service‐providing organizations amid closing civic spaces," Public Administration & Development, Blackwell Publishing, vol. 43(1), pages 3-13, February.
    20. repec:unu:wpaper:wp2012-69 is not listed on IDEAS
    21. Seyla Rizky Amelia & Miranti Kartika Dewi, 2021. "How a nonprofit organization delivers online accountability through social media," International Review on Public and Nonprofit Marketing, Springer;International Association of Public and Non-Profit Marketing, vol. 18(3), pages 317-334, September.
    22. Michael Kopel & Marco A. Marini, 2020. "Mandatory Disclosure of Managerial Contracts in Nonprofit Organizations," Working Papers 2020.26, Fondazione Eni Enrico Mattei.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:36:y:2013:i:1:p:1-14. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.