IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v7y2020i1d10.1057_s41599-020-00575-y.html
   My bibliography  Save this article

Explaining use and non-use of policy evaluations in a mature evaluation setting

Author

Listed:
  • Valérie Pattyn

    (KU Leuven Public Governance Institute)

  • Marjolein Bouterse

    (Regioplan Beleidsonderzoek)

Abstract

Governments worldwide have intensified their efforts to institutionalize policy evaluation. Still, also in organizations with high evaluation maturity, the use of evaluations is not self-evident. As mature organizations already meet many of the factors that are commonly seen to foster evaluation use, they constitute an interesting research setting to identify (combinations of) factors that can make a key difference in minimizing research waste. In this article, we present an analysis of the use of evaluations conducted between 2013 and 2016 by the Policy and Operations Evaluation Department (IOB) of the Dutch Ministry of Foreign Affairs, a typical case of relatively high evaluation maturity. Methodologically, we rely on Qualitative Comparative Analysis as an approach that is excellently suited to capture the causal complexity characterizing evaluation use. The analysis provides useful insights on the link between knowledge production and use. We highlight the relevance of engaging policy makers in developing the evaluation design, and fine-tune available evidence as to what is perceived a good timing to organize evaluations. Contrary to existing research, we show that the political salience of an evaluation does not matter much.

Suggested Citation

  • Valérie Pattyn & Marjolein Bouterse, 2020. "Explaining use and non-use of policy evaluations in a mature evaluation setting," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-9, December.
  • Handle: RePEc:pal:palcom:v:7:y:2020:i:1:d:10.1057_s41599-020-00575-y
    DOI: 10.1057/s41599-020-00575-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-020-00575-y
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1057/s41599-020-00575-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Greene, Jennifer C., 1988. "Communication of results and utilization in participatory program evaluation," Evaluation and Program Planning, Elsevier, vol. 11(4), pages 341-351, January.
    2. Ragin, Charles C., 2000. "Fuzzy-Set Social Science," University of Chicago Press Economics Books, University of Chicago Press, edition 1, number 9780226702773, December.
    3. Kathryn Oliver & Annette Boaz, 2019. "Transforming evidence for policy and practice: creating space for new conversations," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-10, December.
    4. Holvoet, Nathalie & Van Esbroeck, Dirk & Inberg, Liesbeth & Popelier, Lisa & Peeters, Bob & Verhofstadt, Ellen, 2018. "To evaluate or not: Evaluability study of 40 interventions of Belgian development cooperation," Evaluation and Program Planning, Elsevier, vol. 67(C), pages 189-199.
    5. Rockwell, S. Kay & Dickey, Elbert C. & Jasa, Paul J., 1990. "The personal factor in evaluation use : A case study of a steering committee's use of a conservation tillage survey," Evaluation and Program Planning, Elsevier, vol. 13(4), pages 389-394, January.
    6. Turnbull, B., 1999. "The mediating effect of participation efficacy on evaluation use," Evaluation and Program Planning, Elsevier, vol. 22(2), pages 131-140, May.
    7. Hodges, Sharon P. & Hernandez, Mario, 1999. "How organizational culture influences outcome information utilization," Evaluation and Program Planning, Elsevier, vol. 22(2), pages 183-197, May.
    8. repec:ucp:bkecon:9780226702766 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wouter Lammers & Valérie Pattyn & Sacha Ferrari & Sylvia Wenmackers & Steven Van de Walle, 2024. "Evidence for policy-makers: A matter of timing and certainty?," Policy Sciences, Springer;Society of Policy Sciences, vol. 57(1), pages 171-191, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brousselle, Astrid & Petit, Geneviève & Giraud, Marie-Josée & Rietmann, Michèle & Boisvert, Krystel & Foley, Véronique, 2016. "Using the evaluation process as a lever for improving health and healthcare accessibility: The case of HCV services organization in Quebec," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 134-143.
    2. Valérie Pattyn, 2019. "Towards Appropriate Impact Evaluation Methods," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 31(2), pages 174-179, April.
    3. Cheng, Cheng-Feng & Chang, Man-Ling & Li, Chu-Shiu, 2013. "Configural paths to successful product innovation," Journal of Business Research, Elsevier, vol. 66(12), pages 2561-2573.
    4. Mina Baliamoune-Lutz, 2004. "On the Measurement of Human Well-being: Fuzzy Set Theory and Sen's Capability Approach," WIDER Working Paper Series RP2004-16, World Institute for Development Economic Research (UNU-WIDER).
    5. Arts, Bas & de Koning, Jessica, 2017. "Community Forest Management: An Assessment and Explanation of its Performance Through QCA," World Development, Elsevier, vol. 96(C), pages 315-325.
    6. Grohs, Reinhard & Raies, Karine & Koll, Oliver & Mühlbacher, Hans, 2016. "One pie, many recipes: Alternative paths to high brand strength," Journal of Business Research, Elsevier, vol. 69(6), pages 2244-2251.
    7. Jantunen, Ari & Tarkiainen, Anssi & Chari, Simos & Oghazi, Pejvak, 2018. "Dynamic capabilities, operational changes, and performance outcomes in the media industry," Journal of Business Research, Elsevier, vol. 89(C), pages 251-257.
    8. Barry Cooper & Judith Glaesser, 2016. "Analysing necessity and sufficiency with Qualitative Comparative Analysis: how do results vary as case weights change?," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(1), pages 327-346, January.
    9. Kornelakis, Andreas & Petrakaki, Dimitra, 2024. "Technological innovation, industry platforms or financialization? A comparative institutional perspective on Nokia, Apple, and Samsung," LSE Research Online Documents on Economics 124386, London School of Economics and Political Science, LSE Library.
    10. Gonçalves, Helena Martins & Lourenço, Tiago Ferreira & Silva, Graça Miranda, 2016. "Green buying behavior and the theory of consumption values: A fuzzy-set approach," Journal of Business Research, Elsevier, vol. 69(4), pages 1484-1491.
    11. Russo, Ivan & Confente, Ilenia & Gligor, David M. & Autry, Chad W., 2016. "To be or not to be (loyal): Is there a recipe for customer loyalty in the B2B context?," Journal of Business Research, Elsevier, vol. 69(2), pages 888-896.
    12. Jonathan Breckon, 2022. "Communicating and using systematic reviews—Learning from other disciplines," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(4), December.
    13. Bank, André & Richter, Thomas & Sunik, Anna, 2013. "Long-Term Monarchical Survival in the Middle East: A Configurational Comparison, 1945-2012," GIGA Working Papers 215, GIGA German Institute of Global and Area Studies.
    14. Federica Nieri & Luciano Ciravegna & Ruth V. Aguilera & Elisa Giuliani, 2019. "Larger, more internationalized, better behaved? A configurational study of em erging market multinational enterprises' involvement in corporate wrongdoing," Discussion Papers 2019/255, Dipartimento di Economia e Management (DEM), University of Pisa, Pisa, Italy.
    15. Ilenia Confente & Ivan Russo, 2018. "Qualitative Comparative Analysis (QCA): A useful methodological tool for research in the social sciences. An example from the online word-of-mouth context," MERCATI & COMPETITIVIT?, FrancoAngeli Editore, vol. 2018(4), pages 87-108.
    16. Ayala, Luis & Bárcena-Martín, Elena & Cantó, Olga & Navarro, Carolina, 2022. "COVID-19 lockdown and housing deprivation across European countries," Social Science & Medicine, Elsevier, vol. 298(C).
    17. Glückstad, Fumiko Kano & Schmidt, Mikkel N. & Mørup, Morten, 2020. "Testing a model of destination image formation: Application of Bayesian relational modelling and fsQCA," Journal of Business Research, Elsevier, vol. 120(C), pages 351-363.
    18. Chollet, Barthélemy & Géraudel, Mickaël & Khedhaouria, Anis & Mothe, Caroline, 2016. "Market knowledge as a function of CEOs' personality: A fuzzy set approach," Journal of Business Research, Elsevier, vol. 69(7), pages 2567-2573.
    19. Ferguson, Graham & Megehee, Carol M. & Woodside, Arch G., 2017. "Culture, religiosity, and economic configural models explaining tipping-behavior prevalence across nations," Tourism Management, Elsevier, vol. 62(C), pages 218-233.
    20. Borozan, Dj, 2022. "Detecting a structure in the European energy transition policy instrument mix: What mix successfully drives the energy transition?," Renewable and Sustainable Energy Reviews, Elsevier, vol. 165(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:7:y:2020:i:1:d:10.1057_s41599-020-00575-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.