IDEAS home Printed from https://ideas.repec.org/a/kap/policy/v51y2018i2d10.1007_s11077-017-9276-2.html
   My bibliography  Save this article

Experimentation at the interface of science and policy: a multi-case analysis of how policy experiments influence political decision-makers

Author

Listed:
  • Belinda McFadgen

    (VU University Amsterdam)

  • Dave Huitema

    (VU University Amsterdam
    Netherlands Open University)

Abstract

For decades now, scholars have grappled with questions about how knowledge producers can enhance the influence of their knowledge on users and improve policy making. However, little attention has been paid to how policy experiments, a flexible and ex ante method of policy appraisal, obtain influence over political decision-making. To address this gap, an exploratory framework has been developed that facilitates systematic analysis of multiple experiments, allowing hypotheses to be tested regarding how an experiment’s institutional design can influence the views of political decision-makers. Cash’s categories of effectiveness are used to describe an experiment’s conceptual influence; being how credible, salient, and legitimate decision-makers perceive an experiment to be. The hypotheses are tested using 14 experiment cases found relevant to climate adaptation in the Netherlands, with complete survey responses from over 70 respondents. The results show that although, in general, the experiments had medium to high influence on decision-makers, institutional design does have a noticeable impact. Organisers should make choices carefully when designing an experiment, particularly in order to maintain relevance during an experiment’s implementation and to build community acceptance. Suggestions for future research include a comparison of experiment effects with the effects of non-experimental forms of appraisal, such as piloting or ex ante impact assessment.

Suggested Citation

  • Belinda McFadgen & Dave Huitema, 2018. "Experimentation at the interface of science and policy: a multi-case analysis of how policy experiments influence political decision-makers," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(2), pages 161-187, June.
    • Handle: RePEc:kap:policy:v:51:y:2018:i:2:d:10.1007_s11077-017-9276-2
    DOI: 10.1007/s11077-017-9276-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11077-017-9276-2
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11077-017-9276-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Christopher Hood, 2007. "What happens when transparency meets blame-avoidance?," Public Management Review, Taylor & Francis Journals, vol. 9(2), pages 191-210, June.
    2. Peter Teirlinck & Henri Delanghe & Pierre Padilla & Arnold Verbeek, 2012. "Closing the policy cycle: Increasing the utilization of evaluation findings in research, technological development and innovation policy design," Science and Public Policy, Oxford University Press, vol. 40(3), pages 366-377, December.
    3. Susan Owens & Tim Rayner & Olivia Bina, 2004. "New Agendas for Appraisal: Reflections on Theory, Practice, and Research," Environment and Planning A, , vol. 36(11), pages 1943-1959, November.
    4. Ian Sanderson, 2009. "Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning," Political Studies, Political Studies Association, vol. 57(4), pages 699-719, December.
    5. Ian Sanderson, 2009. "Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning," Political Studies, Political Studies Association, vol. 57, pages 699-719, December.
    6. Ansell, Christopher K. & Bartenberger, Martin, 2016. "Varieties of experimentalism," Ecological Economics, Elsevier, vol. 130(C), pages 64-73.
    7. Simo Sarkki & Jari Niemelä & Rob Tinch & Sybille van den Hove & Allan Watt & Juliette Young, 2014. "Balancing credibility, relevance and legitimacy: A critical assessment of trade-offs in science–policy interfaces," Science and Public Policy, Oxford University Press, vol. 41(2), pages 194-206.
    8. Thomas Koetz & Katharine Farrell & Peter Bridgewater, 2012. "Building better science-policy interfaces for international environmental governance: assessing potential within the Intergovernmental Platform for Biodiversity and Ecosystem Services," International Environmental Agreements: Politics, Law and Economics, Springer, vol. 12(1), pages 1-21, March.
    9. Gregory Tassey, 2014. "Innovation in innovation policy management: The Experimental Technology Incentives Program and the policy experiment," Science and Public Policy, Oxford University Press, vol. 41(4), pages 419-424.
    10. Yuval Millo & Javier Lezaun, 2006. "Regulatory experiments: genetically modified crops and financial derivatives on trial," Science and Public Policy, Oxford University Press, vol. 33(3), pages 179-190, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paula Kivimaa & Karoline S. Rogge, 2020. "Interplay of Policy Experimentation and Institutional Change in Transformative Policy Mixes: The Case of Mobility as a Service in Finland," SPRU Working Paper Series 2020-17, SPRU - Science Policy Research Unit, University of Sussex Business School.
    2. Hyensup Shim & Kiyoon Shin, 2021. "Empirical Analysis of Evidence-Based Policymaking in R&D Programmes," Sustainability, MDPI, vol. 14(1), pages 1-15, December.
    3. Dave Huitema & Andrew Jordan & Stefania Munaretto & Mikael Hildén, 2018. "Policy experimentation: core concepts, political dynamics, governance and impacts," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(2), pages 143-159, June.
    4. Haupt, Wolfgang & Eckersley, Peter & Kern, Kristine, 2021. "Transfer und Skalierung von lokaler Klimapolitik: Konzeptionelle Ansätze, Voraussetzungen und Potenziale," IRS Dialog 1/2021, Leibniz Institute for Research on Society and Space (IRS).
    5. Bauknecht, Dierk & Bischoff, Thore & Bizer, Kilian & Heyen, Dirk Arne & Führ, Martin & Gailhofer, Peter & Proeger, Till & von der Leyen, Kaja, 2019. "Exploring the pathways: Regulatory experiments for Sustainable Development - An interdisciplinary approach," ifh Working Papers 22/2019, Volkswirtschaftliches Institut für Mittelstand und Handwerk an der Universität Göttingen (ifh).
    6. Yihang Zhao & Yuanyuan Zhang & Shengyu Wang, 2024. "Can a Policy Mix Achieve a Collaborative Effect? Exploring the Nested Implementation Process of Urban Carbon Emission Reduction Policies," Sustainability, MDPI, vol. 16(15), pages 1-19, July.
    7. Alex Jingwei He & Yumeng Fan & Rui Su, 2022. "Seeking policy solutions in a complex system: experimentalist governance in China’s healthcare reform," Policy Sciences, Springer;Society of Policy Sciences, vol. 55(4), pages 755-776, December.
    8. Kivimaa, Paula & Rogge, Karoline S., 2022. "Interplay of policy experimentation and institutional change in sustainability transitions: The case of mobility as a service in Finland," Research Policy, Elsevier, vol. 51(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. O’Connor John, 2022. "Strengthening the science–policy interface in Ireland," Administration, Sciendo, vol. 70(4), pages 29-52, December.
    2. Crabolu, Gloria & Font, Xavier & Eker, Sibel, 2023. "Evaluating policy complexity with Causal Loop Diagrams," Annals of Tourism Research, Elsevier, vol. 100(C).
    3. Paula Kivimaa & Karoline S. Rogge, 2020. "Interplay of Policy Experimentation and Institutional Change in Transformative Policy Mixes: The Case of Mobility as a Service in Finland," SPRU Working Paper Series 2020-17, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Ansell, Christopher K. & Bartenberger, Martin, 2016. "Varieties of experimentalism," Ecological Economics, Elsevier, vol. 130(C), pages 64-73.
    5. Claire A Dunlop, 2014. "The Possible Experts: How Epistemic Communities Negotiate Barriers to Knowledge Use in Ecosystems Services Policy," Environment and Planning C, , vol. 32(2), pages 208-228, April.
    6. Srinivasa Vittal Katikireddi & Shona Hilton & Chris Bonell & Lyndal Bond, 2014. "Understanding the Development of Minimum Unit Pricing of Alcohol in Scotland: A Qualitative Study of the Policy Process," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-10, March.
    7. Kivimaa, Paula & Rogge, Karoline S., 2022. "Interplay of policy experimentation and institutional change in sustainability transitions: The case of mobility as a service in Finland," Research Policy, Elsevier, vol. 51(1).
    8. Jessica H. Phoenix & Lucy G. Atkinson & Hannah Baker, 2019. "Creating and communicating social research for policymakers in government," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-11, December.
    9. Walton, Mat, 2014. "Applying complexity theory: A review to inform evaluation design," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 119-126.
    10. Greenhalgh, Trisha & Engebretsen, Eivind, 2022. "The science-policy relationship in times of crisis: An urgent call for a pragmatist turn," Social Science & Medicine, Elsevier, vol. 306(C).
    11. Stucki, Iris, 2018. "Evidence-based arguments in direct democracy: The case of smoking bans in Switzerland," Evaluation and Program Planning, Elsevier, vol. 69(C), pages 148-156.
    12. Deas, L. & Mattu, L. & Gnich, W., 2013. "Intelligent policy making? Key actors' perspectives on the development and implementation of an early years' initiative in Scotland's public health arena," Social Science & Medicine, Elsevier, vol. 96(C), pages 1-8.
    13. Pierre-Olivier Bédard, 2015. "The Mobilization of Scientific Evidence by Public Policy Analysts," SAGE Open, , vol. 5(3), pages 21582440156, September.
    14. Gates, Emily F., 2016. "Making sense of the emerging conversation in evaluation about systems thinking and complexity science," Evaluation and Program Planning, Elsevier, vol. 59(C), pages 62-73.
    15. Nguyen, Sun V. & Langston, Nancy & Wellstead, Adam & Howlett, Michael, 2020. "Mining the evidence: Public comments and evidence-based policymaking in the controversial Minnesota PolyMet mining project," Resources Policy, Elsevier, vol. 69(C).
    16. William Ascher, 2021. "Coping with intelligence deficits in poverty-alleviation policies in low-income countries," Policy Sciences, Springer;Society of Policy Sciences, vol. 54(2), pages 345-370, June.
    17. Willis, Cameron David & Corrigan, Crystal & Stockton, Lisa & Greene, Julie Kathryn & Riley, Barbara Lyn, 2017. "Exploring the unanticipated effects of multi-sectoral partnerships in chronic disease prevention," Health Policy, Elsevier, vol. 121(2), pages 158-168.
    18. Giliberto Capano & Jun Jie Woo, 2017. "Resilience and robustness in policy design: a critical appraisal," Policy Sciences, Springer;Society of Policy Sciences, vol. 50(3), pages 399-426, September.
    19. Haynes, Abby S. & Derrick, Gjemma E. & Chapman, Simon & Redman, Sally & Hall, Wayne D. & Gillespie, James & Sturk, Heidi, 2011. "From "our world" to the "real world": Exploring the views and behaviour of policy-influential Australian public health researchers," Social Science & Medicine, Elsevier, vol. 72(7), pages 1047-1055, April.
    20. Plante, Charles, 2018. "Policy or Window Dressing? Exploring the Impact of Poverty Reduction Strategies on Poverty Rates among the Canadian Provinces," SocArXiv xtnfg, Center for Open Science.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:policy:v:51:y:2018:i:2:d:10.1007_s11077-017-9276-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.