Political Science Tools For Assessing Feasibility And Sustainability Of Reforms

We were asked by the Independent Evaluation Office to outline political science methods for assessing the chances of reform implementation in an ex-ante fashion. We agreed to illustrate how these tools 'work' by using Pakistan as a case study. The recent literature on IMF-sponsored reforms points out that successful implementation not only depends on the nature and severity of the economic crisis and on the design of the reforms, but very much also on the political economy of reform politics. We have identified the following as salient political factors for identifying chances of reform success: - the power of sections of the economy and polity that will lose from effective implementation; - the political independence of reform-minded branches of government vis-à-vis politicians that depend on popular support; - the institutional capacity to implement reform; - a high degree of acceptance of the reforms among the major stakeholders (the 'ownership' factor). We have designed three tools that help forecast how these factors will develop in the future. Each tool comprises three dimensions of analysis: - how these factors will develop after signing an agreement, given visible trends in the immediate past (trend extrapolation); - how these factors would be influenced by an effective reform implementation (impact analysis); - how other political framework conditions will evolve and what impact this may have for the reform prospects (scenario building). The three tools belong to different types of forecasting techniques and thus illustrate the wide range methods available. They also address different combinations of the four political factors. The three tools are summarized below. Tool 1: Stakeholder analysis This tool forecasts how the political struggle over reform will end by assessing the relative power and influence of the major stakeholders and by estimating how this balance of power will develop in the future. The three dimensions of analysis could look as follows: Trend extrapolation involves a close inspection of: 1) the reform steps undertaken during and before the negotiation period; 2) the negotiation style of the government (inclusiveness and transparency); 3) the degree of ownership of the reform idea among the major stakeholders. Impact analysis estimates how the power base of the actual government (factor 1) and the relation between civil servants and elected politicians (factor 2) will change due to effective implementation of the reforms and how this in turn influences the probability of continued implementation in the mid-term. Scenario building integrates other independent trends (e.g. declining power base of a party in power) as well as unforeseeable events (such as a foreign policy crisis) into the assessment exercise. The scenarios may be ranked by probability. Tool 2: Institutional analysis This tool would comprise three different elements of analysis. Institutional mapping describes the network of institutions (both governmental and non-governmental and at different levels) involved in decision making and reform implementation. The veto power analysis then determines the relative power and independence of those branches of the bureaucracy that are able and determined to implement reforms. The capacity assessment would look at levels of professionalism, recruitment procedures, educational background and motivation in those branches of government. Trend extrapolation would take into account actual trends of institutional change in determining the chances of reform implementation. Under the impact analysis, the institutional consequences of the reform programme itself and their impact on capacity and willingness to reform can be assessed. The scenario technique could be used to produce different scenarios of mid-term institutional change and see how they influence the prospects for economic reform. Tool 3: Delphi study Delphi studies belong to the pool of expert opinion tools. It consists of at least three rounds of surveys administered by a questionnaire. The experts may adjust their responses in the second and subsequent rounds after having been informed about the mean answers of the previous round. We suggest to ask at least 15 experts from think tanks, advisory bodies, the media, universities etc. to assess a) the prospects for the reforms being implemented given current political trends; b) the political impact of the reforms and how it may affect the possibility of sustained reform; and c) the probability of various mid-term political scenarios and the chances for sustaining reform under these scenarios. One of the comparative advantages of Delphi studies is that the results are not influenced by opinion leader phenomena. They can be used to quickly assess the constellations of opinions with regard to specific policy options and the probabilities associated with different future developments. In the concluding section we recommend - to apply the maximum possible number of tools in order to arrive at a solid assessment of the political feasibility of a programme from different perspectives. - to apply the 'triangulation of methods' approach whenever it is necessary to outbalance different results produced by the different tools. This means to reinterpret results and search for new evidence until more coherent overall conclusions can be reached; - to develop a multi-tier assessment system, where the basic tier, streamlined to all IMFsupported programmes, would consist in the trend extrapolation and impact assessment components of stakeholder analysis; institutional analysis would represent a second tier, to be applied to cases where doubts about implementation prospects are higher; a Delphi study, including scenario building, represents the most complex exercise reserved for the most contested cases; - to rely on careful judgement when deciding to more systematically include political factors, taking into account the risks of becoming involved in political and institutional engineering in sovereign nation states.