The paper highlights that the assessment of effectiveness of anti-corruption interventions is hampered by such factors as:

• Impossibility of accurately measuring corruption and changes produced by anti-corruption interventions, in particular, because of the reluctance of stakeholders to discuss the topic honestly and openly;
• Problem of proving causation and contribution;
• Likelihood of unintended consequences and backlashes etc.

In order to mitigate their impact on evaluations, the U4 experts recommend:

1. Taking into account international best practices.

In particular, evaluation should be:

• Timely, useful, and used;
• Clear, focused, and tailored;
• Genuinely participatory;
• Gender-, human rights-, and sustainability-responsive, and in adherence with quality and ethical standards, as well as aid effectiveness principles.

2. Analysing expediency of conducting comprehensive evaluation.

On order to define the real possibilities of conducting comprehensive evaluation in certain circumstances, the authors of the document recommend responding to a number of questions related to the difficulties of assessment, including:

1. The character of anti-corruption measures: number and clarity of formulations in defining the objectives of introducing these measures; certainty of the actions to be undertaken in implementing the relevant measures; the number of components of the measures adopted etc.;
2. Social institutions and stakeholders: the procedure for funding the anti-corruption interventions, number of their participants, and list of stakeholders;
3. Causal relationship and feasibility of changes: the number of scenarios that may follow anti-corruption interventions; degree of certainty of their implementation; clarity and certainty of solutions;
4. Integration and character of environment: the possibility to foresee the impact of external factors on the outcome of interventions; the presence of mechanisms allowing for achieving comprehensive behavioral change.

3. Carefully collecting and employing data.

The paper highlights that the individuals who evaluate anti-corruption interventions often have difficulty:

• Getting precise data and/or data on specific anti-corruption interventions;
• Getting sufficient data;
• Ensuring data integrity.

In order to address these challenges the authors of the paper suggest:

• Determine the evaluability of the intervention at the outset;
• Select a diverse evaluation team that brings together evaluation, political economy, thematic, and local expertise, as well as access to stakeholders;
• Consider any biases, pressures, or repercussions that they or the evaluation team may face.

4. Honestly evaluating the effectiveness of anti-corruption interventions.

In the course of evaluation, it is necessary to take into account that:

• Some anti-corruption interventions can depend on the factors that fall outside the sphere of influence of participants; this is why the context is extremely important: for instance, in contexts where anti-corruption efforts are being actively undermined, preventing an increase in corruption or holding on to past gains may in itself be considered a success;
• Even minor changes in the level of corruption are difficult to make and maintain: the impact of interventions is often insignificant, and the project cycle too short to achieve considerable and long-term changes.

5. Focusing on changes if their source cannot be identified.

The recommendations stress that it can be rather difficult to prove the causal link between certain anti-corruption interventions and specific changes due to a number of reasons, in particular:

• Multiple donors and interventions may be supporting the same target unit (for example, the judiciary): a corruption offence is detected and the perpetrators are held liable thanks to the investigation conducted by journalists, adoption of a freedom of information law, provisions of the police, prosecutors, and judges with capacity building, election of a new prime minister etc.;
• Data limitations (for instance, their unsuitability for comparative analysis, reliance on the respondents’ opinion if data is based exclusively on surveys, inadequate coverage of data) etc.

This is the reason why it is recommended that evaluation experts focus on the results rather than on the measures that allowed achieving them.

6. Evaluating the degree of compliance of the problem addressed with the measures adopted.

In implementing anti-corruption interventions “wrong” methods for achieving results can be employed. As a result, it is recommended that the evaluation experts analyse the compliance of problems addressed with the measures adopted to this end in order to define the probability of achievement of the desired sustainable change and unforeseen circumstances.

To this end, it is needed to respond, as a minimum, to the following questions:

• Given the context, were the interventions expedient?
• Are the interventions, taken separately or together with other measures, sufficient to address the problem?

7. Defining the degree of compliance of anti-corruption interventions with ideas about corruption and methods to counter it.

The authors of the document highlight that in evaluating the effectiveness of anti-corruption interventions it is needed to take account of international best practices and successful anti-corruption initiatives undertaken at the regional level.

8. Exploring potential unpredictable consequences.

The paper stresses that in evaluating anti-corruption interventions it is important to pay attention to unforeseen consequences that come to light and reverse effects that can be either positive and negative, short- and long-term. To this end, the authors of the paper recommend:

• Assessing the extent to which there is a problem–strategy match;
• Speaking to third party experts with a deep understanding of the local context such as grassroots leaders, anthropologists, etc.;
• Selecting evaluation tools designed to uncover unexpected results such as Most Significant Change (MSC) stories and tracer studies;
• Exploring the political settlement, including whether there is sufficient political will to sustain reform efforts etc.

9. Assessing sustainability.

The authors stress that the evaluation experts should also assess sustainability of changes achieved due to the anti-corruption interventions. To this end, in particular, the following questions can be addressed:

• Are there means to maintain the results achieved?
• Are the stakeholders interested in constant anti-corruption efforts (for instance, will they maintain the results achieved after the funding ends)?
• Did the intervention address the underlying drivers of corruption?
• Is the intervention backed by powerful and influential stakeholders?

10. Using appropriate methods and evaluation tools.

The authors of the papers underline that the evaluation of effectiveness of anti-corruption interventions should be centered on the problems addressed rather than the assessment methods. The evaluation approach, methodology and methods should be determined by:

• the goal and scope of assessment;
• the questions to be addressed;
• the characteristics of the anti-corruption interventions subject to evaluation;
• context;
• data availability;
• resources available (financial, time, human).

Additionally, they should correspond to the interventions with regard to:

• attribution;
• explanation;
• multiple causal pathways, including nature of causal change;
• emergence;
• scope of effects.

The U4 experts also suggest employing the open-access tool developed by the international development network Bond to choose the right method. The list of these methods includes, in particular:

• process tracing: could be used to explore why new anti-corruption legislation was passed, and what role a particular anti-corruption intervention may have had in bringing about this result;
• qualitative comparative analysis: could be used to explore which traits and contexts enable anti-corruption agencies to successfully combat corruption;
• randomised control trial: could help assess the impact of a specific tool that is rolled out in phases over a large number of units.

11. Evaluating anti-corruption interventions according to certain pre-determined criteria.

The evaluation experts should define the criteria for assessing the results achieved. Is it sufficient to determine changes in attitudes? Or should the evaluation aim to determine changes in practice, such as asset declarations being filled out and reviewed, or an increase in corruption convictions? Alternatively, should the evaluation determine whether and by how much the level of corruption has been reduced and whether this has caused a significant change in the lives of beneficiaries?

Additionally, the evaluation can aim at assessing transformational change and systems change, i.e. the extent to which underlying drivers of corruption have been addressed, whether, for example, incentives and social norms have changed.

---

The authors of the document stress that their recommendations do not address all issues related to evaluation of effectiveness of anti-corruption interventions. This is why in addition to the publication the U4 experts suggest consulting Sida’s Evaluation Handbook: Guidelines and Manual for Conducting Evaluations at Sida and the Evaluation Handbook: Guidance for designing, conducting and using independent evaluation at UNODC.