Learn to Walk before you Run? A Review of Methodological Approaches for Evaluating Coherence in the Field of International Cooperation

This study looks into how coherence has been evaluated in different policy fields and, on that basis, puts forward recommendations on how to improve the measurement of coherence in the field of international cooperation. It does so on the basis of a systematic review of past studies that examine coherence inside and across public policies, paying special attention to the applied research methods. The study addressed four research questions: a. In what ways and to what extent can ‘coherence’ be defined and operationalised for evaluation purposes? b. To what extent can the relation between coherence and effectiveness/efficiency be evaluated (i.e. is coherence additional or complementary? c. What methods have been used in past studies and evaluations that look into coherence inside or between policies, at what levels (micro, meso, macro), and what are their respective strengths and weaknesses? d. Based on the answers to the first three questions, what practical and methodological dilemmas can be observed with regard to improving the evaluation of policy coherence in the specific field of policies on international cooperation?

Based on a structured search of academic journals as well as evaluation reports, 22 studies were identified as the basis for the analysis under research questions 2 and 3. These could be grouped under three separate policy ‘fields’:

a. Development cooperation b. Environmental safety and improvement of the environment c. External action towards fragile states and the 3D approach