Making policies work

Publications

Presentations

Thematic evaluation of the visibility of Strengthening European External Action

Presentation of the final report

September 2011

Mackie, J. Thematic evaluation of the Visibility of EU External Action - Presentation of the Final Report. (ECDPM Presentation).

Share Button

Visibility of EU External Action Evaluation Seminar 
European Commission 
7 September 2012, Brussels 
James Mackie, who was the evaluation leader, made a presentation to this meeting to discuss follow-up to the evaluation’s recommendations.

Presentation transcript

Thematic evaluation of the visability of EU External Action

 

1. Thematic evaluation of the Visibility of EU External Action 
Presentation of the Final Report Dissemination Seminar, 7th September. 2012 9.15-14.30, Centre Albert Borschette, Brussels (Evaluation managed by DRN and led by ECDPM)

2. Content of the presentation 
•  Introduction
•  Methodological approach
•  Main findings and analysis
•  Conclusions and recommendations

3. Introduction

4. Introduction: the Evaluation Team  
•  James Mackie – Team Leader (ECDPM)
•  Federica Petrucci – Evaluation Manager & Expert Migration Theme (DRN) 
•  Volker Hauck (ECDPM) & Raphael Brigandi – Communication  
•  Andrew Sherriff – Expert Conflict Theme (ECDPM) 
•  Mark Kowal – Expert Climate Change Theme 
•  Ivo Morawski – Expert Environment Theme (DRN)
•  Isabella Massa – Expert Financial Crisis Theme (ODI) 
•  Pierre van Roosbroek – Expert Food Crisis Theme 
•  Bent Bonde – Communication & Tunisia Case Study  Sara Monti – Evaluation Assistant (DRN)
In Bold: Team members present in the Seminar

5. Introduction: the Mandate
•  Evaluation for 3 „Relex DGs‟: RELEX, DEV & AIDCO (External Relations, Development & EuropeAid)
•  Terms of Reference (February 2010): the purpose:
-To provide an overall independent assessment of the visibility of the Commission’s external action;
-To identify key lessons to improve current and future Commission strategies on visibility.
•  Period covered: 2006-2011
•  Geographic scope: Global (except Balkans)
•  Starting point in policy terms:
-2006 Draft Communication from the Relex Commissioner Mrs Benita Ferrero-Waldner: 
-“The EU in the World: Towards a Communication Strategy for the European Union’s External Policy, 2006-2009” (C[2006]329)

6. Policy position
•  Communication challenges identified in 2006 Draft Comm: 
– Global solidarity, Enlargement, ENP (Neighbourhood policy), Africa, Strategic bilateral relationships
•  Official documents show that Visibility is seen as:- Consequence of EC actions among other things 
-Evolves from static Visibility concept (logos & stickers, etc.) towards a dynamic Visibility influenced by actions 
-Is linked to democratic accountability internally and mutual accountability externally 
-Outside the EU – the objective is to explain to current and potential opinion-formers the EU‟s policies and activities and values and objectives

7. Definition of Visibility…
•  Definition agreed with Commission stakeholder “Reference Group”…
-“The awareness and perception of the image of EU external action among EU and non-EU stakeholders resulting from EU communication activities or from other actions that have an impact on this image”.
•  Underlines 
-Message conveyed ≠ message perceived 
-Work of EU communication activities, but also
-Impact of other actions and events
•  Also keep in mind 

-Importance of mass media in shaping perceptions

8. The Communication Prism

9. Methodology

10. Methodology

•  First step: agree on list of Evaluation Questions with theCommission stakeholders (Reference Group) :
•  Based on TOR + limited in number to focus the study
•  Agreed on 10 Evaluation Questions (EQ): 

– EQ 1, 2 & 4: Audience perceptions of EU external action and the quality of theses actions
– EQ3: Internal unity of purpose on visibility in the Commission
– EQ 5 & 6: Issues of inter-institution collaboration on visibility and coherence
– EQ 7, 8 & 9: Resources and implementation
•  For each worked out Judgement Criteria (JC) + Indicators

11. The Evaluation Questions – shortened version
EQ Issue
EQ 1 The image of EU external action
EQ 2 Do the Visibility communication priorities achieve their objectives?
EQ 3 A single, clearly defined visibility strategy for EU external action?
EQ 4 Perceptions of the benefits of EU external action
EQ 5 Coordination on visibility between the EU institutions and with the EU member states?
EQ6 Coherence of messages across external and internal policies?
EQ 7 Value added of EU as a global actor in major international fora?
EQ 8 Adequacy of EU resources to carry out the visibility strategy?
EQ 9 Stakeholders perceptions of results relative to cost
EQ 10 Translating visibility strategies into action plans

12. Methodological approach to research Complementarity 

13. Methodology: the sources of evidence 

14. Methodology: the sources of evidence 

15. Methodology: the six Thematic Studies
•  Thematic cases Countries Specific EC/ EU (field work) programme/event 
•  Conflict & fragile states – Georgia – 
•  Climate change & energy – Cambodia – GCCA
•  Environment, biodiversity & deforestation – Indonesia – FLEGT 
•  Migration – Mali – CIGEM Italy Lampedusa
•  Financial & economic crisis – Grenada & Barbados – V-FLEX 
•  Food crisis & food security – Kenya – Food Facility

16. Methodology: the six Thematic Studies (2)
•  Key party of study and main data gathering exercise
•  Prior desk research from Europe on EU‟s engagement on thematic issue 
-Selection of country and of programme / event
•  In–country visits – these typically included
-Interviews with EU Delegation and other EU representatives, government ministries, donor representatives including EU member state embassies and UN, media and CSOs
-Visits to funded projects/programmes 
-Local media coverage analysis 
•  Individual Thematic Reports – all published

17. Methodology: MCA – Media coverage analysis

18. Methodology: MCA – Media coverage analysis
•  Systematic and objective look at print media portrayal of EU external action
•  Searches done on 20 EU newspapers & 1 external (NYT)
– Based on key words & defined periods 
–  Little international coverage on most cases, but two very successful: Georgia (603 articles) + Tunisia (165)
•  Country 20 European Newspapers
FR La Croix, Le Figaro, Le Monde, Liberation 
DE Bild Zeitung (online version) Frankfurter Allgemeine, Frankfurter Rundschau, Süddeutsche Zeitung, Die Welt 
UK Daily Mail, Daily Telegraph, Financial Times, The Guardian 
DK Berlingske, Information, Jyllands-Posten, Politiken
PL Gazeta Wyborcza, Rzeczpospolita, Superexpress

19. Methodology: Interviews
•  Some 260 interviews conducted in total: in 6 field mission countries, + Tunisia, Italy and in Brussels:
•  In Brussels structured interviews with 3 principal groups 
-EU Institution officials in RELEX, DEV, AIDCO, Council Secretariat, DEVCO, DG COMM, EEAS, EP secretariat press service (before and after Lisbon changes) 
-Media: dozen journalists both European & foreign 
     Very well informed and critical group – act as filter to messages across Europe 
– CSO & Think Tanks representatives (13) 
     Another well informed group with extensive networks across Europe and around the world

20. Methodology: E-Survey
•  Well informed actors, only exercise with global coverage
•  Focus on civil society – 4,031 addresses world wide 
– E-mail invitation to go on to an internet survey site
•  Response rate: just over 5% (221) – normal rate
•  Simplified version of 4 EQs used: EQ1, EQ4, EQ6 & EQ7

•  Respondent characteristics
-Type respondent: 40% NGOs, 40% Academics …
-Geographic spread: 60% Europe, 28% Africa …
-68% senior staff members, 27% middle level
-57% claimed high familiarity with EU external action & 38% average familiarity …

21. Methodology: Case Studies
•  Strategic Partnership: UN-EU partnership
– Based on documents, web-sites and interview
•  Two unplanned events – what EU reactions
-Lampedusa – inflow of migrants in early 2011 
         Internal event but with impact on EU‟s external image 
-Tunisia – the EU & the democratic uprising        External event to which EU had to react and calling into question past EU external action 
•  Based on media coverage analysis & interviews

22 Methodology: Eurobarometers
55 Eurobarometer surveys examined for 2000-2010
-Professional polling carried out across EU but mostly relate to life in EU and internal affairs
-6 of specific interest for this study all relating to attitudes to development, MDGs and Africa

23. Main Findings and analysis

24. Logical build up of analysis
•  Data from each source
•  Findings from each source analysed by 10 EQs and 29 JCs (Judgement Criteria)
•  Consolidated response to each EQ
•  6 overall Conclusions
•  Recommendations based on conclusions

25. The EQs: use of sources beyond thematic studies
EQ Subject of EQ   Additional sources used
1 Image of EU external action EU, MCA, E-Survey, media, EBS
2 Commctn priorities+objctvs EU, MCA, E-Survey, UN, media
3 A single visibility strategy EU officials
4 Perceptions of the benefits EU, MCA, E-Survey, CSOs+TTs, EBS
5 Coordinatn EU instns. & MS EU, MCA, CSOs+TTs, media
6 Coherence across policies? EU, MCA, CSOs+TTs, media, E-Survey
7 EU as actor in interntnl fora EU, UN, CSOs+TTs, E-survey, MCA, media
8 Adequacy of EU resources EU officials
9 Perceptions of results /cost EU, CSOs+TTs10 Strategies into action plans EU, CSOs+TTs, media

26. The EQs – short answers
EQ Issue – Short answer

1 The image of EU external action – Positive
2 Do commctn priorities achieve objectives? – Mixed, some recognition
3 A single visibility strategy for EU action – Fairly united but could be stronger
4 Perceptions of the benefits of EU action – Mixed, but positive
5 Coordination between EU institutions & MS? – Varied by location, topic & stakeholder
6 Coherence of messages across policies? – Varied by stakeholder
7 EU as a global actor in international fora? – Positive with exceptions Mixed: EU/UN collab
8 Adequacy of EU resources – OK though distribution?
9 Stakeholders perceptions of results / cost – Difficult to assess
10 Visibility strategies into action plans – Variable

27. Conclusions and recommendations

28. Conclusions
1. The image of EU external action is in line with pre- Lisbon priorities for external action communication
2. Communication on EU external action lacks overall direction and leadership
3. Working in partnership with others is essential but there is a trade-off in lower EU visibility
4. The image of EU external action varies geographically as well as by constituency
5. The nature of the EU imposes constraints that impact on its visibility
6. The resources for promoting the visibility of EU external action are adequate

29. Findings –>Conclusion –> Recommendation 1
Overall picture from study is that EU external action has an image that in substance conforms more or less to what officials seek but many observers are critical of that image – more modesty needed
• The image of EU external action is broadly consistent with priorities of Draft Communication (Ferrero-Waldner, 2006) 
• Actions speak louder than words – EU image much more influenced by highly visible actions than by „stickers and flags‟ 
• Raising unrealistic expectations – a frequently cited problem

Conclusion: The image of EU external action is generally in line with pre-Lisbon official priorities

Recommendation: Reaffirm, renew and strengthen the established image of EU external action

30.Findings –>Conclusion –> Recommendation 2
 Implementation of communication activities has suffered set back with reorganisation of services post-Lisbon • Communication work best done with clear political priorities
• Actual conduct of communication activities generally good with coordination mechanisms in COM and with EEAS though not with Council and with MS,
• Quality strengthening measures being taken by COM
• Positive signs of progress post-Lisbon

Conclusion: Communication work on EU external action lacks overall direction and leadership

Recommendation: Provide stronger central direction and leadership for communication work on EU external action

31.Findings –>Conclusion –> Recommendation 3 
Partnerships important for EU so need a clear settlement on this
• Working in partnership with others inevitably reduces EU visibility because there is a need to „share visibility‟ – but this a always trade-off which also has advantages for the EU
• Use of budget support aid modality diminishes EU visibility but increases ownership which is a development benefit
• Working with UN at project level reduces visibility but can be managed with careful use of FAFA – UN an essential partner • Working in multilateral for a – the EU often has good visibility

Conclusion: Working in partnership with others is essential but there is a trade-off in lower EU visibility

Recommendation: Agree that working in partnership with others is essential but imposes a trade-off in lower EU visibility that needs to be sensitively managed

32.Findings –>Conclusion –> Recommendation 4 
Some high-profile features of EU, for instance the € are known around the globe, but otherwise the image varies from place to place and between stakeholders:
• Distance softens the image, but EU positions on sensitive issues are noted, also in Africa, Asia & Latin America better. This also depends on topic (eg. trade, migration, humanitarian aid …)
• Close EU Observers are the most critical: Brussels journalists, some national media, CSOs & Think tanks

Conclusion: The image of EU external action varies geographically as well as by constituency

Recommendation: Manage sensitively geographic and constituency variations in the visibility of EU external action

33.Findings –>Conclusion –> Recommendation 5 
The EU is a particular state construct with special features that can be problematic. This is recognised by well informed observers
• The specific nature of the EU imposes certain constraints – often has to explain itself, parts pull in different directions, leadership with limited image across the EU …
• The problems of internal competition for visibility can be a problem in Brussels, but seem not so serious at country level
• Cooperation between EU & MS is often a problem for visibility • Policy Coherence is more of an issue than officials seem to think

Conclusion: The nature of the EU imposes constraints that impact on its visibility

Recommendation: Pay special attention to the impact on visibility of the EU‟s specific nature

34.Findings –>Conclusion –> Recommendation 6 
Overall conclusion is that there seems to be enough resources for communication work – no strong evidence of inadequacy emerged
• Resources for visibility in general seem adequate … but their distribution does cause some difficulty
• Some EUDs appear to lack resources for visibility
• Resources in projects not always well spent
• Lack of political strategy can be a bigger problem than lack of resources

Conclusion: The resources for promoting the visibility of EU external action are adequate

Recommendation: Review the distribution of resources for promoting the visibility of EU external action particularly at the level of EU Delegations

35. Recommendations
1. Reaffirm, renew and strengthen the established image of EU external action
2. Provide stronger central direction and leadership for communication work on EU external action
3. Agree that „working in partnership‟ with others is essential but imposes a trade-off in lower EU visibility that needs to be sensitively managed
4. Manage sensitively geographic and constituency variations in the visibility of EU external action
5. Pay special attention to the impact on visibility of the EU‟s specific nature
6. Review the distribution of resources for communication work particularly at the country level

36. For more information:
James Mackie – Team Leader (ECDPM) – jm@ecdpm.org
Volker Hauck – Communication (ECDPM) – vh@ecdpm.org
Andrew Sherriff – Conflict Theme (ECDPM) – as@ecdpm.org

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Comments

European external affairsPresentationsEuropean External Action Service (EEAS)