LEADER Monitoring and Evaluation

5. Evaluation Topics

What are these evaluation topics?

Evaluation is often misunderstood as one of those extra tasks which has to be done, something which is done at the end of a project or programme, an additional but not very productive task, a cost or demand on resources. For Rural Development Programmes (RDPs) the purpose of evaluation is much more dynamic and productive than that, the aim here is to ‘support effective implementation and achievement of objectives and to report on programme achievements’¹.

For LEADER, (and not just LEADER) there is therefore so much more to evaluation than merely undertaking full scale evaluations of the whole Local Development Strategy (LDS), LAG or LEADER approach at set points in the life of the programme. As part of that dynamic and ongoing approach LAGs and MAs can identify specific topics or themes they want to investigate further or more specifically. These can therefore sit at both RDP level e.g. relating to RDP LEADER objectives or themes and at LDS level addressing e.g. local priorities. These can either be addressed as specific themes highlighted within existing evaluations or as specific targeted pieces of work.

What is the real value of evaluating specific topics for LAGs and MAs?

Evaluation topics offer LAGs and MAs additional targeted and flexible tools to help them more fully understand and improve their own LEADER implementation, focus on the achievement of their own objectives or to identify and demonstrate specific achievements. By identifying and investigating their own ‘evaluation topics’ LAGs, MAs and others can identify and focus on their own specificities whether at RDP or Local Development Strategy level. These topics can be investigated at any point in the programme when it is deemed to be most appropriate or relevant.

By addressing specific evaluation topics LAGs have an opportunity to deal with some of the challenges of LEADER evaluation. They can focus on their own priorities, those things which they know and expect to demonstrate their real achievements in terms of their own strategy, area and partnership. Through the investigation of specific evaluation topics those involved in LEADER have a great opportunity to really develop their understanding of what their own approach can and does deliver.

For both LAGs and MAs this can offer real advantages in linking the two levels of evaluation, LDS and RDP by developing a better understanding the micro level effects and their contribution to the macro and then communicating this.

The challenge of effectively evaluating LEADER through conventional approaches is well documented and understood. With finite resources and LEADERs increased prominence the need to demonstrate the validity of the approach and the added value it creates has never been greater. The careful selection and use of specific evaluation topics can make a significant contribution to establishing, demonstrating and communicating the added value of LEADER, the LAGs specific intervention logic and the range of achievements both locally and more widely.

How should LAGs and MAs go about identifying their evaluation topics?

Evaluation topics are those elements which LAGs or MAs may wish to investigate more specifically. Identifying these therefore involves the LAG or MA thinking through their specific needs, this can be done independently or together, LAGs may even come together and identify common or shared topics they wish to investigate.

Establishing the relevance, effectiveness, efficiency and impact of LEADER projects are the givens of mainstream evaluation but what about the effects of the method, etc. Think about the particular evaluation questions you want to have answered. The most important thing is to concentrate on identifying those topics which are likely to add value to or complement the main (essential) evaluation activities.

Those topics contributing to improving the performance of their LDS and to establishing the added value of LEADER as an approach and a governance and delivery mechanism are also of particularly high relevance here for LAGs. For MAs investigating those aspects of LEADER implementation which affect achievement are likely to be a priority.

It is important to recognise that the most important topics may include some which are not too significant in terms of the level of allocated resources but are nevertheless important. Examples here might include particular aspects of the LEADER method, the effects of doing things in a particular way, of establishing specific partnership and representation principles in the LAG, the impact on or influence of social or cultural values, contributions through participation or knowledge etc.

In developing an evaluation plan at the beginning of the programme it is necessary at both LAG and MA level to identify the initial topics you think will be important apart from minimum reporting requirements established on EU level. This will allow you to develop your plans, possible methods and resources. In developing the plan it is also important that you also build in some flexibility, a process of review, plan for the unexpected as some of the most important topics may emerge as a consequence of LDS delivery or contextual changes along the way (e.g. as we saw with the impact of the financial crisis during 2007 – 2013).

What are possible topics?

By their very nature and purpose evaluation topics are specific be that to the individual LAG, MA, LDS or RDP according to their specific objectives. The possible topics highlighted here are therefore broad and aim to provide an indication of what is relevant and possible. Of particular relevance are those topics which contribute to improving the performance of LAGs LDS and to establishing the added value of LEADER. MAs will be more interested in programme level effects but many of the topics may be similar.

Examples of topics likely to be relevant include:

Specific elements of the LEADER approach, how it functions and the effects it brings.

• All features of the approach may be considered but innovation is often a high priority;
• Evaluation of the ‘partnership principle’ is a core part of evaluating LEADER at RDP level. This refers to the public private partnership which is inherent in the LAGs, how partners are involved and the way LAGs make decisions and work. At RDP or LDS level there may be specific aspects of how this is done which you wish to investigate further through monitoring and/or evaluation.
• The effects of the use of specific methods e.g. for animation, community engagement or ensuring accountability;
• The diversity and profile of partners represented and engaged in LAG governance
• The effectiveness of particular activities or tools e.g. the benefits of using small grant schemes;
• Examining the additional resources which the LAG or projects can leverage beyond conventional approaches e.g. in kind contributions, voluntary inputs etc.;
• The extent and effects of any links which are developed, emerging networks, links between projects, links between sectors; and
• Piloting or experimenting with new approaches to evaluation.

Specific priorities in the LDS focusing on the intervention logic established on national and LDS level.

• Thematic studies focusing on the achievements re particular priorities or themes identified;
• Area effects, e.g. investigating the effects where particular areas have been targeted or where uptake of support is high or low;
• The effects on particular target groups or sectors which the LDS might focus on;
• Gap analysis e.g. looking in to where and why areas, groups, themes, specific communities may not have been engaged; and
• The contribution of the LDS to the RDPs horizontal themes or elements.

¹ Helpdesk of the European Evaluation Network for Rural Development, Guidelines: Establishing and Implementing the Evaluation Plan of 2014-2020 RDPs, March, 2014