Evaluation of RDPs 2007-2013

Evaluation Concept

The approach to evaluation has changed significantly for the Rural Development programming period 2007-2013, based on identified needs and lessons learned from previous periods. A unique framework is now established for monitoring and evaluation of all measures funded through the European Agricultural Fund for Rural Development (EAFRD).

The Common Monitoring and Evaluation Framework (CMEF) has been developed, in view of guiding Member States towards a more effective system for assessing progress towards Community and national objectives, ensuring the accountability of public spending through Rural Development Programmes, and improving programme performance. The CMEF is put together in a handbook which includes a series of evaluation guidelines and guidance fiches on a set of common indicators for monitoring and evaluation.

The new approach is aimed at better harmonisation between monitoring and evaluation, supporting timely establishment and quantification of common and programme specific indicators, making use of a new set of common evaluation questions based on Community priorities and objectives, and identifying evaluation needs and facilitating appropriate capacity building. Impacts are now being monitored on a regular basis throughout the period by means of an ongoing evaluation system that Member States are required to establish. Evaluation of impacts is ultimately done at programme level in relation to the Community Strategic Guidelines and the specific rural development priorities of different Member States and regions, after first taking account of outputs and results at local level.

The establishment of an Evaluation Expert Network is a key element of the new approach. It helps establish good practice and capacity building in the evaluation of Rural Development Programmes, thereby increasing the utility of monitoring and evaluation as tools for improving the formulation and implementation of Rural Development Policy.

The legal basis for this new approach is laid down in Council Regulation (EC) No 1698/2005 of 20 September 2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) and in the implementing Regulation (EC) No 1974/2006 of 15 December 2006 laying down detailed rules for the application of Council Regulation (EC) No 1698/2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD). Read more about the legal basis.

Ongoing Evaluation

Ongoing evaluation covers all evaluation activities carried out by the Member States during the entire programming period. This comprises ex ante (part of programme preparation), mid-term (in 2010), and ex post evaluations (in 2015), and other evaluation activities, including the interaction between evaluation activities, the compilation and refinement of indicators, and data collection, as well as ensuring adequate capacity building.

Ongoing evaluation therefore consists of three main elements which are closely interlinked and form an integrated approach to optimizing evaluation to help improve programme implementation:

continuous activities for evaluation at programme level with annual reporting on those activities; in 2010 and 2015 the reports include mid-term and ex post evaluations;

accompanying thematic studies to be carried out at the initiative of the Commission, examining in closer detail certain measures, axes, geographic zones, or specific aspects of rural development policy wherever and whenever the need for such examination arises;

an evaluation network and support infrastructure for the Member States and/or regions to be animated by the Commission; this institution provides a help desk function (for the interpretation of evaluation guidelines), helps with capacity building and provides a platform for methodological exchange.

The main objectives of ongoing evaluation are:

timely establishing and quantifying baseline indicators and target levels
better linking monitoring with evaluation in terms of data collection/ provision
ensuring capacity building, including from the early stages
ensuring continuity of evaluation activities, i.e. regular assessment of progress, annual reporting
supporting the establishment of good practices
preparing the ground for the mid-term and ex post evaluations (2010, 2015)
The System of Ongoing Evaluation

Member States and programme regions are responsible for setting up a system of ongoing evaluation and for providing the human and financial resources necessary to carry out evaluations, as stipulated by Article 84(5) and Article 86(1) of Council Regulation 1698/2005.

The system facilitates Member States and programme regions to:

provide for continuous capacity building and exchange of good practice for evaluation stakeholders
examine the progress of the programme in relation to its goals;
improve the quality of programmes and their implementation;
prepare for mid-term and ex post evaluation (through continuous data collection, development of methodologies, etc);
examine proposals for changes to programmes;
allow for continuous feedback through annual reporting;

The ongoing evaluation system consists of two main aspects: administrative arrangements and indicators and data collection, as shown in the figure below.

Diagram Missing

Administrative arrangements

The Managing Authority is responsible for ensuring that programme evaluations are conducted in accordance with the relevant regulations and in conformity with the Common Monitoring and Evaluation Framework (CMEF). Setting up a Steering Group to represent the different departments is advisable. The task of the Steering Group is to support and monitor the work of the evaluators.

The ex ante, mid-term and ex post evaluations must be carried out by independent evaluators from bodies without direct involvement in the implementation, management and financing of the programmes. The evaluators can be public institutions provided that they fulfil the criteria of independence and competence. The same evaluator may deal with the evaluation at all stages of the programming cycle. Such an arrangement may in some cases improve continuity and reduce the costs of evaluation.

A Monitoring Committee needs to be established for each rural development programme but Member States with regional programmes may establish just one national Monitoring Committee. The role of the monitoring committee is to examine ongoing evaluation activities (on an annual basis) and, if needed, to propose adjustments of the rural development programmes to the Managing Authorities. Its composition is decided by each Member State, and includes competent regional/local/other public authorities, economic and social partners, any other appropriate body (civil society, NGOs, including environmental etc.).

Indicators, evaluation questions and data collection

The Managing Authority has to review all indicators and related evaluation questions, to be able to assess what needs to be done in terms of information and data collection and analysis.

Because the common evaluation questions and indicators of the CMEF are defined in a manner that make them applicable across a large number of programmes, more precise indicators, target levels and evaluation questions (i.e. additional to the common indicators, target levels and evaluation questions) have to be established by the Managing Authority to take account of programme specificities. The aim is to be able to answer the evaluation questions in a meaningful and appropriate way during mid-term and ex post evaluations.