How can you make your NRN evaluation count!?
The evaluation of National Rural Networks (NRNs) is often done in the context of the broader RDP evaluation, however, standalone evaluations for NRNs are also possible. Regardless of the type of evaluation of the NRN, before beginning the evaluation of the NRN itself an early emphasis on key facets of the planning and preparation steps of the evaluation process can help to easily provide a solid foundation to ensure later success.
- Planning of NRN evaluations: when commissioning an evaluation, fundamental principles should be followed in order to achieve a high quality evaluation. When considering evaluations of NRNs, the process can be broken up into three parts: planning, preparing and managing the evaluation. Different aspects of the NRN can be assessed in this process: a) full evaluations, which includes the assessment of efficiency, effectiveness, relevance and impact; b) evaluations of the NRN’s intervention logic; c) evaluations of the network structure. Dissemination and communication of evaluation results should also be taken into account at an early stage.
- Communication as a strategic early activity: in the preparation phase, special attention should be given to planning communications and capacity building approaches to be implemented throughout the evaluation. Even though most of the communication activity occurs at the end of the evaluation process, it is essential to communicate throughout the evaluation process. This will provide a clear picture of what kind of information should be transmitted to different evaluation stakeholders, by whom and at what point.
- Review of the intervention logic: review of the intervention logic is an important step in the preparation of evaluations. In many cases, however, the intervention logic is already constructed at the programming stage. The process of constructing the intervention logic starts with conducting context analysis and reflecting on the SWOT and needs assessment in the common NRN objectives and groups of activities spelled out in Article 54 of Regulation (EU) No 1305/2013. The Member States should complete the intervention logic by defining NRN-specific objectives and activities. This step is followed by the specification of NRN outputs, results and impacts. If the intervention logic already exists, the Managing Authority, NSU or evaluator are advised to conduct a review of the intervention logic.
- Definition of evaluation elements: the preparation process continues with the development or review of evaluation elements (evaluation questions, judgement criteria and indicators). The legal framework specifies some common elements (intervention logic, 4 objectives, 7 groups of actions, 1 evaluation question and 3 output indicators). These common elements only represent the basis for NRN evaluations and refer principally to evaluations of NRNs as part of the RDP evaluation. Self-standing evaluations require substantial elaboration of evaluation elements. Hence, the Member States are expected to complement the common elements with NRN-specific elements. Evaluation questions should be derived from the common or NRN-specific objectives. Each evaluation question should be accompanied by judgement criteria, specifying the individual questions and linking them to the indicators.
- Information sources for the NRN evaluation: subsequently, it is necessary to identify which data and information are needed to answer the evaluation questions. The sources of the data should be listed, and it should be clearly outlined which data can be obtained from the Managing Authority, Paying Agency or the NSU and which information should be collected additionally by the evaluator. Most information relating to NRN evaluations will be qualitative due to the nature of the NRN’s actions and the fact that there will not be an abundant amount of quantitative data available. Hence, qualitative or mixed methods will be applied more frequently within NRN evaluations. Considering early on the strategic components of the planning and preparation steps will allow evaluation stakeholders to achieve better quality evaluations. Ultimately, it will also facilitate the role of evaluation as an important governance tool in the policy cycle, contributing positively to the implementation of policy instruments and funds.
To support evaluation stakeholders in conducting their tasks, the Evaluation Helpdesk has developed the guidelines, ‘Evaluation of National Rural Networks 2014-2020’, in close collaboration with the European Commission and the Member States. These guidelines can not only help to plan and prepare evaluations but also give further recommendations on how to implement an NRN evaluation as a stand-alone exercise or as part of a programme evaluation.
Find other publications on NRN evaluation in the Helpdesk eLibrary!