A solid framework for evaluating social innovation: SIMRA’s methodology for evaluating social innovation in rural areas is relevant to monitoring and evaluating agricultural and rural development policies. The evaluation framework proposed by SIMRA is based on the project’s experience gained by reviewing a significant number of cases and critically reviewing an exhaustive number of evaluation frameworks.
The presence of a solid framework for evaluating social innovation is crucial. There are three components of social innovation to be evaluated:
- The social innovation 'process', which refers to the emergence of a social innovation idea in a given context and the interaction of actors, i.e. the preparatory actions undertaken by the core group (the 'innovators' and the 'followers') and the reconfiguration of social practices (the 'transformers'). The process is dynamic and paves the way for the social innovation 'project'.
- The social innovation 'project', which consists of the tasks and activities that are implemented by project partners for realising in practice the social innovation idea. The social innovation project is thus a way to provide answers to societal needs and challenges.
- The 'effects', which represent the causal relationship between an intervention and its effects. These effects could affect the economic, social, environmental and/or institutional/governance domains.
SIMRA proposes, a rapid evaluation approach based on indicators and a detailed approach based on more complex, composite indicators. For example, to answer the specific evaluation question related to ‘the extent to which the social innovation initiative affected social cohesion inside and outside the territory according to the beneficiaries’, the rapid evaluation approach calculates an indicator based on responses from a questionnaire survey. A long list of themes and their corresponding simple and composite indicators for both approaches is provided in the technical annexes to the evaluation manual.
The SIMRA evaluation framework can be used for both micro and macro level evaluations of social innovation. For instance, macro level evaluations would look at the contributions of social innovations to broader policy targets such as the Green Deal. Evaluations would also allow to compare impacts, which reflect changes in the contexts of rural communities.
A database of frameworks, approaches, methods and tools for evaluation: It is helpful to those charged with monitoring and evaluating innovation. It can serve as a basis for evaluations of social innovation and what is delivered and can link social innovation with indicators. For instance, participants in social innovation projects and financial resources directed to support social innovation projects can be considered inputs to the estimation of I.01 and R.01. These can be further disaggregated by the special objective(s) in which the innovation took place. The evaluation framework focuses on various aspects and evaluation questions directly or indirectly related to territorial development, such as ‘how a social innovation responds to societal challenges’ or ‘how civil society engaged’. Thus, it is relevant in evaluating issues related to the CAP’s specific objective to ‘Promote employment, growth, social inclusion and local development in rural areas’. Therefore, the database is helpful for evaluators who can be inspired by the broad range of existing evaluation frameworks.
An evaluation manual: The evaluation manual is also a significant contribution to evaluation by providing specific practical guidance for evaluating social innovation, particularly the section on ‘Preparing and Designing the Evaluation’ and the section on ‘Tools’. The latter includes a significant exposition to questionnaires targeting various segments of the social innovation environment (innovators, followers, transformers, project partners, beneficiaries, etc.).
The Manual includes a very useful tool (number 9 in the manual) which is an Excel file that allows the evaluator to insert the data collected through the use of tools (questionnaires) and to automatically compute the indicators at different levels of aggregation. All the indicators in the SIMRA Manual are new, and can be aggregated into new composite indicators and complex indexes.
Besides getting ideas and inspiration, evaluators and other stakeholders can use the evaluation manual, the database and the MOOC to support and complement their educational tools and training material.
In conclusion, what is special about the SIMRA evaluation framework and methods:
- The science-stakeholders co-constructed process of development, testing and validation
- The inclusion of contemporary, emerging issues in the evaluation (e.g., social capital, networks, governance, actors’ satisfaction)
- Its potential for being a complementary tool in M&E of other EU initiatives (e.g. EIP-Agri, LEADER)
- The possibility to use it in self-evaluation processes (e.g., LEADER Community Led Local Development implemented by LAGs)
- The full integration of both qualitative and quantitative approaches, tools and information
It is empirically tested to be a flexible tool for different users, allowing evaluators to analyse the different stages, elements and aspects of social innovation according to their objectives.
