DOC

GUIDELINES FOR AN EX ANTE EVALUATIONS OF RURAL DEVELOPMENT

By Ricky Peters,2014-05-17 00:37
9 views 0
The analysis of Strengths, Weaknesses, Opportunities and Threats (SWOTThe SWOT analysis refers to the main structural components of rural areas,

Guidance note C Ex-ante evaluation guidelines including SEA

    This guidance note presents the guidelines for an ex-ante evaluation of rural development

    programmes 2007 2013 including the requirements for the Strategic Environmental

    Assessment (SEA).

    The purpose of the guidelines are:

    ? To provide a synthetic operational tool to draw-up and carry out ex-ante evaluation of rural

    development programmes

    ? To give an overview of the methodological framework for ex-ante evaluation;

    ? To clarify the role of ex-ante evaluation for rural development programmes

    ? To explain the requirements for and the specific tasks of ex-ante evaluation

    ? To clarify the requirements concerning the Strategic Environmental Assessment

    55062071.doc

    Table of Content

    1. Introduction ................................................................................................................ 3 2. The Regulation on support for Rural Development .................................................. 3

    3. Methodological framework of ex ante evaluation ..................................................... 4

    3.1. The intervention logic ................................................................................................ 5

    3.2. Indicators ................................................................................................................... 6 4. The regulatory framework for ex ante evaluation ..................................................... 7

    5. Roles and responsibilities .......................................................................................... 8

    6. Specific tasks of ex ante evaluation ........................................................................... 8

    6.1. Assessment of the programme-related SWOT analysis ............................................ 8

    6.2. The assessment of programme targets ....................................................................... 9

    6.3. The assessment of expected impacts ....................................................................... 10

    6.4. The assessment of proposed implementation procedures, including monitoring,

    evaluation and financial management ..................................................................... 11

    7. Reporting ................................................................................................................. 11 Annex 1: Indicative Outline of an Ex-ante Evaluation Report ................................................ 12

    Annex 2: Ex ante Evaluation and Strategic Environmental Assessment ................................. 14

     2

1. Introduction

    Council Regulation 1698/2005 on support for rural development by the European

    Agricultural Fund for Rural Development - EAFRD provides the legal framework for the

    preparation and the implementation of rural development programmes in the Member States

    for the period 2007 2013. Following Art 16 and 85 of Council Regulation 1698/2005, ex ante evaluation is an obligatory task in establishing a rural development programme for a

    geographical region concerned.

    The purpose of these guidelines is to provide Member States with a synthetic operational tool

    to draw up and carry out ex-ante evaluation of rural development programmes being a first

    step within the “Common Monitoring and Evaluation Framework”. Ex-ante evaluation aims

    at improving programmes and contributing to capacity building for future monitoring and

    evaluation activities.

    These guidelines will be part of the “Handbook of the Common and Evaluation Framework”

    that will provide Member States with a full set of guidance documents covering monitoring,

    ex ante evaluation and ongoing evaluation.

    2. The Regulation on support for Rural Development

    The main changes of rural development policies relevant for ex ante evaluation are:

    1. A thorough simplification of policy implementation through the introduction of a

    single funding system, and the modification of programming, financial management

    and control framework for rural development programmes;

    2. The definition of three core objectives for rural development measures (Article 4):

    ? Improving the competitiveness of agriculture and forestry by supporting for

    restructuring, development and innovation;

    ? Improving the environment and the countryside by supporting land management;

    ? Improving the quality of life in rural areas and encouraging diversification of

    economic activity.

    A thematic axis corresponds to each core objective, around which rural development programmes have to be built, whilst a fourth horizontal and methodological axis is dedicated

    to the mainstreaming of the LEADER approach.

    The hierarchy of objectives inherent in Council Regulation 1698/2005 identifies the main

    priorities for rural development at Community level. It is schematically presented in the

    following chart:

     3

The Member States have to conceive their rural development strategies in the light of this

    hierarchy of objectives and, based on the analysis of their own situation, they choose which

    measures are the most appropriate ones to implement each specific strategy. Rural

    development programmes will then translate the strategy into action through the

    implementation of these measures, which are foreseen into the four operational axes (Articles

    20, 36, 52, and 63 of Council Regulation 1698/2005).

    3. Methodological framework of ex ante evaluation

    Ex ante evaluation supports the preparation of proposals for new or renewed community

    actions. Its purpose is to gather information and to carry out analyses which help to ensure

    that the policy objectives will be delivered successfully, that the measures used are cost-1effective and that reliable evaluation will be subsequently possible. Accordingly, ex ante evaluation in the context of rural development programmes analyses in

    detail for each programme its underlying strategy and objectives - including baselines,

    quantifiable objectives and target levels. It takes place before the implementation of the

    programme, with a view to support programming, and has to be designed to clearly identify

    needs and development strategies.

     1 Evaluating EU Activities: a practical guide for the Commission Services, Directorate General for the Budget, July 2004, page 12.

     4

    3.1. The intervention logic

    An intervention logic represents a methodological instrument which establishes the logical

    link between programme objectives and the envisaged operational actions. It show the

    conceptual link from an intervention’s input to its outputs and, subsequently, to its results and impacts. Thus the intervention logic allows an assessment of a measure’s contribution to achieving its objectives.

    The different elements of an intervention logic are shown in the following diagram:

    Needs

    Overall Impacts

    objectives

    Specific Results

    objectives

    Operational Outputs

    objectives

    Inputs

Needs relate to socio-economic or environmental requirements to which the programme

    should respond. The inputs are financial and/or administrative resources, which will generate

    the outputs and will achieve the intended operational objectives (or measure objectives) through programme activities. The subsequent results are the most immediate impacts of the

    assistance, in other words the contribution of the operational objectives to the specific

    objectives.

    For example, in the framework of an intervention aiming at improving a production technique

    in a given area, investments on equipment are examples of output, and the availability of these

    equipments to assisted farms would be considered as the results. The impacts derive from the

    results; in the same example, an intermediate impact might be a higher productivity and/or a

    positive impact on the environment in the area of intervention.

    Overall impacts are linked to the overall objectives of the programme and, in a well designed

    programme, they meet the previously identified needs that led to the implementation of the

    programme. For the above-mentioned example, an overall impact might be a higher level of

     5

agricultural income or the improvement of competitiveness of farms due to the availability of

    more efficient production systems.

    3.2. Indicators

    Ideally, indicators established to measure a programmes progress towards the achievement

    of pre-defined targets have to be specific, measurable, available/achievable in a cost effective

    way, relevant for the programme, and timely available.

    Ex ante evaluation has to include an assessment of indicators proposed for measuring success

    of respective rural development programmes. Ex ante evaluation will assess the coherence

    between these indicators and the programme’s targets and ensure that indicators are applied and filled with data in a meaningful manner in order to form a useful basis for monitoring

    and evaluation.

    The Rules for the Application of Council Regulation 1698/2005 define a list of common

    baseline indicators which reflect Community priorities and objectives. In addition to the

    common baseline indicators, programme-specific baseline indicators shall be defined, at

    programme level, in view of covering specific national or regional needs, as well as specific

    national or programme-related priorities.

    For both the common and the programme-specific baseline indicators, two types of indicators

    are distinguished:

    First, context-related baseline indicators are used to describe the general circumstances of the geographical area covered by the programme. They reflect the socio-economic situation,

    environmental aspects, or the agricultural structure, and are not directly addressed by the

    programme. They provide relevant information to assess the weaknesses and strengths of the

    respective region(s) and concern drivers that may have either a supportive or adverse effect

    on achieving a programme’s objectives. Second, objective-related baseline indicators are used to provide relevant information on the initial situation of the parameters that the programme intends to change. These indicators

    represent the baseline for the measurement of progress with respect to achieving the

    programme targets.

    Establishing baseline indicators means to measure the situation relevant to the targeted

    parameters at the beginning of the programme; the change in the baseline indicators over

    time will thus provides a reference for the assessment of effects (impacts) of the interventions

    by objective. Objective-related indicators relate to pre-defined target levels, which in turn

    represent the expected change of the starting situation of a given baseline indicator.

    In addition, the “Rules for Application of Council Regulation 1698/2005” foresee seven

    common impact indicators to assess the impact of measures with respect to explicit objectives established by the European Council and the Strategic Guidelines for rural

    development. The impact of both single measures and the programme as a whole should be

    assessed against these seven indicators reflecting the contribution in terms of growth, jobs,

    productivity, biodiversity, high nature value areas, water and climate change. These common

    impact indicators must, again, be complemented by programme-specific impact indicators.

     6

4. The regulatory framework for ex ante evaluation

    As stipulated by Article 85 of Council Regulation 1698/2005,

    Ex ante evaluation shall form part of drawing up each rural development programme and

    aim to optimise the allocation of budgetary resources and improve programming quality. It

    shall identify and appraise:

    ? the medium and long term needs;

    ? the goals to be achieved;

    ? the results expected;

    ? the quantified targets particularly in terms of impact in relation to the baseline

    situation;

    ? the Community value-added;

    ? the extent to which the Community’s priorities have been taken into account;

    ? the lessons drawn from previous programming;

    ? the quality of the procedures for implementation, monitoring, evaluation and

    financial management

    Article 85 of Council Regulation 1698/2005 states that ex ante evaluation shall be carried out under the responsibility of the Member States. Moreover, following Article 84(4) of the same regulation, evaluations shall be carried out by independent evaluators.

    Ex ante evaluation will also verify to what extent the assistance of the EAFRD is consistent

    with the objectives of Economic and Social Cohesion and those of the Community support

    instrument for fisheries in particular (Article 5 of Council Regulation 1698/2005), and the

    extent to which the proposed implementation system could promote equality between men

    and women, and prevent any form of discrimination (Article 8 of Council Regulation

    1698/2005).

    Rural development programmes must duly reflect the strategic approach referred to under the

    Title II of Council Regulation 1698/2005. In this respect, ex ante evaluation will focus on the

    extent to which the respective rural development programmes reflect the priorities defined

    under:

    ? The “Community Strategic Guidelines for Rural Development Policy for the

    programming period from 1 January 2007 to 31 December 2013” (Article 9 of

    Council Regulation 1698/2005) setting out, at Community level, the strategic

    priorities for each axis of rural development programmes;

    ? The National Strategy Plans for Rural Development, established by Member States on

    the basis of the Community Strategic Guidelines, identifying national strategies and

    priorities to be pursued by rural development programmes (Article 11 of Council

    Regulation 1698/2005);

    Ex ante evaluation will also consider the provisions of the Rules for application of Council

    Regulation 1698/2005 which provide detailed information on the implementation of rural

    development programmes.

     7

    As stipulated in the Rules for the application of Council Regulation 1698/2005, the environmental assessment required by Directive 2001/42/EC on the assessment of the effects of certain plans and programmes has to be integrated into ex-ante evaluation.

    5. Roles and responsibilities

    The task of ex ante evaluation is to provide an independent judgement and recommendations on technical and/or policy issues linked to the programme in view of improving and strengthening its quality.

    Ex ante evaluation represents the starting point for ongoing evaluation within the Common Monitoring and Evaluation Framework. Ex ante evaluation will establish the basis for

    effective monitoring, mid-term and ex post evaluations, by ensuring that there are explicit, quantified objectives and appropriate indicators reflecting the strategic and operational objectives of the programme.

    The interaction between programme formulation and ex-ante evaluation should permit the responsible authorities to develop each component of the programme in the light of the input progressively acquired from the ex-ante evaluation. During the different stages of the programming process, the competent authorities can decide which of the recommendations made by ex ante evaluation to take on board, and which are the changes to be introduced to the programme. This interaction may lead to revisions of programmes. For example, the evaluation of the impact of the strategy might lead to a reconsideration of the policy mix and to a revision of the programme strategy.

    Considering its important function to critically review the (draft) programme in relation to strategies and objectives, it is clear that ex-ante evaluation has to be carried out by an ex-ante evaluator who is not directly involved in conceiving, implementing, managing or financing the programme. In any case, validating baselines, assessing target levels, benchmarks and programme-specific indicators are demanding tasks that require expertise in evaluation methodologies. In carrying out these tasks, the experience of professional evaluators will be particularly helpful for programme authorities. Implementing environmental assessment equally requires specific expertise that needs to be covered by independent evaluators. 6. Specific tasks of ex ante evaluation

    6.1. Assessment of the programme-related SWOT analysis The analysis of Strengths, Weaknesses, Opportunities and Threats (SWOT analysis) of the geographical area covered by the programme precedes ex-ante evaluation. It represents the first step to define the strategy of the programme, and aims to bring out the needs of the rural areas concerned. The SWOT analysis refers to the main structural components of rural areas, and has to lead to an appropriate and coherent ranking of disparities that need to be addressed.

    The SWOT analysis and the definition of the programme strategy are based on the use of quantified data and appropriate baseline indicators.

    Ex-ante evaluation has to assess the results of the programme-related SWOT analysis. In this respect Article 85 of Council Regulation 1698/2005 requires to:

     8

    ? identify and appraise the medium and long term needs.

    In carrying out this task, ex ante evaluation has to:

    - assess the completeness of the SWOT analysis;

    - analyse causes of disparities identified;

    - identify and assess the driving forces towards sustainable rural development;

    - contribute to the quantification of context and objective related baseline indicators

    (common and programme-related) by verifying and, where appropriate, suggesting

    modifications of the proposed indicators and figures;

    - assess and, where appropriate, suggest revisions of the ranking of disparities and

    priorities assigned to identified needs and their translation into objectives and

    concrete priorities for action.

    6.2. The assessment of programme targets

    With respect to ensuring consistency of the proposed programme targets with the strategic

    orientations of programmes and National Strategy Plans, Article 85 of Council Regulation

    1698/2005 requires to:

    ? identify and appraise the goals to be achieved;

    ? identify and appraise the results expected;

    ? identify and appraise the lessons drawn from previous programming;

    ? identify and appraise the Community value-added;

    ? identify and appraise the extent to which the Community’s priorities have

    been taken into account.

    In view of fulfilling these tasks, ex ante evaluation has to:

    - assess the programme related targets as derived from needs and the relevance of

    the programme objectives (“the goals to be achieved”) to those needs;

    - identify and examine the intervention logic of measures, from output to result and

    to impact, and to assess the extent to which the expected results of measures

    contribute to achieving the targets identified and to the overall objectives both at

    community and national level;

    - assess the internal coherence of programmes by verifying the balance between the

    operational objectives of the different measures, with a view to verify the

    existence of mutually reinforcing interactions and the absence of possible

    conflicts and contradictions between them;

     9

- take into consideration, where applicable, the lessons learnt from the past

    programming period, in particular concerning best practices, key incidents

    affecting policy implementation, the effectiveness and costefficiency of

    measures, and possible measure-specific problems, as identified in evaluation

    reports or other sources of information.

    - assess the extent to which the expected objectives can be realistically achieved

    with the financial resources allocated to the different measures within each axis;

    - assess the extent to which rural development programmes contribute to achieving

    Community priorities and priorities for intervention defined under the National 2Strategy Plan, in particular with respect to growth, jobs and sustainability;

    - assess the consistency of rural development programmes with the objectives of

    Economic and Social Cohesion, and those of the Community support instrument

    for fisheries;

    - analyse the effects of programmes on the environment by applying Strategic

    Environmental Assessment in line with Directive 2001/42/EC (see Annex II).

    6.3. The assessment of expected impacts

    On the basis of the information contained into the draft programme documents, Article 85 of

    Council Regulation 1698/2005 requires to:

    ? identify and appraise the quantified targets particularly in terms of impact in

    relation to the baseline situation

    In view of fulfilling this task, ex ante evaluation has to:

    - assess whether targets are quantified in a meaningful and verifiable manner,

    allowing subsequent programme monitoring and evaluation, in particular with

    respect to the utility and sustainability of the programme;

    - assess the correct application of common baseline indicators and the usefulness of

    programme-specific baseline indicators as well as programme-specific impact

    indicators reflecting the specific objectives and circumstances of a programme

    concerned;

    - assess expected impacts of measures against baselines; in this respect ex-ante

    evaluation should pay particular attention to the verifiability of the impacts of the

    measures concerned.

    - verify the functioning of data collection mechanisms in view of ensuring a regular

    follow up on the trends during different phases of programme reflected in the

    indicators applied.

     2 Outcomes of ex ante evaluations as well as related recommendations by ex ante evaluators may lead to

    revisions of National Strategy Plans.

     10

Report this document

For any questions or suggestions please email
cust-service@docsford.com