Final Project Terminal Evaluation for Climate Risk Finance for Sustainable and Climate Resilient Rain-fed Farming and Pastoral Systems

In accordance with UNDP and GEF M&E policies and procedures, all full and medium-sized UNDP support GEF financed projects are required to undergo a terminal evaluation upon completion of implementation. These terms of reference (TOR) sets out the expectations for a Terminal Evaluation (TE) of the project Climate Finance in Sudan.

Terminal Evaluation Terms of Reference

Evaluation cross-cutting issues sample questions

Human rights

• To what extent have poor, indigenous and physically challenged, women and other disadvantaged and marginalized groups benefited from the work of UNDP in the country?

Gender equality

• To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?
• Is the gender marker data assigned to this project representative of reality?
• To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

Project finance / cofinance

The Evaluation will assess the key financial aspects of the project, including the extent of co-financing planned and realized. Project cost and funding data will be required, including annual expenditures.  Variances between planned and actual expenditures will need to be assessed and explained.  Results from recent financial audits, as available, should be taken into consideration. The evaluator(s) will receive assistance from the Country Office (CO) and Project Team to obtain financial data in order to complete the co-financing table below, which will be included in the terminal evaluation report. 

Mainstreaming

UNDP supported GEF financed projects are key components in UNDP country programming, as well as regional and global programmes. The evaluation will assess the extent to which the project was successfully mainstreamed with other UNDP priorities, including poverty alleviation, improved governance, the prevention and recovery from natural disasters, and gender.

Impact

The evaluators will assess the extent to which the project is achieving impacts or progressing towards the achievement of impacts. Key findings that should be brought out in the evaluations include whether the project has demonstrated: a) verifiable improvements in climate risk financing, b) verifiable reductions in impact of climate change and/or c) demonstrated progress towards these impact achievements.[1]

Conclusions, recommendations & lessons

The evaluation report must include a chapter providing a set of conclusions, recommendations and lessons. 

Implementation arrangements (Management and Implementation arrangements)

The principal responsibility for managing this evaluation resides with the UNDP CO in Sudan. The UNDP CO will contract the evaluators and ensure the timely provision of per diems and travel arrangements for the evaluator. The Project Team will be responsible for liaising with the Evaluators team to set up stakeholder interviews, arrange field visits, coordinate with the Government etc. This TOR shall be the basis upon which compliance with assignment requirements and overall quality of services provided by the consultants will be assessed by UNDP.

Location and evaluation timeframe:

The final detailed evaluation workplan will be agreed upon between the UNDP and the selected consultant. The Project evaluation will take place between 15 August and 30 Sep 2020. including a combination of home-based work, and in-country work which includes travel to selected project implementation areas (security situations permitting).  In-country, work including visits to project implementation locations, is subject to COVID-19 operating context.

The consultant is expected to commence the assignment on 15 August2020. The assignment and final deliverable is expected to be completed, no later than 30 Sep 2020, with the detail as described in the below

Evaluation deliverables

The evaluator is expected to deliver the following: *When submitting the final evaluation report, the evaluator is required also to provide an ‘audit trail’, detailing how all received comments have (and have not) been addressed in the final evaluation report.

Team Composition

The evaluation team will be composed of an international and national evaluator. The International consultants shall have prior experience in evaluating similar projects.  Experience with GEF financed projects is an advantage. The evaluator selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

Experience and qualifications required:

• Master  degree in  evaluation , economics or environment related issues;
• Minimum 15 years of relevant professional experience

The Evaluator shall possess:

• Knowledge of UNDP and GEF Previous experience with results-based monitoring and evaluation methodologies. Technical knowledge in the targeted focal area(s)
• Evaluator Ethics

Evaluation consultant will be held to the highest ethical standards and are required to sign a Code of Conduct (Annex E) upon acceptance of the assignment. UNDP evaluations are conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluations’

Payment modalities and specifications

(this payment schedule is indicative, to be filled in by the CO and UNDP GEF Technical Adviser based on their standard procurement procedures) 

Application process

Applicants are requested to apply online by insert new date. Individual consultants are invited to submit applications together with their CV for these positions. The application should contain a current and complete C.V. in English with indication of the e-mail and phone contact. Shortlisted candidates will be requested to submit a price offer indicating the total cost of the assignment (including daily fee, per diem and travel costs).

UNDP applies a fair and transparent selection process that will take into account the competencies/skills of the applicants as well as their financial proposals. Qualified women and members of social minorities are encouraged to apply.

TOR annexes:

• Intervention results framework and theory of change (CRF approved/signed Prodoc)
• Please list the project partners, stakeholders and donors’ names
• Documents to be reviewed and consulted.
• Evaluation matrix template.
• Outline of the evaluation report format required for the evaluation report (example: content and length, etc).
• Code of conduct forms.

The Main objective of the Terminal Evaluation is four-fold: firstly, to assess the results obtained by the project as stipulated by its three outcomes; secondly, to draw lessons learned and identify the best practices; thirdly, to inform about all steps taken so far and those to be taken thereafter in order to ensure/enhance sustainability; and fourthly, put forward recommendations that would guide the implementation of similar projects in the future as well as  better inform the preparation of the Exit Strategy.

The Scope of the Consultancy:

The consultancy will focus on gaging the achievements made and results obtained through rigors assessment of the accomplished status of the three main outcomes of the project. The consultancy will also vision the impact of the project on coverage of insurance, volume of micro financing and compensation being paid to farmers and pastoralists (number and amount). The project’s outcomes stipulate the following:

Outcome 1: Institutional and technical capacity for climate observation, forecasting and early warning strengthened at national and local levels

Outcome 2: Residual climate risk to rural livelihoods in the states of greatest rainfall variability addressed through parametric insurance products.

Outcome 3: Improved access of needy farmers and pastoralists to financial services for climate change adaptation and disaster risk reduction.

The CRF Project is linked and aligned to the UNDP Corporate Strategic Plan Output number 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict:

SDG 13: Take urgent action to combat climate change and its impacts

SDG Target: 13.2 Integrate climate change measures into national policies, strategies and planning

The TE will also dwell on the limitations and challenges that faced the implementation of the project and how these were effectively contained.

valuation approach and method

An overall approach and method for conducting project terminal evaluations of UNDP supported GEF financed projects has developed over time and a lot of knowledge and experience have been gained over time. The evaluator is expected to frame the evaluation effort using the criteria of relevance, effectiveness, efficiency, sustainability, and impact, as defined and explained in the UNDP Guidance for Conducting Terminal Evaluations of UNDP-supported, GEF-financed Projects. A set of questions covering each of these criteria have been drafted and are included with this TOR (Annex C) The evaluator is expected to amend, complete and submit this matrix as part of an evaluation inception report, and shall include it as an annex to the final report. 

UNDP evaluations cover at a minimum the following five major criteria: 

These five evaluation criteria should be further defined through a series of questions covering all aspects of the project intervention, broken out in three main sections: a) project formulation; b) project implementation; and c) project results.  Evaluation questions should be agreed upon among users and other stakeholders and accepted or refined in consultation with the evaluation team.

METHODOLOGY OF THE EVALUATION

An overall approach and methods for conducting UNDP/GEF project terminal evaluations has developed over time, and involves using the following tools, including evaluation criteria and key guiding questions

Relevance

•The extent to which an objective has been achieved or how likely it is to be achieved.

Effectiveness

•The extent to which results have been delivered with the least costly resources possible; also called cost affectedness or efficacy.

Efficiency

•The positive and negative, foreseen and unforeseen changes to and effects produced by a development intervention. •In GEF terms, results include direct project outputs, short to medium-term outcomes, and longer-term impact including global environmental benefits, replication effects and other local effects.

Results’ sustainability

•The likely ability of an intervention to continue to deliver benefits for an extended period of time after completion. •Projects need to be environmentally, as well as financially and socially sustainable

• Were suitable strategies for sustainability developed and implemented?
• To what extent are the project results likely to be sustained in the long-term?
• To what extent did the benefits of a project continue after donor funding ceased?
• What were the major factors which influenced the achievement or non-achievement of sustainability of the project?.

The evaluation must provide evidence-based information that is credible, reliable and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts, in particular the GEF operational focal point, UNDP Country Office, project team, UNDP GEF Technical Adviser based in the region and key stakeholders, particularly the groups of farmers and pastoralists in the project’s targeted states. The evaluator is expected to conduct a field mission to the six states and conduct intensive interviews with the main stakeholders including organizations and individuals.

Impact:

Analyzing the positive and negative changes produced by the Project, directly or indirectly, intended, or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental, and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of social and economic conditions.

• What has happened as a result of the project?
• What real difference has the activities made to the beneficiaries?
• How many people have been affected?

The evaluator will review all relevant sources of information, such as the project document, project reports – including Annual APR/PIR, Community Based Adaptation (CBA) interventions reports, project budget revisions, midterm review, quarterly progress reports, minutes of the Bard meetings, best practices, project files, national strategic on climate change, technical studies and any other materials that the evaluator considers useful for this evidence-based assessment. A list of documents that the project team will provide to the evaluator for review is included in Annex B of this Terms of Reference. 

The methodology should be robust enough to ensure high quality, triangulation of data sources, and verifiability of information. It is expected that the evaluation methodology can include, but would not be limited to the following elements:

• Desk review of project documents, annual reports, MTR and other relevant documents.
• In-depth interviews with key informants such as government officials, and members of local, national, coordination bodies; and questionnaires
• Focus group discussions with the targeted beneficiaries, and Project/UNDP staff.
• Interviews with the project team, and UNDP’s Senior Management.
• Consultations with donors/ international partners and national non-governmental organizations that were directly engaged in project implementation.

Assistance will be provided by UNDP in the identification of key stakeholders, and in organizing the schedule of interviews, focus groups, and site visits.

Key Deliverables:

The Consultancy Firm will produce the following:

• Inception Report detailing the evaluation methodology and includes evaluation matrix with methodology, data collection tools, and data sources for evaluation; within 11 days of starting the assignment
• Draft Evaluation Report to be submitted to UNDP and presentation to the UNDP Team on the draft report outlining the key following aspects: (i) overall findings of the ICRRP Programme; (ii) overall findings and in-depth analysis of each component/outputs; within 32 days of starting the assignment
• Final Evaluation Report (using UNDP Evaluation Report Template/UNDP Outline of the evaluation report format) should be submitted to UNDP no later than two weeks after receiving feedback. All evaluation tools and summary should be annexed to the evaluation report and all stakeholders should be de-briefed on the findings and recommendations: within 36 days of starting the assignment

It should be noted that the above list of deliverables, together with the implementation time-frame  might be subject to review and revision by UNDP in discussion with the consultant in the event of unexpected changes to the context/ working environment in Sudan during the consultancy period, given the operating context of COVID-19.

valuation Criteria & Ratings

An assessment of project performance will be carried out, based against expectations set out in the Project Logical Framework/Results Framework (see  Annex A), which provides performance and impact indicators for project implementation along with their corresponding means of verification. The evaluation will at a minimum cover the criteria of relevance, effectiveness, efficiency, sustainability and impact. Ratings must be provided on the following performance criteria. The completed table must be included in the evaluation executive summary.   The obligatory rating scales are included in  Annex D.

Required Competencies

• Demonstrates commitment to the UN values and ethical standards
• Displays cultural, gender, religion, race, nationality, and age sensitivity and adaptability
• Treats all people fairly and with impartiality.
• Good communication, presentation and report writing skills including proven ability to write concise, readable and analytical reports and high quality academic publications in English;
• Ability to work under pressure and to meet deadlines.
• Flexible and responsive to changes and demands;
• Experience managing a small research team;
• Strong interpersonal skills and ability to work with people from different backgrounds to deliver quality products within a short timeframe;
• Client-oriented and open to feedback

UNDP seeks to recruit Consultant, with a maximum of the following criterion:

• Proven (minimum 3 years) technical expertise and experience in conducting evaluations of GEF project/programme.
• Proven (minimum 3 years) technical expertise and experience in undertaking both qualitative and quantitative research, qualitative and quantitative data collection in hard-to-reach areas, including high-risk and insecure settings context.
• Overall organizational capability which has a direct impact upon implementation of the Project Evaluation (includes Management structure, Management arrangement including quality assurance processes for the scope of work in the ToR, Operational Capacity to arrange logistics, and financial management capacities)
• Previous experience of GEF funded projects evaluations

Education and Experience

• Minimum Master’s degree in evaluation or related filed
• At least 10 years of professional expertise working with International Organizations in evaluating development projects.

Language

• Proficiency in English-language.

Evaluation Criteria and selection of the consultant

The offers received from the candidates will be evaluated using combined scoring method. The combined scoring method assesses the offers with technical merits of the proposals – where the qualifications and methodology will be weighted a maximum of 70%, and later combined with the price offer which will be weighted a max of 30%.

Eligibility Criteria:

MSc Degree or higher in climate change, economic, agriculture, water and related development field; minimum 7 years’ experience in climate change adaptation, weather index, microfinance, water and agricultural strategic frameworks and issues;

Price Proposal:

The price proposals of candidates obtaining 49 points and above (or 70% and above) will only be technically qualified; then their price proposals will be reviewed and compared for the assessment of overall ranking of the proposals. Those obtaining lower than 49 points (or lesser than 70%) will be technically non-responsive proposals; price proposals of such candidate will not be compared.

Assessment of the Price Proposals (30 Points) or 30%

The lowest priced bid from among the technically qualified Offerors will obtain the full marks of 30 points in the price proposal. Price proposals of remaining qualified bidders will be prorated against the lowest priced bid using the following formula to derive the marks in their price proposal:

Marks obtained by a Bidder = Lowest Priced Bid (amount) / Bid of the Offeror (amount) X 30 (Full Marks)

Award of the Contract/Award Criteria:

The contract will be awarded to the candidate (bidder) whose proposal obtains the highest cumulative marks (points) when the marks obtained in technical and price proposals are aggregated together.