This is the Terms of Reference (ToR) for the UNDP-GEF Midterm Review (MTR) of the full -sized project titled Promoting climate-resilient development and enhanced adaptive capacity to withstand disaster risks in Angolan’s Cuvelai River Basin (PIMS #5166) implemented through the Ministry of Environment of the Government of Angola, which is to be undertaken in 2018. The project started on February 11th, 2016 and is in its second year of implementation. In line with the UNDP-GEF Guidance on MTRs, this MTR process was initiated before the submission of the second Project Implementation Report (PIR). This ToR sets out the expectations for this MTR. The MTR process must follow the guidance outlined in the document Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects (http://web.undp.org/evaluation/documents/guidance/GEF/mid-term/Guidance_Midterm%20Review%20_EN_2014.pdf).
Project Background information
The project is funded by the Least Developed Countries Fund (LDCF) to implement a Full-Size Project in Angola, specifically in the region of Cuvelai River Basin (province of Cunene). The project focuses on two of the national priorities presented in Angola’s National Adaptation Programme of Action (NAPA) submitted to the United Nations Framework Convention on Climate Change (UNFCCC), namely priorities 7 (Create an early warning system for flooding and storms) and 13 (Climate monitoring and data management system).
Angola, particularly the Southern region of Cunene Province, is vulnerable to increasing frequency and severity of droughts, floods and severe storms and these events impact sectors such as agriculture and livestock, water resources, rural development and food security, as well as soil erosion, built infrastructures and livelihoods. Of particular concern are the Cuvelai River Basin communities and sectors such as agriculture, livestock and water resources which are an important component of the economy in the region and form the basis of rural livelihoods in Cuvelai Basin.
The development of the Province of Cunene’s capacity to adapt to climate-related hazards is therefore an urgent priority to mitigate the negative impacts of climate change and address the region’s socio-economic and developmental challenges effectively. A large proportion of Cunene Province’s population is ill-equipped to adapt to climate change. Climate change impacts are likely to be particularly negative on Cuvelai’s rural population because of their high dependence on rain-fed agriculture and natural resource-based livelihoods. One way to support effective adaptation planning – in particular for an increase in intensity and frequency of droughts, floods and severe storms – is to improve climate monitoring and early warning systems. For Angola’s region of the Province of Cunene to improve the management of these climate-related hazards it is necessary to:
- Enhance the capacity of hydro-meteorological services and networks to predict climatic events and associated risks;
- Develop a more effective and targeted delivery of climate information including flood and drought forecast early warnings;
- Build skilled human resources to guarantee long-term sustainability of hydro-meteorological services and the Flood Forecasting and Early Warning System;
- Support improved and timely responses to forecasted climate-related risks by strengthening the capacity of the Civil Protection Services; and
- Strengthen the technical capacity of the agriculture extension services to increase resilience of smallholder farmer communities in the Basin.
Barriers that need to be overcome to establish an effective FFEWS in the Province of Cunene and promote climate-resilient development to enhance adaptive capacity of Communities to withstand disaster risks include the following: i) limited knowledge and capacity to fully assess risks posed by climate change to disaster risks in the Province of Cunene; ii) lack of capacity of the extension network to enhance responsiveness and adaptability of subsistence agriculture in the Province of Cunene; and iii) poor intersectoral coordination and weak policy framework to respond to change risks.
Other obstacles in the path include obsolete and inadequate weather and climate monitoring infrastructure, which limits data collection, analysis and provision of meteorological and hydrological services and the absence of an operational Climate Change Environmental Information System in Angola to allow systematic storage and mainstreaming of digital information to support decision making in sector planning. This LDCF-financed project, implemented by the Ministry of Environment, is:
- Enhancing the capacity of national and local hydro-meteorological services, civil authorities and environmental institutions to monitor extreme weather and climate change in the Province of Cunene;
- Increasing the resilience of smallholder farmer communities in the Basin to climate-induced risks and variabilities via access to locally-appropriate climate data and germplasm resources;
- Strengthening local institutional capacities for coordinated, climate-resilient planning; and
- Improving the capacity for effective community-based climate change adaptation (including traditional knowledge practices) at local level.
The project is articulated around three components:
- Component 1: Transfer of appropriate technologies and related capacity building for climate and environmental monitoring infrastructure;
- Component 2: Enhanced human and institutional capacity for increased sustainable rural livelihoods among those communities’ areas most prone to extreme weather events (flooding and drought) in the region;
- Component 3: Increased understanding of climate change adaptation and practices in climate-resilient development planning at the local community and government levels.
The Project duration is 4 years starting on 11 February 2016 and ending on 10 January 2020 with an overall budget of US$8,200,000 and co-financed by UNDP ($517,000 (cash) + $400,000 (in-kind); MINAMB ($2,000,000); MINEA – PIP ($39,037,712); MINEA-NDHR ($1,000,000); INAMET ($968,292); USAID ($1,800,000); DWA ($950,000).
The project is nationally implemented (NIM) by the Ministry of Environment (MINAMB) with UNDP Country Office support, in line with the Standard Basic Assistance Agreement (SBAA of 18 February 1977) and the UNDP Country Programme Action Plan (CPAP 2009-2013 of 14 May 2009) signed between the UNDP and the Government of Angola. The project is implemented in close collaboration with the Government of the Cunene Province, the Ministry of Energy and Water through the National Institute of Water Resources (INRH) and the Cabinet for the Administration of the Cunene River Basin (GABHIC), the Ministry of Telecommunications and Information Technologies (MTTI ) through the National Institute for Meteorologist (INAMET), the Ministry of Interior through the Civil Protection (SPCB), the Ministry of Agriculture (MINAGRI) through the Agrarian Development Institute (IDA), the Institute of Agronomic Research (IIA), The University Agostinho Neto through the Center for Phytogenetic Resources (CRF), and the Center for Tropical Ecology and Climate Change (CETAC).
Objectives of the MTR:
The MTR will assess progress towards the achievement of the project objectives and outcomes as specified in the Project Document, and assess early signs of project success or failure with the goal of identifying the necessary changes to be made in order to set the project on-track to achieve its intended results. The MTR will also review the project’s strategy, its risks to sustainability.
MTR approach & methodology
The MTR must provide evidence based information that is credible, reliable and useful. The MTR consultant will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Environmental & Social Safeguard Policy, the Project Document, project reports including Annual Project Review/PIRs, project budget revisions, lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based review). The MTR consultant will review the baseline GEF focal area Tracking Tool UNDP-GEF MTR ToR Standard Template 1 for UNDP Procurement Website 4
submitted to the GEF at CEO endorsement, and the midterm GEF focal area Tracking Tool that must be completed before the MTR field mission begins.
The MTR consultant is expected to follow a collaborative and participatory approach1 ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), the UNDP Country Office(s), UNDP-GEF Regional Technical Advisers, and other key stakeholders.
Engagement of stakeholders is vital to a successful MTR.2 Stakeholder involvement should include interviews with stakeholders who have project responsibilities, including but not limited to MINAMB; INRH; GABHIC; INAMET; SPCB; IDA; IIA; CRF; CETAC; executing agencies, senior officials and task team/ component leaders, key experts and consultants in the subject area, Project Board, project stakeholders, academia, local government and CSOs, etc. Additionally, the MTR consultant is expected to conduct field missions to Ondjiva town (Cunene Province, Angola), including project sites (i.e. Cuvelai, Cunhama and Namacunde municipalities).
The final MTR report should describe the full MTR approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses? about the methods and approach of the review.
Detailed scope of the MTR:
The MTR team will assess the following four categories of project progress. See the Guidance for Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for extended descriptions.
- Review the problem addressed by the project and the underlying assumptions. Review the effect of any incorrect assumptions or changes to the context to achieving the project results as outlined in the Project Document.
- Review the relevance of the project strategy and assess whether it provides the most effective route towards expected/intended results. Were lessons from other relevant projects properly incorporated into the project design;
- Review how the project addresses country priorities. Review country ownership. Was the project concept in line with the national sector development priorities and plans of the country (or of participating countries in the case of multi-country projects);
- Review decision-making processes: were perspectives of those who would be affected by project decisions, those who could affect the outcomes, and those who could contribute information or other resources to the process, taken into account during project design processes.
- Review the extent to which relevant gender issues were raised in the project design. See Annex 9 of Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for further guidelines. If there are major areas of concern, recommend areas for improvement.
- Undertake a critical analysis of the project’s logframe indicators and targets, assess how “SMART” the midterm and end-of-project targets are (Specific, Measurable, Attainable, Relevant, Time-bound), and suggest specific amendments/revisions to the targets and indicators as necessary.
- Are the project’s objectives and outcomes or components clear, practical, and feasible within its time frame.
- Examine if progress so far has led to, or could in the future catalyse beneficial development effects (i.e. income generation, gender equality and women’s empowerment, improved governance etc…) that should be included in the project results framework and monitored on an annual basis.
- Ensure broader development and gender aspects of the project are being monitored effectively. Develop and recommend SMART ‘development’ indicators, including sex-disaggregated indicators and indicators that capture development benefits.
Progress Towards Results
Progress Towards Outcomes Analysis:
- Review the logframe indicators against progress made towards the end-of-project targets using the Progress Towards Results Matrix and following the Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects; colour code progress in a “traffic light system” based on the level of progress achieved; assign a rating on progress for each outcome; make recommendations from the areas marked as “Not on target to be achieved” (red).
In addition to the progress towards outcomes analysis:
- Compare and analyse the GEF Tracking Tool at the Baseline with the one completed right before the Midterm Review.
- Identify remaining barriers to achieving the project objective in the remainder of the project.
- By reviewing the aspects of the project that have already been successful, identify ways in which the project can further expand these benefits.
Project Implementation and Adaptive Management
- Review overall effectiveness of project management as outlined in the Project Document. Have changes been made and are they effective. Are responsibilities and reporting lines clear. Is decision-making transparent and undertaken in a timely manner. Recommend areas for improvement.
- Review the quality of execution of the Executing Agency/Implementing Partner(s) and recommend areas for improvement.
- Review the quality of support provided by the GEF Partner Agency (UNDP) and recommend areas for improvement.
- Review any delays in project start-up and implementation, identify the causes and examine if they have been resolved.
- Are work-planning processes results-based? If not, suggest ways to re-orientate work planning to focus on results.
- Examine the use of the project’s results framework/ logframe as a management tool and review any changes made to it since project start.
Finance and co-finance:
- Consider the financial management of the project, with specific reference to the cost-effectiveness of interventions.
- Review the changes to fund allocations as a result of budget revisions and assess the appropriateness and relevance of such revisions.
- Does the project have the appropriate financial controls, including reporting and planning, that allow management to make informed decisions regarding the budget and allow for timely flow of funds.
- Informed by the co-financing monitoring table to be filled out, provide commentary on co-financing: is co-financing being used strategically to help the objectives of the project. Is the Project Team meeting with all co-financing partners regularly in order to align financing priorities and annual work plans.
Project-level Monitoring and Evaluation Systems:
- Review the monitoring tools currently being used: Do they provide the necessary information. Do they involve key partners. Are they aligned or mainstreamed with national systems and UNDP requirements. Do they use existing information.
- Are they efficient. Are they cost-effective.? Are additional tools required? How could they be made more participatory and inclusive.
- UNDP-GEF MTR ToR Standard Template 1 for UNDP Procurement Website 10
- Examine the financial management of the project monitoring and evaluation budget. Are sufficient resources being allocated to monitoring and evaluation? Are these resources being allocated effectively
- Project management: Has the project developed and leveraged the necessary and appropriate partnerships with direct and tangential stakeholders.
- Participation and country-driven processes: Do local and national government stakeholders support the objectives of the project. Do they continue to have an active role in project decision-making that supports efficient and effective project implementation.
- Participation and public awareness: To what extent has stakeholder involvement and public awareness contributed to the progress towards achievement of project objectives.
- Assess how adaptive management changes have been reported by the project management and shared with the Project Board. Assess how well the Project Team and partners undertake and fulfil UNDP/GEF reporting requirements (i.e. how have they addressed poorly-rated PIRs, if applicable.)
- Assess how lessons derived from the adaptive management process have been documented, shared with key partners and internalized by partners.
- Review internal project communication with stakeholders: Is communication regular and effective? Are there key stakeholders left out of communication.? Are there feedback mechanisms when communication is received.? Does this communication with stakeholders contribute to their awareness of project outcomes and activities and investment in the sustainability of project results.
- Review external project communication: Are proper means of communication established or being established to express the project progress and intended impact to the public (is there a web presence, for example? Or did the project implement appropriate outreach and public awareness campaigns.)
- For reporting purposes, write one half-page paragraph that summarizes the project’s progress towards results in terms of contribution to sustainable development benefits, as well as global environmental benefits.
- Validate whether the risks identified in the Project Document, Annual Project Review/PIRs and the ATLAS Risk Management Module are the most important and whether the risk ratings applied are appropriate and up to date. If not, explain why.
In addition, assess the following risks to sustainability:
Financial risks to sustainability:
- What is the likelihood of financial and economic resources not being available once the GEF assistance ends (consider potential resources can be from multiple sources, such as?UNDP-GEF MTR ToR Standard Template 1 for UNDP Procurement Website 11? the public and private sectors, income generating activities, and other funding that will be adequate financial resources for sustaining project’s outcomes).
Socio-economic risks to sustainability:
- Are there any social or political risks that may jeopardize sustainability of project outcomes? What is the risk that the level of stakeholder ownership (including ownership by governments and other key stakeholders) will be insufficient to allow for the project outcomes/benefits to be sustained.
- Do the various key stakeholders see that it is in their interest that the project benefits continue to flow.? Is there sufficient public / stakeholder awareness in support of the long term objectives of the project? Are lessons learned being documented by the Project Team on a continual basis and shared/ transferred to appropriate parties who could learn from the project and potentially replicate and/or scale it in the future.
Institutional Framework and Governance risks to sustainability:
- Do the legal frameworks, policies, governance structures and processes pose risks that may jeopardize sustenance of project benefits. While assessing this parameter, also consider if the required systems/ mechanisms for accountability, transparency, and technical knowledge transfer are in place.
Environmental risks to sustainability:
- Are there any environmental risks that may jeopardize sustenance of project outcomes
Conclusions & Recommendations:
- The MTR consultant will include a section of the report setting out the MTR’s evidence-based conclusions, in light of the findings.8
- Recommendations should be succinct suggestions for critical intervention that are specific, measurable, achievable, and relevant. A recommendation table should be put in the report’s executive summary. See the Guidance For Conducting Midterm Reviews of UNDP-Supported, GEF-Financed Projects for guidance on a recommendation table.
- The consultant should provide specific findings, lessons learned and recommendations for accelerating the implementation of the project and for ensuring that project deliverables can be achieved by the end of the project.
- The MTR team should make no more than 15 recommendations total.
- The MTR team will include its ratings of the project’s results and brief descriptions of the associated achievements in a MTR Ratings & Achievement Summary Table in the Executive Summary of the MTR report. See Annex E for ratings scales. No rating on Project Strategy and no overall project rating is required.
- The total duration of the MTR will be approximately 30 days over a time period of 12 of weeks starting September, and shall not exceed five months from when the consultant(s) are hired.
- The principal responsibility for managing this MTR resides with the Commissioning Unit. The Commissioning Unit for this project’s MTR is UNDP Angola.
- The commissioning unit will contract the consultants and ensure the timely provision of per diems and travel arrangements within the country for the MTR consultant. The Project Team will be responsible for liaising with the MTR consultant to provide all relevant documents, set up stakeholder interviews, and arrange field visits.
- An independent consultant will conduct the MTR – (with experience and exposure to projects and evaluations in other regions globally). The consultant cannot have participated in the project preparation, formulation, and/or implementation (including the writing of the Project Document) and should not have a conflict of interest with project’s related activities.