Outcome Evaluation of UNDP’s Environment and Sustainable Development Programme in Ukraine
In line with the Evaluation Plan, UNDP in Ukraine will commission an outcome evaluation to assess the impact of UNDP’s development assistance in the Practice Area of Environment and Sustainable Development. The proposed evaluation will evaluate the country programme outcome(s) in this practice area (Outcome #10: “Government adopts policy frameworks and mechanisms to ensure reversal of environmental degradation; climate change mitigation and adaptation; and prevention of and response to natural and human-caused disasters”) and related outputs implemented under the Ukraine’s Country Programme Document 2012-2016. http://www.ua.undp.org/content/ukraine/en/home/library/poverty/country-programme-document-for-ukraine--2012--2016-.html .
The evaluation is also intended to provide forward looking recommendations to the environment and sustainable development programme in the new cycle of UNDP Ukraine Country Programme (2018-2022).
Ukraine is a middle-income country with a strong industrial base and has traditionally been a source of agriculture and food products. However, its economy took a deep dive after the break-up of the Soviet Union. In the past two decades its population has shrunk by around 8 million since its 1990 peak. Ukraine’s GDP fell sharply over the first 10 years of its independence from the Soviet Union. The economy experienced a deep recession during the 1990s, including hyperinflation and a drastic fall in economic output. In 1999, Ukraine’s per capita GDP was less than half of that achieved before independence. Its share of global GDP has more than halved since the 1980s. In comparative terms, it has performed worse than practically all of its peers among the former Soviet Republics. Its infrastructure remains largely as it was inherited from the Soviet era and suffers from inefficiencies and a lack of maintenance and investments. A period of rapid growth took place between 2000 until 2008, but a 15-20% decrease in GDP took place over 2008 and 2009, as a result of the effects of the global financial crisis. After a period of slight recovery in 2010-2013, since 2014 the country has been experiencing a period of instability and insecurity, marked by a steep economic downturn and political tensions.
In 2014-2015, the annexation of the Autonomous Republic of Crimea and Sevastopol city by the Russian Federation as well as the anti-terrorist operation in some areas of Donetsk and Lugansk oblasts have radically changed Ukraine’s development course. The need has arisen to build defence fortifications, to increase the production of weapons, ammunition and other means of defence, which required upgrowth in output of heavy industry products, metals, cement, etc. Besides, there is a critical need to restore basic infrastructure and access to basic services (reconstruct ruined industrial facilities and infrastructural networks, including railway infrastructure, gas and oil pipelines, water supply systems, sewerage networks, and to repair and build new residential houses and social facilities).
Environment, Energy and Sustainable Development
Soviet policies of raising industrial and agricultural productivity with little regard to environmental considerations have had a devastating effect on the environment.
Ukraine has the unique status of being home to some of the richest natural environments and resources in Europe while at the same time being one of the most heavily polluted countries in the region.
Poor environmental management in the past has resulted in an increased number of natural and man-made disasters in recent years in Ukraine, and worsened the health of the population. At present, 40% of the total territory of Ukraine is now eroded land, and is growing at approximately 100,000 hectares annually. Extensive dehumufication (process of humus loss) of soils, including black soil is observed. Experts value the direct annual damage from erosion as USD 5 bln, and indirect, as a result of crop losses on the eroded soils – USD 1 bln. At the same time, it has many natural assets in biodiversity and international waters of global importance and its industrial activities and energy consumption practices have important implications for global climate change.
Ukraine has been and remains one of the least energy efficient countries in the world having one of the highest greenhouse gas emission intensity amongst CIS countries, and holds the 24-th place amongst the world's largest emitters of greenhouse gasses.
Greenhouse gas (GHG) emissions in Ukraine amounted to 944.4 Mt CO2eq in 1990, and 402.7 Mt CO2eq (excluding LULUCF) in 2012, i.e. 42.6% of the 1990 level. GHG emissions including LULUCF amounted to 874.6 Mt CO2eq in 1990 and 375.4 Mt CO2eq in 2012, i.e. 42.9% of the 1990 level. This reduction resulted mainly from a GDP decrease and a decline in the population and social living standards, which have been expected to be recovered and improved to reach the EU level . It was especially relevant for the period 1991-1999. For the period 2000 -2007 GHG emissions mostly stabilized with the modest growing tendency due to the economic restructuring, and increasing share of the finance and service sectors in GDP. Certain modernization of the industry that lead to decrease of power-intensity and related carbon intensity of the Ukrainian industry. GHG emissions in 2008-2013 were also affected by the world economic crisis.
Ukraine belongs to the list of countries that have signed and ratified the United Nations Framework Convention on Climate Change and its Kyoto Protocol and committed themselves not only to protect the climate system for the benefit of the present and future generations of mankind, but also to fulfill their individual obligations as Parties to the Convention and Protocol. In particular, Ukraine has committed to implement policies and measures aimed at combating the climate change, taking into consideration the real socio-economic conditions of the country, to cover all sources and sinks of greenhouse gases as well as related economy sectors.
In 2015 Ukraine has defined an ambitious target with regard to GHG emissions. Emissions will be limited to 60% of 1990 levels in 2030. However, pledge will be revised after the country’s “territorial integrity” is restored.
Ukraine is also a party to many International Treaties and Conventions in the field of environmental protection.
Ukraine’s National Environmental Policy priorities for 2012-2016 have been defined as “Achieving a state of the environment which is safe for human health; improvement of the environment and increase of the level of ecological safety; improvement of the system of integrated environmental management through the inclusion of an environmental component in economic sector-wide reform programmes ensuring an environmentally sound use of natural resources, increase of public awareness”.
The fundamental document that defines the national priorities in the field of prevention of anthropogenic climate change is the Underlying Principles (Strategy) of the State Environmental Policy of Ukraine for the period until 2020. The Strategy defines tasks that directly or indirectly are aimed at reducing emissions and increasing removals of GHG in the country, including but to limited to the following:
- “... optimizing the structure of the energy sector of the national economy by increasing the use of energy sources with low carbon dioxide emissions by 2015 by 10%, and by 2020 by 20%” (in relation to the baseline in 2010);
- “... reducing greenhouse gas emissions in accordance with the declared by Ukraine international obligations under the Kyoto Protocol to the United Nations Framework Convention on Climate Change”;
“... designing and implementing in stages of the National Action Plan on climate change mitigation and prevention of human impact on climate change for the period up to 2030”;
- “Increasing by 2020 wooded area to 17% of the state land through reforestation and afforestation of the Forestry Foundation’s lands”;
- “Creating of economic conditions for infrastructure development of environmentally friendly modes of transport by 2015”;
- “Improving energy efficiency by 25% by 2015 and by 50% by 2020” (relative to the baseline year 2010);
- “Increasing the use of renewable and alternative energy sources by 25% by 2015 and by 55 % by 2020 against the baseline” (relative to the baseline year 2010) etc.
It should be also noted the target on share of renewable energy sources set by the Energy Community for Ukraine is 11% of primary energy by 2020.
The National plan on implementation of Kyoto Protocol to the UNFCCC. http://zakon5.rada.gov.ua/laws/show/346-2005-%D1%80 should be also taken into account.
Besides, there is a number of other legislative and regulatory acts of both national and regional and sectoral levels, which address issues closely related to the issue of reducing GHG emissions, as well as drafts of such documents. Among these documents the leading role is played by the Energy Strategy of Ukraine for the period until 2030 (the Energy Strategy). Energy Strategy identifies goals and objectives of the state policy in the field of energy conservation and efficiency of use of energy resources in the country; its implementation leads directly to a reduction in GHG emissions in Ukraine. The Energy Strategy also identifies quantitative targets to reduce energy intensity of GDP for the period up to 2030 and the directions of development, which are to ensure the achievement of these targets. In particular, this applies to:
- Accelerated development of renewable energy sources;
- The demand to bring, in a very short period of time (by 2028), powerful plants burning fossil fuels, in compliance with stringent EU regulations on emissions of polluting substances (ash, NOx and SO2);
- Revision of the agreements with the Russian Federation concerning the conditions of import and transit through the territory of Ukraine for Russian gas, etc.
Other important legislative documents include:
- Law of Ukraine “On the Ratification of the Association Agreement between the European Union and the European Atomic Energy Community and their member states, of the one part, and Ukraine, of the other part” dated 16.09.2014 ? 1678 – V??;
- Decree of the President of Ukraine “On the “Ukraine-2020” Sustainable Development Strategy dated 12.01.2015 ? 5/2015;
- Decree of the Cabinet of Ministries of Ukraine “On approval of the National Action Plan on Renewable Energy by 2020” dated 01.10.2014 ? 902-?.
- Protocol on Ukraine joining the Energy Community (http://zakon5.rada.gov.ua/laws/show/994_a27).
Covering more than a half of its energy demand through external supplies and having one of the world’s most energy intensive economies, Ukraine makes persistent efforts towards energy conservation and efficiency of energy production..
Energy saving tasks for all sectors of the economy have received particular priority in connection with recent political developments (armed conflict in eastern Ukraine and political tension with the Russian Federation) and price growth for imported (mainly from Russia) natural gas, which accounts for three fourths of total Ukraine’s gas consumption. This priority is also emphasized in Ukraine’s Energy Strategy for the Period Until 2030, adopted in March 2006.
Ukraine has its own fuel and energy complex, which includes coal mines, oil and gas wells, power plants, and power lines. Enterprises are grouped into industries that are the main elements of the structure (coal industry, oil and gas industry, and electrical energy industry). The peculiarity of the fuel and energy balance of Ukraine is the high proportion of coal and nuclear power, and small of hydroenergy, oil and renewables.
The electric power is the basic industry of Ukraine, which completely supplies the country with electric energy. The excess capacity enables to export a small portion of the energy generated (2-5%) to the neighbouring (Russian Federation, Belarus, Moldova) as well as to European countries. The oil industry of Ukraine is characterized by low production figures, although the potential possibilities for oil production and refining are much more. Ukrainian oil has a relatively high production cost due to the use of outdated technologies. To date, with the needs of 40 million tons of crude oil per year the own production is only about 2.5 million tons. That’s why Ukraine has to import the missing oil and oil products (mainly from Russian Federation).
Ukraine’s gas industry is a relatively young and promising, however, lacking capital investments. Ukraine has a unique gas transportation and gas storage system (inherited from the Soviet Union) and would like to become a stable platform for signing long-term contracts on gas supplies from the Russian Federation to the EU, not to mention the increase in domestic production of natural gas, which in its turn has a positive effect on the energy balance of the country.
Currently Ukraine is trying to reduce its dependence on imports by diversifying suppliers, switching to other energy sources (for example, the use of biomass) as well as increase its own production volumes. However, there is a number of legislative, financial and other barriers challenging and delaying the implementation of the energy security strategy.
Housing Sector and Related Infrastructure
The energy efficiency (EE) in communal sector in Ukraine is on average approximately four times lower than that in the Western European countries.
The Ukrainian housing and communal sectors are highly inefficient in energy terms, consuming 44% of all energy resources for the country. According to official statistics, the biggest waste of gas is related to the central heating system – 22% of the heat wasted during production, 25% during transportation and 30% during distribution, including by the end users. Ukraine’s outdated energy infrastructure is in need of major upgrades. The country’s thermal power stations are on average 40 years old and mostly have not been rehabilitated since they were built. The same goes for the transition and distribution networks where 40-50% energy losses are common.
The current state of heat and hot water supply systems has led to public discontent over dramatically raised tariffs for unsatisfactory heat supply services (e.g., low temperatures, lack of quantitative and qualitative regulation). Among the major sectoral problems are massive deterioration of the housing stock; poor energy performance of residential buildings; legal constraints for energy efficiency improvements; insufficient management and maintenance as well as lack of financial resources for maintaining of the infrastructure.
Overall, the economy of Ukraine requires significant structural changes, infrastructural development, technological modernization and recovery after military operations in eastern Ukraine. Consideration of sustainable development factor in the planning and implementation of reforms provides for addressing new policies. Ambitiousness of stated CO2 emissions reduction target envisages considerable efforts to substantially prevent increase of GHG emissions under the planned structural changes, restoration and development of infrastructure, including the post-conflict reconstruction. All these actions will require development and implementation of efficient and effective policies as well as significant financial investments.
Land Degradation and Desertification
The principal document that defines the national priorities in the field is the National Plan to Combat Land Degradation and Desertification, endorsed by the decision of the Cabinet of Ministers of Ukraine on 30.03.2016 ? 271-?.
UNDP’s Response to Developmental Challenges
UNDP has been active in Ukraine since 1993 and is one of the largest international development organizations on the ground, with 183 national and international staff working all around Ukraine and 40 staff in its Kyiv office. UNDP Ukraine has established a programming presence in each administrative district of Ukraine and has a strong experience of implementing environmental projects contributing to UNDP’s longstanding work on energy innovations and environmental policies and will help Ukraine to become a “green and clean” and energy-efficient country, moving forward on its environmentally sustainable development path and successfully addressing climate change mitigation on the policy level through advocacy and policy advice, and through practical initiatives, and will support local energy efficiency initiatives and community organizations protecting the environment.
In recent years UNDP in Ukraine, largely through GEF, provides assistance in the following key areas: policy development and public awareness; climate change, energy and energy efficiency; sustainable development; chemicals; biodiversity; international waters management; and disaster risk reduction and emergencies.
Being deeply involved in the Ukrainian national environment policy process over the last decade, resulted, in particular, in the ratification of the Kyoto protocol, development of the “National Environmental Policy of Ukraine” (2007) and further of the Law of Ukraine “On fundamental principles (strategy) of the State Environmental Policy till 2020” (2010), UNDP continues supporting Ukraine in translating the principles of sustainable development into the policy agenda and plans of action for the national and local level governments.
As new challenges emerged in 2014-2015, the UNDP Energy and Environment programme targeted its efforts on the enhancement of resilience, security and competitiveness of Ukraine through improved energy efficiency and sustainable use of natural resources. While known as a long-standing challenge, this area falls under the priorities of the newly elected government and is the key to the overall stabilization of the economy.
UNDP has been strengthening the Ukrainian government’s overall institutional capacity to design and implement sustainable development policies and practices through the development of appropriate decision-making tools and integration of the provisions of Rio Conventions into national policy framework.
UNDP supported advocacy and policy advice on climate change mitigation including assistance in development and national dialog of the long-term Concept of National Low Carbon Growth Strategy. A proactive position of Ukraine in the global climate negotiation (the United Nations Climate Change Conference 2015 in Paris) including identification of the new ambitious target for GHG emission reduction has been ensured through the support in development of Intended Nationally Determined Contributions (INDC).
The issue of energy intensity, as vitally important for the national security of Ukraine, has remained the priority focus of UNDP. It was addressed through promotion and introduction of a wide spectrum of energy efficiency policies and practices, mostly in the communal sector. National scale introduction of the energy efficient lighting in public and residential buildings done in line with UNDP supported “Road-Map for transition to Energy Efficient Lighting“ has led to significant decrease of watt-hour energy usage and related decrease of Green House Gas emissions.
Certain progress in reduction of energy, resource and carbon intensity of Ukrainian municipal economy is envisaged due to accelerated utilization of the agricultural biomass for municipal heat and hot water services, actively pursued by UNDP. In case of massive replication, utilization of cheap and accessible energy from agricultural wastes will assist Ukraine with diversification of its energy supply away from dependence on imported fossil fuels as well as with creation of new green jobs, income and revenue streams in both urban and rural territories.
Reacting to the new challenges in Ukraine related to resilience, energy security and climate change UNDP has focused its efforts on development and securing of funds for the new projects promoting green and energy efficient cities and renewable energy.
Purpose of the Evaluation
The purpose of this outcome-level evaluation is to find out how UNDP in Ukraine has gone about supporting processes and building capacities that have, indeed, helped make a difference, and whether and to what extent the planned outcomes have been or are being achieved as a result of UNDP’s work in the area of Environment and Sustainable Development covering the period 2012-2016. In doing so, evaluation aims to identify which UNDP approaches have worked well and which have faced challenges, and to use lessons learned to improve future initiatives and generate knowledge for wider use.
Specifically, the outcome evaluation will assist UNDP in gaining a better understanding of the following aspects of its interventions:
- Status of outcome(s) (i.e. the extent to which the planned outcomes and the related outputs have been or are being achieved);
- Relevance of outcome/outputs;
- The mechanisms by which outputs led to the achievement of the specified outcomes;
- Concrete evidence of the UNDP contribution to outcomes;
- If and which programme processes e.g. strategic partnerships and linkages are critical in producing the intended outcome (i.e. partnership);
- Factors affecting the outcome(e) (i.e. factors that facilitate and/or hinder the progress in achieving the outcome, both in terms of the external environment and those internal to the portfolio interventions including: design, management, human resource skills, and resources;
- Lessons learned from the implementation of the interventions, as also evidenced, and relevant for next Country Programme formulation;
- Strategic positioning of UNDP;
- Coordination and mutual reinforcement of the inputs, results and outputs of the projects, their integration into larger governance objective of UNDP Ukraine , their alignment and synergies;
- Sustainability: whether there is ownership and capacity to maintain and manage development in the Outcome.
To the extent possible, answers to the above questions shall address the implications for women and men, their participation in design and implementation of the outcome and particular programmes and projects in the outcome area, whether the latter had addressed the issues of gender inclusion, equality and empowerment and contributed to strengthening the application of these principles to various development efforts in the country, and how gender issues had been mainstreamed across the outcome area by UNDP. Evaluation shall also address the extent to which UNDP had advocated for the principle of equality and inclusive development, and has contributed to empowering and addressing the needs of the disadvantaged and vulnerable population.
The recommendations and lessons from this outcome evaluation will feed into the planning process of the next UNDP Country Programme cycle 2018-2022.
Duties and Responsibilities
The following Outputs falling under this Outcome, as stated in UNDP CPD 2012 – 2016, are to be part of this evaluation:
The outcome to be assessed is whether “Government adopts policy frameworks and mechanisms to ensure reversal of environmental degradation; climate change migration and adaptation; and prevention of and response to natural and human-caused disasters”. The outcome indicator is the percent of natural and subnational government bodies that integrate environment, DRR and climate change in development and management plans.
The output that falls under this outcome is the capacity of partners strengthened to design and implement measures on climate change, energy efficiency and DRR, including at the local level. The output indicators are (1) the number of active green investment schemes and energy efficiency projects and (2) the DRR platform. The baseline is in the year 2010, and include 0 green investment schemes (GIS), 250 energy efficiency projects; and no DDR platform. The corresponding targets are 100 GIS, 500 energy efficiency projects; and DRR platform functions.
There are 2 related UNDAF Outcomes. They are as follows:
- Outcome 2: Reduced energy, resource and carbon intensity of economy through the application of energy efficient technologies, renewable and alternative sources of energy.
- Outcome indicator: Climate change risks and opportunities integrated in the agricultural and rural development policy of Ukraine
- Baseline (2010): No
- Target (2016): Yes
- Outcome indicator: Percentage of renewable energy in the overall energy consumption/production of Ukraine
- Baseline (2010): TBC
- Target (2016): 20 percent increase
- Outcome indicator: Carbon intensity of economy
- Baseline (2010): Ukraine is amongst top 20 most carbon intensive economies
- Target (2016): Ukraine is not among top 20 carbon intensive economies
- Outcome indicator: Reduction in volume of CO2 emissions from stationary and mobile sources
- Baseline (2010): TBC
- Target (2016): 20 percent of baseline
- Outcome indicator: Volume of CO2 emissions into the atmosphere by industrial enterprises
- Baseline (2008): 90.6 Mt Co2
- Targets (2016): Reduction by 50 Mt
- Outcome indicator: Level of energy expenses per unit of production, kg of conditional fuel/1$
- Baseline (2010): 0.89
- Target (2016): 0.5
- Outcome indicator: Climate change risks and opportunities integrated in the agricultural and rural development policy of Ukraine
- Outcome 3: Regulatory and legislative mechanisms for sustainable management of natural resources are created
- Outcome indicator: Laws pertaining to sustainable management of national resources enacted and enforced, Yes/No;
- Baseline (2010): No.
- Target (2016): Yes.
- Outcome indicator: Share of protected areas of various categories as % of overall area of the territory of Ukraine
- Baseline (2010): 3.5
- Target (2016): 5
- Outcome indicator: Annual area of reforestation and afforestation, ha
- Baseline (2008): (reforestation 80.2; afforestation 44/6)
- Target (2016): 50 % increase
- Outcome indicator: Level of resource productivity and pollution intensity of companies involved in the Resource Efficient and Cleaner Production (RECP) Programme
- Baseline (2011): TBC
- Target (2016): At least 80 percent of companies achieve at least a 20 percent increase in resource productivity and/or at least a 30 percent reduction in pollution intensity
- Outcome indicator: Laws pertaining to sustainable management of national resources enacted and enforced, Yes/No;
There are a total of 8 UNDP projects to be assessed. They are as follows:
- Capacity Building for Low Carbon Growth in Ukraine (2011-2015);
- Integrating Rio Conventions Provisions into Ukraine’s National Policy Framework (2014-2016);
- Support Ukraine in development of INDC (2015).
Climate Change, Energy and Energy Efficiency:
- Transforming the Market for Efficient Lighting (2011-2016);
- Development and Commercialization of Bioenergy Technologies in the Municipal Sector of Ukraine (2014-2018).
- Clima East: Conservation and Sustainable Use of Peatlands (2013-2016) (also contributing to Land degradation reversal and Biodiversity).
- GEF Small Grants Programme Fifth Operational Phase (2011-2015).
- Initial implementation of Accelerated HCFC Phase-Out (2013-2016).
- Improving Environmental Monitoring in the black sea (2013-2014).
Methodology and Guidance on Outcome Evaluation
UNDP has in place a range of guidance on evaluation, which provides a conceptual review of outcome evaluation, identifies key tasks, offers a framework methodology for conducting outcome evaluations and includes a sample outline for an outcome evaluation report. As useful and concise resources for outcome evaluators, the following publications/documents should be consulted as early as possible and used in the evaluation process:
- The UNDP Evaluation Policy (UNDP, ‘The evaluation policy of UNDP’, DP/2011/3, 10 November 2010, p.10);
- The UNDP Handbook on Planning, Monitoring and Evaluating for Development Results (PME Handbook, 2009), available at http://web.undp.org/evaluation/evaluations/handbook/english/documents/pme-handbook.pdf;
- UNDP Guidance on Outcome-level Evaluation, 2011, available at http://web.undp.org/evaluation/documents/guidance/UNDP_Guidance_on_Outcome-Level%20_Evaluation_2011.pdf;
- Guidelines for Outcome Evaluators, 2002, available at http://web.undp.org/evaluation/documents/HandBook/OC-guidelines/Guidelines-for-OutcomeEvaluators-2002.pdf.
Although it is generally the responsibility of the evaluation team (evaluator) to decide on the concrete evaluation methodology to be used, the following elements should be taken into account for the gathering and analysis of data:
- Desk review of relevant documents, including files and correspondence related to projects implementation;
- Discussions with the Senior Management and programme staff of UNDP Ukraine;
- Interviews with and participation of partners and stakeholders;
- Field visits to selected key projects.
Duties and Responsibilities
The Consultant (evaluator) will perform the following tasks:
- Desk review of documents, detailing the evaluation scope, development of methodology, detailed work plan and Evaluation outline;
- Plan, lead and manage the outcome evaluation mission;
- Participate in the evaluation kick-off meeting with UNDP;
- Conduct the outcome evaluation in accordance with the proposed objective and scope of the evaluation and UNDP evaluation guidelines, including, but not limited to:
- Review the programmes and projects of UNDP in Ukraine contributing to the Environment and Sustainable Development portfolio with a view to understand their relevance and contribution to national priorities and relevant international obligations;
- Review the processes of engagement of key stakeholders on the stages of planning the program, its implementation and monitoring. - Review the status of the outcome and the key factors that have affected (both positively and negatively, contributing and constraining) the outcome;
- Assess the extent to which UNDP outputs and implementation arrangements have been effective for reaching the outcome and for strengthened linkages between the outcomes (the nature and extent of the contribution of key partners and the role and effectiveness of partnership strategies in the outcome);
- Conduct field visits to the project sites (if required) and interview national and local level stakeholders;
- Recommend corrections that may be required for enhancing effectiveness of UNDP’s development assistance in the practice area of Environment and Sustainable Development;
- Provide recommendations for future country programme outcome(s) in the area of Environment and Sustainable Development
- Prepare the draft evaluation report;
- Debrief with UNDP in Ukraine; and present draft findings, recommendations and conclusions of the evaluation report;
- Provide forward looking recommendations to the environment and sustainable development programme and an outline of future UNDP interventions in the new cycle of UNDP Ukraine Country Programme (2018-2022);
- Finalize the Evaluation report and Recommendations and outline of future UNDP interventions in the practice area of environment and sustainable development and submit it to UNDP Ukraine CO.
- The key product expected is a comprehensive Evaluation Report that includes, but is not limited to the following components: (see UNDP Handbook on Planning, Monitoring and Evaluating for Development Results for more guidance on outcome evaluations (http://web.undp.org/evaluation/evaluations/handbook/english/documents/pme-handbook.pdf) and UNDP Guidance on Outcome-level Evaluation http://web.undp.org/evaluation/documents/guidance/UNDP_Guidance_on_Outcome-Level%20_Evaluation_2011.pdf).
- Title and opening pages;
- Table of contents;
- List of acronyms and abbreviations;
- Executive summary;
- Evaluation scope and objectives;
- Evaluation approach and methodology;
- Development context (incl. development challenges);
- Description of the interventions;
- Data analysis and key findings and conclusions (incl. UNDP response to challenges, contribution to results);
- Conclusions, lessons learnt and recommended corrections, and recommendations for the future (including viable project ideas and other recommendations for the development of the new CPD 2018-2022);
- Annexes: TORs, field visits, people interviewed, documents reviewed, etc.
- Additional knowledge product is a forward looking Recommendations to the environment and sustainable development programme and an outline of future UNDP interventions in the new cycle of UNDP Ukraine Country Programme (2018-2022).
The Consultant (evaluator) will determine the specific design and methods for the evaluation during the initial inception period.
The evaluator should provide a proposed report structure to UNDP prior to the start of his/her fieldwork (evaluation mission). The report should be prepared in English. It should take into account the opinion/voices of people from Ukraine, government representatives, donors, academia and NGOs.
Recommendations and outline (deliverable 2) for the future UNDP interventions in the area of Environment and Sustainable development should be produced based on the findings and recommendations of the outcome evaluation mission. The format of the Recommendations and outline will be agreed between UNDP and the evaluator prior to the start of the evaluation. The evaluator is required to discuss the full draft of the evaluation report prior to departure from Ukraine. Both products (deliverables) shall be submitted in electronic form.
The evaluation shall be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ (http://www.uneval.org/ search/index.jsp?q=ethical+guidelines). The Evaluator will take every measure to safeguard the rights and confidentiality of key information providers in the collection of data.
The results shall be presented at a roundtable to all key stakeholders (representatives of Government, projects beneficiaries and NGOs etc.). The final evaluation report will be placed on the UNDP website and distributed through regular Government channels to interested parties.
- Good analytical and strategic thinking skills;
- Extensive knowledge of qualitative and quantitative evaluation methods;
- Sound knowledge of results-based management systems, and monitoring and evaluation methodologies; including experience in applying SMART indicators;
- Understanding of issues related to gender; experience in gender sensitive evaluation and analysis;
- Strong working knowledge of the UNDP and its mandate, and more specifically the work of UNDP in support of Environment and Sustainable Development initiatives in the region;
- Proven commitment to the core values of the United Nations;
- Familiarity with the political, economic and environmental situation in Ukraine;
- Excellent inter-personal, communication, and teamwork skills;
- Result orientation;
- Ability to meet tight deadlines;
- Excellent written and spoken English and presentational capacities.
Required Skills and Experience
- Minimum Master's degree in Environmental sciences, economics, public administration, regional development/planning, or other sciences related to environment and sustainable development field;
- Trainings in management, monitoring and evaluation will be an advantage.
- Minimum 10 years of professional experience in the area of Development, Environment and Sustainable Development, Energy, Regional development, including participatory planning, monitoring and evaluation;
- At least 7 years of experience in conducting complex evaluations, especially in Environment and Sustainable Development practice area, with proven accomplishments in undertaking evaluation for international organizations, preferably with UNDP;
- Extensive knowledge of results-based management evaluation, as well as participatory M&E methodologies and approaches;
- Working experience in CIS countries or Eastern European countries.
- Fluency in written and spoken English;
- Knowledge of Ukrainian and/or Russian will be an asset.
The detailed schedule of the evaluation and the length of the assignment will be discussed with the evaluator prior to the assignment. The estimated duration of the assignment is up to 30 working days.
The International consultant will work in a team with a local consultants that will help with the analysis and research of the available relevant documentation, with setting up the meetings with the external actors and with the needed ad-hoc translations/ interpretation. To facilitate the Outcome evaluation process an Evaluation Focal Team (EFT) comprising of representatives of UNDP Ukraine (DCD/Programme) and relevant project staff will be set up. The EFT will assist in connecting the evaluation team with the senior management and key stakeholders. In addition, the EFT will assist in developing a detailed evaluation plan and conducting field visits. During the evaluation, EFT will help identify key partners for interviews. Otherwise, the evaluation will be fully independent and the evaluation team will retain enough flexibility to determine the best approach in collecting and analyzing data for the Outcome evaluation.
Indicative Schedule of the Assignment
- Activity: Evaluation design, methodology and detailed work plan:
- Number of days: 2;
- Place: home-based, online;
- Responsible party: EFT, Evaluation Team.
- Activity: Desk review:
- Number of days: 5;
- Place: home-based, online;
- Responsible party: Evaluation Team.
- Activity: Evaluation mission (Interviews, consultations, data collection), Briefing and Debriefing with UNDP, Presentation of findings, conclusions and Recommendations of the evaluation report, Presentation of the 1st Draft Recommendations and Outline for future interventions:
- Number of days: 10;
- Place: 8 days in Kiev, estimated 2 days in the field;
- Responsible party: EFT, Evaluation Team.
- Activity: Preparation and submission of the draft Final Evaluation report:
- Number of days: 10;
- Place: home-based, online;
- Responsible party: Evaluation Team.
- Activity: Preparation and submission of the Final Evaluation report and Recommendations and Outline for future interventions (Revised report addressing feedback and comments received):
- Number of days: 3;
- Place: home-based, online.
- International travel will be required to Ukraine during the outcome evaluation mission;
- The Basic Security in the Field II and Advanced Security in the Field courses must be successfully completed prior to commencement of travel;
- Individual Consultant is responsible for ensuring he/she has vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director;
- Consultant is required to comply with the UN security directives set forth under https://dss.un.org/dssweb/.
Payment Modality and Arrangements
Each candidate will be required to submit an aggregated financial offer (“aggregated financial offer” is the total sum of all financial claims of the candidate for accomplishment of the task), which includes proposed consultancy fee, travel costs, visa costs (if required), per diem (for accommodation, meals and local transport / communication). The consultant will be provided with the necessary administrative and logistical support to enable them deliver on the expected outputs.
Payment will be disbursed in three installments upon submission and approval of deliverables and certification by the UNDP DCD/Programme that the services have been satisfactorily performed.
- Instalment 1:
- 10 % of total consultancy fee;
- Milestone: Upon approval of the evaluation methodology, detailed work-plan and schedule and submission of an invoice.
- Instalment 2:
- 30 % of total consultancy fee;
- Milestone: Upon submission of the draft Evaluation report and Recommendations and Outline for future interventions.
- Instalment 3:
- 60 % of total consultancy fee;
- Milestone: Upon finalization of the Evaluation report and Recommendations and Outline for future interventions.
- All envisaged travel costs (including ticket, accommodation, etc.) must be included in the offeror’s financial proposal. The individual offeror should consider the prevailing price for an economy class tickets serving the most direct routes in his /her financial proposal;
- Individual contractor wishing to upgrade his/her travel to business or first class shall do so at his/her own expense;
- The project will provide arrange local transportation domestic travel- therefore the cost to be excluded from this contract. As for living allowances, the cost will be included in and covered by this contract. Therefore the offeror is required to include the foreseen cost (living allowances) for Kyiv, Ukraine;
- The offeror is therefore encouraged to check the ceiling of living allowances for Kyiv, Ukraine following link: http://icsc.un.org?, and to include the amount in the financial proposal. However, the reimbursement of local travel costs will be made upon receipt of travel claim form and based on the actual travel dates;
- Each payment will be made in US dollars upon satisfactory completion of the tasks and respective deliverables as per submission of deliverables/claims by the consultant and the project/UNDP approvals;
- Each payment will be transferred by UNDP through Electronic Fund Transfer to the Dollar account number of the contractor introduced through an official letter indicating full banking information;
- Any payment under this contract will be made using UN Operational Rate of Exchange. For update rates please see: http://treasury.un.org/operationalrates/OperationalRates.aspx;
- Payments will be made according to UNDP regulations as explained in the contract documents;
- The International Consultant shall not do any work, provide any equipment, materials and supplies or perform any other services which may result in any cost in excess of the above mentioned amount.
Interested individual consultants must submit the following documents/information to demonstrate their qualifications:
- Letter of Confirmation of Interest and Availability using the template provided by UNDP;
- Financial Proposal that indicates the all-inclusive fixed total contract price and all other travel related costs (such as flight ticket, per diem, etc), supported by a breakdown of costs, as per template attached to the Letter of Confirmation of Interest template;
- Duly completed and signed a Personal History Form (P11 form), including information about past experience in similar assignments and at least 3 contact details for referees;
- Brief description of approach to work and why the individual considers him/herself as the most suitable for the assignment (brief information on each of the required qualifications, item by item).
If an applicant is employed by an organization/company/institution, and he/she expects his/her employer to charge a management fee in the process of releasing him/her to UNDP under Reimbursable Loan Agreement (RLA), the applicant must indicate at this point, and ensure that all such costs are duly incorporated in the financial proposal submitted to UNDP.
UNDP will use a two-stage procedure to assess applications with. Technical proposals will be assessed prior to any commercial proposals. The Technical proposals will be assessed based on compliance with the Terms of Reference. The following criteria will be rated as indicated below:
- Master's degree in Environmental sciences, economics, public administration, regional development/planning, or other sciences related to environment and sustainable development field – 10 points max:
- PhD – 7 points;
- Master’s degree – 5 points.
- Trainings in management, monitoring and evaluation – 3 points max:
- trainings proven by available training certificates – 3 points;
- no training – 0 points.
- Minimum 10 years of professional experience in the area of Development, Environment and Sustainable Development, Energy, Regional development, including participatory planning, monitoring and evaluation – 20 points max:
- more than 15 years of experience – 20 points;
- from 13 to 15 years of experience – 15 points;
- from 10 to 13 years of experience – 12 points.
- At least 7 years of experience in conducting complex evaluations, especially in Environment and Sustainable Development practice area, with proven accomplishments in undertaking evaluation for international organizations, preferably with UNDP – 30 points max:
- more than 10 years of experience in conducting complex evaluations – 15 points;
- from 8 to 9 years of experience in conducting complex evaluations – 12 points;
- from 7 to 8 years of experience in conducting complex evaluations – 10 points;
- experience in conducting evaluations in Environment and Sustainable Development practice area – additional 5 points;
- experience in applying SMART indicators – additional 5 points;
- experience in gender sensitive evaluation and analysis – additional 5 points.
- Working experience in CIS countries or Eastern European countries – 5 points max:
- Availability of experience – 5 points;
- No experience – 0 points.
- Fluency in written and spoken English – 3 points;
- Knowledge of Ukrainian and/or Russian – additional 2 points.
Maximum available technical score – 70 points.
Contract award shall be made to the incumbent whose offer has been evaluated and determined as:
a) responsive/compliant/acceptable, and
b) having received the cumulative highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.
* Technical Criteria weight: 70%
* Financial Criteria weight: 30%
Only candidates obtaining a minimum 70% from the maximum available technical score (49 points) would be considered for the Financial Evaluation
The maximum number of points assigned to the financial proposal is allocated to the lowest price proposal and will equal to 30. All other price proposals will be evaluated and assigned points, as per below formula:
30 points [max points available for financial part] x [lowest of all evaluated offered prices among responsive offers] / [evaluated price].
The proposal obtaining the overall cumulatively highest score after adding the score of the technical proposal and the financial proposal will be considered as the most compliant offer and will be awarded a contract.