India nationals: Consultant (2) positions – Lead Consultant (1) and Consultant (1) for conducting the Terminal Evaluation of the project on “Enhancing Institutional and Community Resilience to Disasters and Climate Change”
The second phase of the USAID funded project on “Enhancing Institutional and Community resilience to disasters and climate change” will be ending in December 2020. As per the UNDP evaluation guidance, conducting a “Terminal Evaluation” during project closure is mandatory. The evaluation must aim to address the extent to which the project has been able to develop resilient cities through risk reduction in the context of disaster and climate change. The evaluation must provide evidence-based information that is credible, reliable and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with government counterparts The results of the terminal evaluation will be presented to the Implementing partner (Ministry of Home Affairs, Government of India) and will be used to highlight success stories and lesson learning for future endeavours.
Duties and Responsibilities
Objectives: The objectives of the terminal evaluation is to assess the achievement of project results, and to draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming. Accordingly, proposed evaluation of the project will undertake:
- Outcome analysis - what and how much progress has been made towards the achievement of the outcome (including contributing factors and constraints);
- Output analysis - the relevance of and progress made in terms of the UNDP outputs (including analysis of both project and non-project activities);
- The evaluation report must include a chapter providing a set of conclusions, recommendations and lessons
Scope: Project intervention areas include six cities- Cuttack, Navi Mumbai, Shimla, Shillong, Vishakapatnam and Vijayawada.
Review Criteria and key guiding questions
An assessment of project performance will be carried out, based against expectations set out in the Project Logical Framework/Results Framework, which provides performance and impact indicators for project implementation along with their corresponding means of verification. The evaluation will cover the criteria of: relevance, effectiveness, efficiency, sustainability and impact. Ratings must be provided on the following performance criteria.
As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. If it is not possible to travel to or within the country for the evaluation then the evaluation team is expected to develop a methodology that takes this into account the conduct of the evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. This should be detailed in the Inception report and agreed with the Evaluation Manager. If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/ computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the evaluation report. If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom etc.)
The evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments.
- Document review of all relevant documentation. This would include a review of inter alia (will be provided to selected candidate on Day 1 of assignment)
- Project document (contribution agreement).
- Programme and project quality assurance reports.
- Consolidated quarterly and annual reports. (APRs/PIRs)
- Project budget
- Mid-term review / progress reports
- Results-oriented monitoring report.
- Highlights of project board meetings.
- Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organizations and implementing partners:
- Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed.
- Key informant and focus group discussions with men and women, beneficiaries and stakeholders.
- All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
- Surveys and questionnaires including participants in development programmes, and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels.
- The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries.
- Other methods such as outcome mapping, observational visits, group discussions, etc may be used.
- Data review and analysis of monitoring and other data sources and methods.
- Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources.
It is preferable that the interviews/questionnaires with the Ministry of Home Affairs will need to take place on a face to face basis in Delhi. Interviews will also be held with the following organizations and individuals at a minimum:
- Joint Secretary, Disaster Management, Ministry of Home Affairs (face to face meeting)
- Programme Management specialist – Disaster management, USAID
- Chief, Climate Change, Resilience and Energy, UNDP
- Municipal Commissioners/representatives of Shimla, Navi Mumbai, Vishakhapatnam, Vijayawada, Cuttack and Shillong.
1- Evaluation products (deliverables)- refer to Annex for templates.
- Evaluation inception report: The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review, and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits)
- Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
- Draft evaluation report: The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period of time, addressing the content required and quality criteria as outlined in these guidelines.
- Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
- Final evaluation report.
- Presentations to the Ministry of Home Affairs and/or the evaluation reference group
The evaluation team will comprise of 2 team members (all national) as follows:
- One Lead Consultant who will be the overall lead to coordinate and manage the Terminal evaluation in accordance with the objective, scope, quality and timely submission of the deliverables; provide strategic guidance; consolidate, review, quality assure, and finalize the reports and deliverables; lead discussions, presentations and sharing of key findings to the government and other stakeholders; and lead briefing meetings and presentations to UNDP India team.
- The Lead Consultant should have Master’s degree (Ph.d. would be an advantage) in environment, disaster management or climate change field with at least 10 years of broader development experience. Previous experience with results-based monitoring and evaluation methodologies. Experience on handling projects around disaster risk management and climate change adaptation / mitigation
- One Consultant to support the Lead Consultant: “Minimum 5 years of relevant professional experience with post graduate degree in engineering/ environment/ management or related filed domain. Previous experience with results-based monitoring and evaluation methodologies.
Key technical and core competencies:
Knowledge of the political, social, cultural, and economic context of the Country; Ability to lead informed and strategic discussions with government counterparts and stakeholders;
- Ability to undertake gender sensitive review and analysis and demonstrated understanding of issues related to gender-mainstreaming and other cross cutting development issues;
- Knowledge of qualitative and quantitative evaluation methods; Proven experience in data analysis as well as report writing; ensuring the triangulation of the findings, obtaining strong evidence for the analysis of information from multiple sources;
- Strong analytical skills (quantitative and qualitative) and strong ability to communicate and summarize this analysis in writing;
- Experience of conducting stakeholder interviews and collecting data remotely;
- Ensuring comments and feedback of the stakeholders is sufficiently addressed in the draft and final reports.
- Deliver the agreed products to the right standard and quality;
- Excellent inter-personal, teamwork and communication skills;
- Flexibility in remote working arrangements and experience of conducting stakeholder interviews and collecting data remotely;
- Openness to change and ability to receive and integrate feedback;
- Excellent report writing, presentation and editing skills in English;
- Keep to the standards and ethical principles in line with UNEG norms and standards and Ethical Guidelines.
Required Skills and Experience
Evaluation Method & Criteria:
The consultant would be selected based on the following criteria:
Technical: 70 points
Financial: 30 points
Criteria for Technical Evaluation would be as under:
* Educational Qualification, relevant development and work experience – 50 marks
* Technical Proposal – 20 marks
Technically qualified consultants will be requested to submit their daily fee rate i.e. consultants who score more than 70% i.e. 49 marks with respect to the above-mentioned evaluation criteria. Consultant should not specify their consultancy fee on their CV or with the submission. The CV will not be evaluated further in case the consultant submits the same.
The Consultant is required to submit the following documents, in a single combined PDF file, as the system has provision for uploading only one attachment:
- Indicate upfront the position for which you are applying (i.e. Lead Consultant (or) Consultant)
- Personal CV with relevant experience to the TOR; and at least 3 professional references
- Short technical proposal (max 2-pages) including methodology, approach & assessment criteria, process followed, data collection and analytical tools.
- No Financials (Daily Fee) to be submitted at this stage.
This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners
The Consultants will be held to the highest ethical standards and are required to sign a Code of Conduct upon acceptance of the assignment.