- Lao, Bhutan
- Contract
- Contract type: Consultancy
Company Description
SNV in collaboration with the Institute of Sustainable Futures, University of Technology Sydney (ISF-UTS) and CBM Australia, the governments of Nepal, Bhutan and Lao PDR and local partners are concluding a multi-country project Towards Inclusive Climate Resilient Rural WASH Services. The project is a two-year extension phase (2023-24) of the previous 5-year multi-country project Beyond the Finish Line (2018-2022)[1] which was evaluated in 2022. The project is funded by the Australian Government’s Water for Women (WfW) Fund[2] which is supporting improved health, gender equality and well-being in Asian and Pacific communities through climate-resilient and socially inclusive water, sanitation and hygiene (WASH) projects and research. Water for Women is the Australian Government’s flagship WASH international development program and is being delivered in 16 Asia Pacific countries, investing AUD159.9 million over seven years from 2018 to 2024, in its extension phase it has pivoted to focus on identifying and responding to climate risks to strengthen community climate resilience through WASH. SNVs multi country project summaries are provided in Annex 1.
[1] The extension phase investment is AUD 6 million over the 2 year period in addition to the AUD $13,309.362 5 year investment.
[2] https://www.waterforwomenfund.org/en/
SNV is a global development partner, deeply rooted in the countries where we operate. We are driven by a vision of a better world: A world where across every society all people live with dignity and have equitable opportunities to thrive sustainably. To make this vision a reality, we need transformations in vital agri-food, energy, and water systems. SNV contributes by strengthening capacities and catalysing partnerships in these sectors. We help strengthen institutions and effective governance, reduce gender inequalities and barriers to social inclusion, and enable adaptation and mitigation to the climate and biodiversity crises. Building on 60 years of experience we support our partners with our technical and process expertise and methodological rigour. We do this in more than 20 countries in Africa and Asia with a team of approximately 1,600 colleagues.
By being adaptable and tailoring our approaches to these different contexts, we can contribute to impact at scale, resulting in more equitable lives for all. Within WASH programmes our approach is based on the recognition that access to water and sanitation are human rights and is part of a broader strategy of water security. SNV builds the capacity of government duty bearers and stakeholders in civil society and private sector to strengthen sustainable service systems for all
Job Description
Introduction
SNV will undertake a final evaluation with three project teams and partners of the 2-year extension phase multi-country project 2023-2024 as a summative assessment of project achievements, learning and contribution to outcomes. These terms of reference present the objective, the methodology, the deliverables and responsibilities of this process. As a final evaluation it will address both project impact and effectiveness to understand the difference the multi-country project has made and how well it has contributed to outcomes in each setting. It is also important that the evaluation consider the sustainability of outcomes – including in relation to its previous phase, along with equity considerations.
For SNV, the purpose of the evaluation is for accountability as well as program improvement and knowledge generation. As the project extension focused on developing program approaches to strengthening climate resilience in WASH programming, the learning from this evaluation will inform wider approaches in future climate resilient programming in SNVs Water sector and beyond.
To enable this, SNV are seeking applications for a Lead Evaluator, or team that draws on upon relevant WASH, Gender Equality, Disability and Social Inclusion (GEDSI) and climate technical advice with combined demonstrated experience in undertaking WASH programme evaluations, tailored to the projects focus on capacity building, system strengthening and transformative approaches.
Objectives
The objectives of the evaluation are to:
- Evaluate the impact, effectiveness, equity and sustainability of the three projects’ outcomes
- Support learning and improvement,
- Document and share knowledge from implementation[1].
Methodology
This extension phase evaluation is intended to be conducted as a hybrid evaluation[2] which draws on a mix of both internal and external resources that engages project teams, partners and external evaluators working together, through defined roles, sense-making processes and facilitated reflection workshops.
The evaluation is expected to use suitable qualitative research methods, including review of project documentation and literature. In consultation with SNV, the consultants will develop a detailed evaluation plan[3] that further refines the evaluation questions, data collection methods and the division of roles and tasks between the evaluators, SNV and partners.
The methodology proposed will need to be in line with the project’s principles, partnership approaches and m&e plan, ensuring they are fit-for purpose and contextually appropriate and meet the requirements of the Water for Women Fund Guidance. The evaluator/team will design the approach to respond to the evaluation questions and then work with SNVs in-country teams (Bhutan, Nepal, Lao PDR) to undertake further data collection as necessary and to facilitate joint sense making. The external evaluator/team will then provide the final review and assessment of findings to meet the evaluation objectives. It is envisioned that support is provided as a combination of in-country and remote/online to maximise efficiencies, but that each project will be visited.
The evaluation is expected to draw on existing performance monitoring data which has been collected in line with SNVs Performance Monitoring Guidelines[4] by SNV teams end 2022 and October 24 and is not intended to duplicate survey processes. This includes existing household and institution level surveys (baseline, endline), annual capacity assessments with government line agencies and SMEs, FGDs with key groups (eg people with disabilities, women, households living in poverty), WASH GEM analysis[5] as well as project deliverables and knowledge and learning products.
Evaluations should address quality of evidence factors such as triangulation (using multiple data sources rather than relying on one data source such as interviews or a survey), contribution (establishing a basis for the projects role in causing the changes identified), uniqueness (ruling out alternative explanations that might have caused changes), transparency (being open about how data was collected, analysed, and reported) and independence (explicitly addressing potential biases). Evaluators might find it useful to use a tool such as the QoE Rubric (Aston, 2020), or the Bond Evidence Checklist as part of their evaluation design and review process.
Evaluation questions (from the M&E Plans)
Impact: How has the project implemented in 3 countries contributed to climate resilience and increased equitable access to and use of WASH services? What JMP service levels were achieved? Where outcomes as expected? Why? Why not? What project strategies were most effective?
Sustainability: To what extent did national or local authorities, institutions or communities demonstrate an increased capacity and commitment to inclusive climate resilient WASH? What evidence is there that systems and services have integrated considerations of climate risks and resilience?
Equity: How has the project implemented in three countries furthered gender equity, disability and social inclusion/or transformation? Is there any evidence of uptake of practice, innovation To what extent did the project reach and meaningfully involve the most marginalised and socially excluded people? What project strategies were most effective?
Contributing to this, the evaluation should consider the four end of programme outcomes, which are aligned to the overall Water for Women Fund Theory of Change.
End of Programme Outcomes are to be evaluated against each projects agreed results. The following are indicative guidance questions from the project m&e plans.
Systems Strengthening: 1 What changes have occurred in the capacity, agency and performance of governments, private sector, community-based organisations to better respond to climate risks and resilience when planning, investing and delivering inclusive WASH services?
Access to WASH Services 2. To what extent have changes occurred in equitable, universal access to and use of climate resilient WASH services, particularly for marginalised communities and community members?
Gender, disability and social inclusion 3. What broader social inclusion changes have occurred in households, community and institutions as a result of gender and socially inclusive WASH approaches and outcomes? How has the project understood the linkages between inclusion, transformation and climate-resilient WASH?
Knowledge and Learning 4: How have the projects commonly understand climate risk and resilience? How can this be further developed to better inform the design of WASH interventions? In what ways have other CSOs, governments, or other organisations (within and beyond Fund participants) taken up and used project generated documentation, evidence and/or demonstrated practice? How has this informed, influenced or changed policy or practice?
The evaluation will involve 4 stages – design, data collection, synthesis and results sharing. It includes both individual project level processes within country workshops, and the overarching multi-country programme level. These stages and their roles are described as follows.
The following are the key indicative tasks aligned to the different stages
Design
- Consultation with Multi-Country Project Manager (MPM), Country team leads, partners to determine the overall evaluation approach, roles and responsibilities.
- Online workshop and follow up – At the workshop, and subsequently via email, agree/refine key evaluation questions, methodology, roles and plans
- Refined methodology for evaluation and supporting tools developed to guide in-country data collection supported by evaluation plans (3)
- Quality assurance review – Feedback on methodology, tools and evaluation plans with input from the WfW Fund Coordinator M&E Advisor.
Project level data collection
- Local teams/partners orientated to conduct/support data collection and collate existing data to respond to the evaluation methodology, questions and aligned to evaluation plans by the Lead Evaluator.
- SNV Project teams compile all available data, including from the performance monitoring which includes surveys, FGDs, capacity assessments and KIIs to provide evidence for evaluation questions.
- Lead external evaluator/team conducts data collection with key stakeholders at multi-country level (eg ISF-UTS, CBM) and WfW
- Quality-assurance check-in.
- Local teams/partners/evaluator conduct any additional data collection with key stakeholders in Nepal, Bhutan and Lao PDR with oversight by lead evaluator and logistically supported by SNV country offices.
Interpretation
- Data synthesis – The country evaluation teams, together with the evaluation lead/team synthesize and analyze data collected.
- Sense-making workshops (3) – engaging each project at the country level together with key stakeholders, partners, to share and reflect on the findings.
- Combined (remote) multi-country programme level workshop (1) brings all actors together, to present findings, reflect on and agree any overall review findings and recommendations.
Results Sharing
- Presentation of findings – senior management team
- Drafting the (3) country level reports and recommendations for review
- The MPM, partners, peer reviewers provide two rounds of substantive comments.
- The quality assurer writes a separate report on the appropriateness and robustness of the process and findings. This is included in the evaluation reports.
- Communication Outputs – The evaluation lead/team finalizes the report and produces supporting presentation per project (3) using infographics were suitable.
- Final report and supporting outputs to be completed by January 20th 2025 for donor submission.
Follow up
- Follow up actions agreed.
Key Roles aligned to tasks
Evaluation Lead/team (LOE 58 Days) and travel.
Design Phase (8 Days)
- Prepare for assignment (reading of relevant documents such as project document, baseline survey report, annual reports, etc.)
- Determine overall evaluation approach, with country team leads, MPM, partners.
- Facilitate the design workshop (online 0.5 day)
- Finalise evaluation approach, revised evaluation questions and scope of additional data collection requirements
- Orientate Bhutan, Lao PDR and Nepal local teams on data collection processes and tools.
Project data collection (5 Days Remote + 5 Days in country for each country– 20 days in total) plus travel
- Provide advice, oversight and backup support on demand to in country teams
- Collect data at multi-country level (eg ISF-UTS)
- Collect data in country with support from local translators/team.
Interpretation (15 Days)
- Lead data synthesis and data analysis (3)
- Lead preparation of 3 country level sense-making workshops
- Facilitate 3 x 1 Day sense-making workshops – in country
- Lead preparation for programme level workshop (0.5 day online)
- Facilitate combined programme level workshop (0.5)
Results Sharing (15 Days)
- Lead the drafting of the 3 country evaluation report and recommendations
- Finalise the 3 country evaluation reports
- Produce supporting summary presentation per project
- Overall project management and consultations with country teams and MPM
- Final presentation to management team (2 hours)
Tasks for SNV Country Teams– 10 days per country project (Nepal, Bhutan, Lao PDR)
Support from SNV Country Teams is planned indicatively as 10 days per country project. This includes time for preparing for the assignment, participating in the design work and providing input into the development of the evaluation plan, methodology and tools and final reports. The teams will also provide support in completing agreed data collection in country in coordination and with oversight from evaluation lead, supporting data synthesis and analysis in line with the evaluation plans. Active roles are foreseen in collaboratively identifying key findings and participating in country level sense making workshops.
Key Deliverables
The main deliverables of the Evaluation are:
- Design Workshop (1)
- Evaluation Methodology (1), supporting data collection tools and three country level evaluation plans
- Sense making workshops (3 minimum) and combined Programme Level Workshop (1)
- Draft Evaluation Reports (3)
- Summary presentations (3)
- Final Review reports (3) including follow up actions.
SNV teams will provide logistical support throughout the process and facilitate meetings with key government partners, stakeholders and partners as required.
Further resources
To help inform and guide evaluation methods and approaches there are many resources available including the recent IDS M&E Framework for rural sanitation and hygiene and the DFAT M&E standards. Additional guidance is available from the Fund on disability measurement, knowledge updated as well as ISF-UTSs on qualitative approaches for understanding GESI changes. For technical WASH monitoring the Monitoring and Evaluation for Rural Sanitation and Hygiene: Framework (IDS, 2021) may be of interest. Projects should refer to the Gender Equality in Monitoring and Evaluation – Good Practice Note (DFAT, 2020) for integrating GEDSI into their evaluations. Projects should also refer to Monitoring and evaluation for climate resilient WASH (GWP/UNICEF, 2017) for guidance on M&E for climate resilient WASH.
Timeline and Duration
The assignment is estimated to cover 58 days for an Evaluation Lead or team approach, plus additional days for international travel.
The entire assignment is anticipated to commence at the start of October with the final deliverables to be submitted to SNV on or before 20 January 2025.
Payment Terms and Conditions
The payment schedule shall be as follows:
– 25% upon signing of contract
– 45% upon completion of the interpretation phase
– 30% upon submission of final approved deliverables.
SNV are seeking applications for a lead evaluator or team with demonstrated experience in undertaking WASH programme evaluations, with a focus on capacity building, system strengthening and transformative approaches. The lead evaluator/team must have availability throughout the duration of the project and bring excellent evaluation, coordination and communication skills and a demonstrated ability to work across culturally diverse settings and teams. Experience in facilitating remote sessions and processes preferred.
[1] Water for Women Fund: Evaluation Guidance, June 2024
[2] https://www.betterevaluation.org/methods-approaches/methods/hybrid-internal-external-staff
[3] Refer to Standard 8 > https://www.dfat.gov.au/about-us/publications/dfat-design-monitoring-evaluation-learning-standards
[4] These are available online at https://snv.org/assets/explore/download/2019-1-impact-indicators-ssh4a-performance-monitoring-review.pdf and https://snv.org/assets/explore/download/2019-2-outcome-indicators-ssh4a-performance-monitoring-review.pdf
[5] Projects have collected baseline and endline data collection supported by ISF using the WASH Gender Equality Monitoring Tool. More info > https://waterforwomen.uts.edu.au/wash-gem/
Qualifications
Composition of the evaluation team
The evaluation team is expected to be a Lead Evaluator but a team approach may also be appropriate with adequate steering. As highlighted in the DFAT MEL Standards (see 8.10) it is important that the evaluation team comprises relevant skill sets to address the project needs and priorities. As per DFAT guidance it is preferable that the evaluations are led by a person with evaluation expertise and the team draws upon relevant WASH, GEDSI and climate technical advice. The team should include sufficient balance and representation with respect to gender, disability, local experience as appropriate.
The lead should meet the following criteria:
· At least 10 years of experience within the field of international development cooperation;
· A strong background in evaluation and research, including applied to WASH programming;
· Experience with qualitative research methods.
All team members should meet the criteria below:
· Demonstrable familiarity with the domain of civil society strengthening, evidence generation and dissemination, and advocacy in WASH programming;
· Experience in ensuring inclusive evaluation processes and an understanding of ensuring accessibility and reasonable accommodation in evaluation processes (or similar).
· Familiarity with the (political) local context of the to-be-visited countries;
· Familiarity with the sector the project is operating in;
· Fluency in English communication, reporting and presentation skills.
Evaluation Criteria
– Experience in similar work with evidence (40%), preferably in the focus countries
– Competencies (40%)
– Financial (20%).
Due Diligence
The consultant shall comply with the SNV Code of Conduct and the Child Protection Policy during the assignment period.
Submission of proposal
Through this request SNV invites interested parties to submit a concise proposal that:
· Expresses understanding of the purpose of the evaluation
· Proposes a broad outline of approach and methods to be employed
· Provides daily consultancy fees, travel budget and an overall budget on headlines
· Includes consultants’ competencies and proof thereof (include 2 evaluation/research reports related to the subject), evaluators’ CVs, company profile
· Declares consultants’ availability in the period October 2024 – December 2024
· Includes a declaration of no conflict of interests by persons and organizations involved in the evaluation.
Application Requirements
Interested parties are requested to submit the following to SNV at email: bhutan@snv.org
Subject: External evaluation of Beyond Finish Line Programme
- Proposal responding to the TOR, including number of work days and breakdowns per role and task area.
- Curriculum Vitae for Lead Evaluation Specialist/team members
- Portfolio (link to website preferred)
- Three equivalent examples of work, including references
- Proposed day rate per position and travel budget.
- Availability.
Deadline for applications is on 9 September 2024
For more information, please contact
Gabrielle Halcrow, ghalcrow@snv.org
Additional Information
Annex A Project Summaries
Towards inclusive climate resilient WASH Services in Bhutan
https://www.waterforwomenfund.org/en/project/water-for-women—bhutan.aspx
The Towards Climate-Resilient Inclusive WASH Services in Rural Bhutan project aims to accelerate progress in rural WASH in Dagana, Zhemgang and Trashigang by strengthening the adaptive capacities of stakeholders, including rights holder groups and communities, to ensure sustainable service delivery and achieve quality and equity at scale. Building on the successes and achievements of SNV’s Water for Women project, Beyond the Finish Line – Sustainable Sanitation and Hygiene for All, completed in 2022, this project is addressing emerging challenges and needs, with technical advisory services, evidence-based advocacy, strategic partnerships, multi-stakeholder processes, gender and social inclusion mainstreaming, and knowledge, learning and exchange integrated in the program approach.
From 2023-2024, this project aims to deliver lasting impact through:
- Improved capacity of two national and 10 subnational government agencies in planning, delivering, monitoring and targeting of investments in climate-resilient inclusive WASH services that adapt to climate risks and water security challenges
- Increased capacity of two national and 10 subnational government agencies in designing, investing, implementing, and monitoring social and behaviour change communications that respond to climate-driven health risks and increase disaster preparedness
- Improved performance of 30 rural water supply operators (Water Users’ Committees) and strengthened adaptive post-construction support services delivering climate-resilient inclusive WASH services
- Strengthened consumer supply chains and finance to deliver affordable market-based solutions for changing consumer needs, climate resilience and service levels, including for potentially disadvantaged groups
- Strengthened leadership, agency and voice of women, people with disabilities, potentially disadvantaged groups and civil society actors
- Documenting and sharing of climate-resilient and gender, disability and socially inclusive evidence and practices with other CSOs, national/subnational sectors and international WASH and water sector actors.
Towards inclusive climate resilient WASH Services in Nepal
https://www.waterforwomenfund.org/en/project/water-for-women—nepal.aspx
he Towards Climate-Resilient Inclusive WASH Services in Rural Nepal project aims to support adaptation and strengthen resilience within current area-wide rural water supply services in four RMs in Dailekh and Sarlahi.
Building on the successes and achievements of SNV’s Water for Women project, Beyond the Finish Line: Inclusive and sustainable rural water supply services in Nepal, completed in 2022, technical advisory services, mentoring and capacity building, evidence-based advocacy, strategic partnerships, a twin-track approach to gender, disability and social inclusion (GEDSI), and knowledge, learning and exchange are integrated in the program approach.
From 2023-2024, this project aims to deliver lasting impact through:
- Strengthened decentralised WASH sector systems with greater emphasis on climate resilience, sustainability, GEDSI and safety managed WASH and water security
- Increased equitable universal access to and use of sustainable, climate-resilient WASH services and adoption of hygiene practices, particularly for potentially disadvantaged groups 
- Strengthened climate resilience and GEDSI in local bodies, CSOs, communities and households 
- Strengthened use of new evidence, innovation, and practices in climate-resilient inclusive WASH services by subnational and national stakeholders, CSOs and international WASH and water sector actors.  
Towards inclusive climate resilient WASH Services in Lao PDR
https://www.waterforwomenfund.org/en/project/water-for-women-Lao-pdr.aspx
Towards Climate-Resilient Inclusive WASH Services in Rural Lao PDR aims to strengthen the capacity of local governments and private sector actors to support adaptation and integrate climate change resilience within inclusive, area-wide rural WASH services and systems in the districts of Atsaphone, Champhone and Phalanxay in Savannakhet Province. Building on SNV’s Water for Women project, Beyond the Finish Line: Sustainable Sanitation and Hygiene for All, completed in 2022, from 2023-2024 this project is consolidating area-wide approaches, generating evidence of the costs of climate-resilient WASH services, innovating flood-resilient toilet designs, and developing climate-risk messaging on water quality and sector alignment with disaster risk reduction responses.
From 2023-2024 this project aims to deliver lasting impact through:
- Targeted support to 42 government health officers, 15 health care facilities and 50 schools to improve access to basic water supplies, basic sanitation — including menstrual hygiene management — and improve hand hygiene behaviour
- Verification of the three project districts as Open Defecation Free (ODF), with strategies in place to sustain this outcome
- Delivery of meaningful and scalable interventions for climate-resilient adaption processes, including facilitation of government learning, budgeting and planning, and development of climate-resilient water safety plans, learning and capacity-building for inclusive community WASH committees
- Research and development of climate-resilient support options for poor households to ensure WASH access for all and leave no one behind
- Strengthened decentralised WASH systems that are responsive to local climate risks, adaptation and security challenges, capacities and priorities
- Strengthened private sector engagement in the delivery of climate-resilient inclusive WASH services
- Strengthened gender equality, disability and social inclusion (GEDSI) in the WASH sector, including increased female leadership among local WASH provisioners, civil society organisations (CSOs), communities and households
- Strengthened use of evidence-based practice and innovation in climate-resilient, safely managed and GEDSI WASH delivery by subnational and national stakeholders, CSOs and WASH sector actors.
