Project Final External Evaluation TWISA


Dushanbe, , TJ

1. Background:Oxfam in Tajikistan has been awarded a multi-year GPSA grant for improving social accountability in the water supply and sanitation sector through the development of improved service performance standards and the promotion of citizen participation in service performance monitoring. TWISA has 4- year duration, starting in April 2014 and is implemented in Tajikistan with focus on 5 pilot districts of the Republic of Tajikistan. This TOR sets out responsibilities of the consultant to design and carry out project final evaluation in rural communities of Kulyab, Vose, Farkhor, Muminobod and Rudaki districts in the Republic of Tajikistan.

2. Description of the assignment

2.1 Project Overall Objective: A civil society-based monitoring framework (gender-sensitive) generates feedback on performance of service delivery in WS&S, using improved standards, ensuring citizen’s participation and promoting partnership in service management in target areas 2.2 Project Specific Objectives: The following project outcomes are expected to be achieved by end of the project:
1. Service quality indicators (gender-sensitive) in the WS&S sector are developed and disseminated in all 5 project operating areas, then nationally
2. A civil society-based monitoring framework (gender-sensitive) is developed consolidating new monitoring mechanisms
3. Consumers, service providers and other stakeholders have enhanced awareness and capacity and take effective part in a constructive dialogue
4. The knowledge base is created, developed and maintained and the learning process established for stakeholders about project components
2.3 Evaluation Objectives: The main objective of this consultancy assignment is to assess the overall achievement of the project and quality of implementation as well as the overall impact of the project on Tajik stakeholders. The consultant will work with Oxfam to document the project progress based on the outcomes with clearly indicative measurements and result-oriented methodology. To this purpose, it is expected:
  • To assess level of development and dissemination at national level of service quality indicators (gender-sensitive) in the WS&S sector
  • To assess level of consumer’s satisfaction with the indicators in the WS&S sector in all 5 project operating areas
  • To assess level of participation and quality of consumers’ inputs into the development, testing and implementation of monitoring framework (Community Advisory Boards (CAB) Model)
  • To assess the role of CAB in service quality assurance and operationality of water service providers
  • To assess the level of awareness and capacity of consumers, service providers and other stakeholders in target areas on their roles and responsibilities towards WS&S management, usage and allocation within village, disaggregated by gender;
  • To assess dynamics in interactions and exchanges of information among project stakeholders with other actors and projects to share practices, that are used to improve the learning and quality of the project
  • To assess knowledge and learning frameworks that the project stakeholders have increased throughout the project lifespan
  • To evaluate the relevance and consistency of the activities and outputs of the project with the overall goal, impact and the attainment of the objectives
  • To measure the factors influencing the achievement or non-achievement of the objectives
  • To evaluate the cost-efficiency, timeliness and skills of the project staff involved in the project
  • To quantify and assess the project impact on beneficiaries
  • To evaluate the project sustainability through social accountability lens
  • To assess women’s awareness of social accountability approach and impact on their life
  • To review the level of meeting the overall development objectives vis-à-vis the agreed results framework indicators of the project, and updating the results indicators
  • To report against each project results/monitoring framework table with outcomes achieved.
3. Scope of evaluation
This is an overall project and the consultant is expected to the evaluate the project at strategy, outcome and output level. The target audience of the evaluation is the population of 5 districts (Kulob, Muminabad, Vose, Farkhor, Rudaki) and project stakeholders (local water suppliers, SUE KMK, Vodokanal, Ministry of Energy and Water Resources, Ministry of Health, community members) at local and national level
3.1 Deliverables: The consultant will be responsible for identifying a relevant sampling methodology, drawing a sample, and training of enumerators, field testing of the questionnaire and administering the questionnaire in face-to-face interviews with key informants. The consultant will also perform data analysis based on the collected data. At the end of the analysis process a draft report will be shared with the TWISA project staff for their comments/feedback. Based on the comments/suggestions received from the client, the draft report will be updated to incorporate suggestions. The final report will be submitted at the end of the assignment. Final report will include the following as minimum:  Evaluation Methodology and detailed timeframe (inception report);
  • Draft evaluation report, supported with annexes of quantitative/qualitative analysis:
  • Relevance and Quality of project design and implementation
  • Project Effectiveness
  • Project Efficiency
  • Project Sustainability
  • Report on recommendations for future project follow-up
  • Final evaluation report taking comments and reviews from Oxfam into account
  • Annexes to the final evaluation report
4. Methodology It is proposed that consultant to follow the following sequence to achieve the desired results:
Phase-I: Inception Phase
4.1 Home Based Activities:
4.1.1 Pre-Project Inception Meetings: At the very beginning of the project, the consultant will have Skype meeting with TWISA staff. This is a very important aspect of the assignment as the ToRs will be discussed at length to ensure that Oxfam and the consultant create mutual understanding on the volume and quality of work expected.
4.1.2 Development & Finalization of Study Design: A detailed work plan/study design will be developed by the consultant in line with the discussion/suggestions with the TWISA project manager. The plan will describe detailed activities and their time line of completion. The work plan will be approved from the TWISA project manager.
4.1.3 Development of Methodology The consultant will undertake a comprehensive desk review of the existing project literature. The literature review will include but not limited to the TWISA project proposal, Logical Framework, reports, Detailed Implementation Plan, monitoring reports, case studies, baseline/endline reports and other relevant documents and reports prepared by Oxfam and its partner. The project staff will provide all relevant key project documents through email for the review.
Based on the literature review the consultant will develop the methodology for primary data collection. The methodology will be developed in line with project indicators, purpose and objectives of the study.
Phase-II: Data Collection Process:
4.2. On-site activities:
4.2.1 Initial Briefing Session with Oxfam: Immediately after the arrival of the consultant in Tajikistan, a briefing session will be held by Oxfam to cover all the aspects of the project and the objective and its intended outcome.
4.2.2 Development of Sampling: The consultant will propose and share the sampling after the desk review period. During desk review period the consultant will develop the selection criteria for the selection of districts among the 5 target districts.
4.2.3 Development & Finalization of Data Collection Plan: Data collection plan will include schedule and details of field visits/interviews. The field work plan will be finalized in close consultation with Oxfam. As per the agreed sample size and work plan the qualitative and qualitative data collection process will be initiated through FGDs, KIIs and interviews. The consultant will be responsible for overall management and implementation of the project.
Phase-III: Data Management:
4.3. Reporting
4.3.1 Data Analysis and Report Writing: Data analysis will start parallel to the data collection exercise. Qualitative and quantitative data will be analysed. At the end of analysis period, a draft report will be produced and shared with Oxfam and its stakeholders for their comments/feedback. Report will be produced on the agreed format and will be updated in line with comments/feedback of project team and its stakeholders.
5. Reporting The consultant will report to the Oxfam WASH Project Manager.
6. Timeframe and duration It is anticipated the evaluation will take approximately 11-12 days as per the table provided. 
 7. Profile of consultant required Individual consultants to apply.
  • The profile of the individual consultants needs to meet the qualification requirements below. Qualifications and Skills:
  • PhD degree, in areas of overseas development/humanitarian action/economics or social science. Master degree in these areas and extensive (more than 7 years) managerial experience in the relevant fields would compensate for the absence of PhD degree;
  • More than 10 years of experience in research and strategy/organisational development/project management and review;
  • Extensive experience of working with international organisations globally and specifically in complex environments in the field of social accountability, good governance, civil society engagement, public budgeting and institutional development ;
  • Excellent attention to detail and evidence of quality outputs from previous assignments;
  • Fluent English (Russian/Tajik is an asset);
  • Good qualitative and quantitative analytical skills;
  • Good reporting and presenting skills;
  • Punctuality and availability to complete the work on time;
  • Experience in social accountability in water sector is an asset;