Terms of Reference For The Final Evaluation of the Ethiopian Joint Response 2022-2023 – SOS Children’s Villages Ethiopia

Consultancy and Training

SOS Children’s Villages Ethiopia

Background

Dutch Relief Alliance Overview

The Ethiopia Joint Response is part of the mechanism of the Dutch Relief Alliance (DRA). The DRA is an alliance of 14 Dutch aid organizations that was set up in 2005 to increase the effectiveness of Dutch humanitarian assistance by the Dutch Minister for International Trade and Development Cooperation. This was done in response to the challenges of the humanitarian system and the growing gap between humanitarian needs and humanitarian funding. The DRA responds to chronic crises as well as acute crises, for which they receive funding from Netherlands Ministry of Foreign Affairs (MFA). The main objectives of the Dutch Relief Alliance are to deliver effective, efficient, relevant and timely humanitarian aid through collaboration to more beneficiaries in a better way. DRA partners implement Joint Responses (JRs) to address protracted crises enabling capacity strengthening, localisation and investment in community resilience.

The Ethiopia Join Response 

The current Ethiopia Joint Response (EJR) runs from 1^st of January 2022 until the 31^st of December 2023. The Joint Response is implemented by 6 international and 6 national organizations: SOS Kinderdorpen (lead) together with Tesfa Birhan, Plan International together with Mothers and children multisectoral development organization (MCMDO), Stichting Vluchteling / IRC together with Action for needy in Ethiopia (ANE), Terre des Hommes (TdH) together with African Network for the Prevention and Protection Against Child Abuse and Neglect (ANPPCAN), Cordaid together with Ethiopia catholic church social development commission (ECC-SDCO), and Tearfund/FH together with Ethiopia Kale hiwot church development commission (EKHCDC). The EJR is providing humanitarian assistance in Oromia Region – East Hararghe Zone, Amhara Region – North Wollo Zone and North Shewa Zone, and Tigray Region, in selected woredas. These Regions, zones and woredas were selected based on the highest needs, and the capacity of the DRA partners. The assistance is targeting the most vulnerable, food insecure population of the host community, internally displaced people (IDP) and returnees, who are provided with multi-sectoral assistance, with a focus on Food Security & Livelihoods, WASH, Health, Education, Protection and Multi-Purpose Cash; throughout all activities protection is mainstreamed. Other cross-cutting themes are localization and accountability to affected population. The Joint Response is set up in such a way that partners collaborate and learn from each other, which is the overall approach of the DRA towards humanitarian assistance.

Evaluation Details

Purpose and Scope of the Evaluation of the Joint Response

The EJR has a contractual obligation towards the DRA, and the back donor, the Dutch Ministry of Foreign Affairs, to ensure the realisation of an evaluation of the response. The purpose for this evaluation is twofold. On the one hand, the evaluation will assess the performance of the EJR and ensure accountability towards the Dutch Government and public, as well as to the targeted population. On the other hand, it offers a learning aspect for all stakeholders. The lessons learned and recommendations for programme implementation and design should be part of the evaluation report and will be used to improve the implementation of the next phase of the EJR (2024-2026). The findings of the evaluation should be presented in meeting with all the EJR partners.

The evaluation, should include the following objectives:

Objective 1: Performance / achievements of the EJR: The evaluation report will assess the overall performance of the EJR against selected OECD-DAC criteria and the Core Humanitarian Standards, ensuring accountability towards the Dutch Government, the Dutch public and the targeted population within the scope of the program. It should be noted that the consolidated final logframe with project results on indicator level will only be available towards the end of March 2024, and therefore might not be relevant for this evaluation; therefore evaluation will focus on the period from start until end of Q3 (Dept ’23), for which monitoring data will be made available.

 I. Relevance/ appropriateness:

a. How relevant have the activities of the EJR been in addressing humanitarian needs in Ethiopia?

       i. Are affected people satisfied with the interventions?

b. To what extent was the EJR able to adapt and provide appropriate response to context changes and evolving needs and capacities, and the priorities of the people, taking into account the specific needs of the most vulnerable groups, including women and children?

c. To what extent and how was protection mainstreamed across the program activities?

d. Which attempts were/are being made to ensure participation of affected population in identifying the needs?

e. How were the communities involved in assessments, planning, implementation & monitoring of the program?

f. How, if so, is this involvement influencing decisions on the activities that are implemented and how they are done?

g. How satisfied are affected people with their level of involvement, with the information they receive on project activities and their ability to provide feedback and get a response? 

II. Effectiveness:

a. To what extent and how have the planned results of EJR (according to project plan, including logframe) been reached?

b. What were the major factors influencing the achievement or non-achievement of the response objectives?

c. How effective were the strategies and tools used in the implementation of the project? 

III. Efficiency:

a. Was the process of achieving results efficient? To what extent has the EJR been implemented in a timely manner?

b. Were the resources allocated in the project appropriate to the scale of the project?

c. To what extent has the joint response model of collaboration (with local and international partners) led to multi sectoral support and cost-effectiveness?

      i. What is the role of the local authorities and how does this influence the multi-sectoral support and cost-effectiveness?

IV. Impact:

a. Have there been any unexpected positive or negative side-effects on affected populations as a result of the assistance provided by the EJR?

b. How has the EJR impacted the lives of the affected population in sustaining their lives?

V. Sustainability:

a. To what extent do the benefits of the intervention continue, or are likely to continue beyond the life of the project?

b. What mechanisms has the EJR put in place to sustain the key project outcomes?

c. To what extent does the intervention reflect and consider factors which have a major influence on sustainability, i.e. economic, ecological, social and cultural aspects? 

d. What are the contributing factors and constraints that require attention in order to improve prospects of sustainability of the project outcomes?

Objective 2: Collaboration and learning in the EJR: The evaluation report will assess to what extent the collaboration between EJR partners^[1] contributed to the delivery of effective, efficient, relevant and timely assistance to the affected populations, and how have different stakeholders benefited from the collaboration in EJR. What generated learning and collaborative impact and what didn’t

I. Learning:

a. To what extent has the EJR collaborative way of working facilitated peer-learning between EJR partners?

b. How have activities to increase learning affected delivery of humanitarian assistance by the EJR partners?  Is there any substantial anecdotal evidence of this?

c. Did EJR partners make any changes to their programming as a result of learning activities, leading to better program quality? Please provide concrete examples.

II. Complementarity and harmonization:

a. To what extent were the activities of the EJR partners complementary to the work of other stakeholders, prevented duplication, and contributed to the larger humanitarian response activities in the country?

b. Is there any substantial (anecdotal) evidence of how collaboration between EJR partners, specifically related to complementarity, has led to positive impact on the lives of beneficiaries?

III. Localization (participation of local partners in decision making in the EJR, capacity strengthening of local organisations and/or Government):

a. To what extent and how did local partners feel empowered / felt ownership in the design and implementation of the EJR programme? What factors played a role in this?

b. How has the role of the Local Advisory Group (LAG) representative of the EJR increased decision-making powers and ownership of the national EJR partners?

c. How did the EJR partners work together with local authorities, and how does this collaboration with the authorities affect the collaboration among EJR partners and the targeting of beneficiaries, positively or negatively the achievement of the project?

IV. Cross-cutting themes:

a. How have cross cutting issues, including gender, age, protection mainstreaming, been addressed in the EJR, and how does this compare to the project proposal?

Lessons learned and recommendations that could improve the next program cycle of the Ethiopia Joint Response should be collected as well during the evaluation of the above topics/questions.

Methodology

The evaluation will be carried out in a transparent and participatory manner by involving relevant stakeholders (EJR partners, affected population, cluster representatives, authorities and a diverse and inclusive representation of community leaders and representatives).The methodology will be developed by the consultant(s) as well as all relevant tools, based upon this TOR, and presented in the inception report (using as well an evaluation matrix).

The evaluation should include fieldwork, security permitting, to verify achievements of objectives. The evaluation design should include a creative, participatory methodology to collect several significant stories of change among the affected population and other stakeholders. It is suggested to consider as a minimum

• Desk study and review of all relevant program documentation and monitoring data
• Key Informant Interviews with key stakeholders
• Focus Group discussions with target population and staff
• Questionnaire for EJR partner staff and/or structured questionnaire/household survey
• Links to resource portals need to be included

The use of creative or participative qualitative methods (to eg. draw out and document learnings) is welcomed. The evaluation should be inclusive taking into account gender, age, disability, and other vulnerability considerations, sensitive of social norms and practices, and be considered of ethical data collection.

Role and Responsibility of the Consultant

• Prepares an inception report, detailing the methodology, stakeholders to be interviewed, tools to be developed, time frame for the evaluation and budget
• Holds the overall management responsibility of the evaluation, including designing and carrying out the evaluation (data collection and analysis), drafting the final report and debriefing the project team and key stakeholders.
• Liaise with SOS CV staff throughout the process, providing weekly updates and seeking their input and advice where necessary. Request approval in case of deviation from budget, and for miscellaneous costs.

• Sign SOS CVE code of conduct Policy and abide by the terms and conditions thereof.

Deliverables

The consultant is expected to lead, accomplish and submit the following deliverables within the agreed timeframe and budget:

1.  An inception report will highlight the methodology and the guiding principles of the evaluation, and will serve as an agreement between parties on how the evaluation will be conducted. Items to address

• Understanding of the issues and questions raised in the ToR
• Data sources; how to assess the questions in the ToR.
• Research methodology using an evaluation matrix (addressing research questions and methodology used)
• Team composition
• Schedule of activities and travel (timeline)
• Detailed budget
• Appropriate validated draft data collection tools (e.g. methodological guidelines, group interview questions)
• Mainstreaming gender considerations during the evaluation
• Risk analysis and mitigation approaches
• Access & insecurity scenarios analysis and data collection contingencies
• Debriefs including a feedback and verification workshop for SOS CV and EJR partners on the major qualitative and quantitative findings, and including debriefing after fieldwork per project zone of intervention.

2. A max 30 page draft and final report (in MS Office and PDF for final version), excluding annexes, in English, in the following format at a minimum, to be submitted to SOS CVE and SOS CV NL. The report should consist of:

• Cover page
• Table of Contents
• List of Acronyms
• List of Tables
• Executive Summary in bullets (max. 2 pages)
• Introduction (Background, purpose/scope of evaluation)
• Methodology, including sampling and limitations
• Analysis and findings of the evaluation
• Lessons learned, best practices and recommendations
• Conclusions and specific recommendations with details how they might be implemented
• Several stories of change and quotes from respondents
• Annexes

                  -Project log frame

                  -ToR of the evaluation

                  -Schedule (including time, location) of the different components of the evaluation

                  -Relevant maps and photographs of the evaluation areas

                   -Bibliography of consulted secondary sources

                   -Finalized data collection tools

                   -List of interviewees.

3. Feedback and verification and learning workshop for EJR Partners in Addis Ababa (and online for partner representatives in NL). The evaluator will present preliminary findings before the publication of the final report to EJR partners. This will be done by organising a (max one day) workshop in Addis Ababa (preferable with online participation of NL partners), after finalizing the evaluation.

Indicative Timescales

Assignment is to be executed in December 2023 (exact dates to be confirmed with contracted party).

Contract, Budget and Disclosure of Information/Ethics   

SOS CVE will provide a consultancy contract to the lead consultant; the consultant shall adhere to the SOS CVE Code of Conduct. The consultant will submit a cost proposal with the Expression of Interest; this should include all costs related to this evaluation, including salaries, meal cost of the consultant/consulting company personnel, stationary, printing, transportation, accommodation and any other direct or indirect costs related to the performance of this assignment. The consultant is also responsible for providing and covering the costs of any medical, insurance and evacuation needs of their personnel. SOS CVE will not be responsible for covering any expenses that may be incurred as a result of any type of accident or any type of security incident nor will SOS CVE be liable in any way for any injury or death due to any cause incurred by any of the consultant’s personnel. Expression of Interest will be evaluated based upon cost effectiveness, as well as technical quality. Payment will be done in 3 tranches: 30% advance payment, 70% after successful completion of the assignment, and comprehensive, qualitative and satisfactory reports are submitted to SOS CVE and SOS CV NL. The payment methodology will be further agreed by the winning consultant and SOS CV.

It needs to be understood and agreed that the consultant shall, during and after the effective period of the contract, treat as confidential and not divulge, unless authorized in writing, any information obtained in the course of the performance of the contract. The ethics process and research needs to comply with specified requirements (Code of Conduct, Research Policy and Standards).

Interested experts/consultancies are required to provide CVs detailing the experience with similar type of assignments completed in the past.

Required expertise:

• Academic degree in relevant field of study (International Development, Development Studies, Disaster Risk Management, Food Security, Humanitarian Action, or a related field)
• Demonstrated experience in humanitarian response and knowledge of humanitarian standards (CHS, Sphere, Code of Conduct);
• Demonstrated experience in leading evaluations of humanitarian response programs, and evaluating consortia/joint responses;
• Experience in working different regions in Ethiopia context;
• Experience with the DRA is an asset;
• Demonstrated experience with quantitative and qualitative research, data base management and statistical data analysis;
• Demonstrated experience in the use of participatory research methodology.
• Ability to assess and further develop a conceptual evaluation tool;
• Relevant subject matter knowledge and experience regarding the thematic areas of the EJR;
• Demonstrated analytical, communication and report-writing skills;
• Proven record of communicating with affected population;
• Language Proficiency: Proficiency in English and the ability to produce good quality written documents in English is a mandatory requirement of this assignment;
• Understanding of local languages relevant for the assignment will be an added value;
• The consultant/team leader must have demonstrated consultancy track record and be recognized professional in conducting evaluations/surveys with a high degree of proficiency.

Lien