Terms of Reference (ToR) For End-of-Project Evaluation – Mahibere Hiwot for Social Development (MSD)

Development and Project Management, Research and Development

Mahibere Hiwot for Social Development (MSD)

The Contract No. FED/2020/414-544

Background

Mahibere Hiwot for Social Development (MSD) is a national CSO engaged in enhancing citizens’ engagement in peace building and improving attitudes, perceptions, and skills about coexistence, trust building and an appreciation of diversity in different parts of Ethiopia. It has been implementing the project in seven regions (Amhara, Addis Ababa, Benshangu Gumuz, Oromia, Sidama, SNNPR, and Tigray ) of Ethiopia.

 As part of its overall programme, and with a financial support from the European Union Civil Society Fund III, MahibereHiwot for Social Development (MSD), with its partners Hiwot Ethiopia (HE), Young Men Christian Association (YMCA), Eshet Youth and Children Development Organization (EYCDO) and Save Your Holy Land Association (SYHLA) is implementing a project entitled Peace Matters: Dialogue for Peaceful Coexistence project contract ref. FED/2020/414-544 in the period February, 2020 to February, 2023. The project had the overall objective of to contribute towards the national effort in enhancing citizens’ engagement in peace building and improving attitudes, perceptions, and skills about coexistence, trust building and an appreciation of diversity, and the following are specific objectives envisaged to be attained from successful implementation:

• To increased participation, awareness and influence of youth and the community in the promotion of culture, intercultural dialogue, trust-building, and the appreciation of diversity to advance peace-building goals in the country
• To Increased inter-generational and inter-cultural dialogue among communities, notably youth, women, and elders, in peace building and democratization processes
• To enhanced the capacity of the community, notably youth, women, and elders, to be active players in peace building

In line with these objectives, MSD has coordinated the implementation of various activities that are collectively expected to produce the following results

•  Local branches of the national platform are established in the target areas .

• Capacity of young people on life-skills, identity, democracy, rule of law and active citizenship.
• Indigenous structures are strengthened to be able to play active role in the conflict resolution.

• Dialogue sessions organized and establish open communication amongst representatives of the conflicting parties on the root causes of conflicts, their possible solutions, ethnicity, diversity, identity, democracy, governance, and other related subjects

MSD has deployed technical staff for implementation and maintained a system of progress tracking and reporting to the donor as well as other key stakeholders to the project. Now, with the completion of the implementation period MSD has decided to commission and engage an external consultant to evaluate the overall performance of implementation.

 The evaluation will assess the approaches applied for achieving results, with due emphasis on efforts related to capacity building and partnerships (the interplay between the efforts of Partners, Associates, target groups and associations of beneficiaries).

 Purpose and objectives

 The overall purpose of the final evaluation is: to contribute to strengthening accountability to key stakeholders and the internal learning of MSD with regard to performances in programme implementation, generating lessons learnt and assuring result orientation.

This evaluation, within the overall thematic area of Good Governance and Human Right has the following specific objectives:

•  To identify and assess outputs and outcomes of activities implemented as part of the project Peace Matters: Dialogue for Peaceful Coexistence. To the extent possible the contribution of the activities to the overall objectives of the project as well as to that of the mission of MSD will be assessed.
• In comparison to expected results in project document, assess how and to what extent the approaches applied by MSD contribute to alleviate peace and security problems of the project areas, stakeholders capacity has been enhanced, skills of beneficiaries in conflict resolution, good governance, mediation, arbitration, life skills has been enhanced.

Relevance – among others, how well matched are the project interventions to the socio-cultural, institutional, political and economic contexts of the target groups and the country/region? Were opportunities, entry points and risks clearly articulated and employed or managed?

Efficiency – Are projects/program inputs consistent with efficient achievement of outputs and outcomes? Have project funds been disbursed in ways consistent with efficient achievement of objectives? Have all partners been able to provide their contributions to the project? Given objectives, were alternative approaches available that could have been used resources more efficiently?

Effectiveness – Have the interventions achieved or are likely to achieve their objectives? Have the Objectively Verifiable Indicators (OVI’s); targets according to the logframe, been achieved as planned to date? What is the quality of results achieved? What is the likelihood of the Specific Objectives to be achieved as envisaged and measured in the OVI’s of the Logframe? To what extent is the choice of intervention channels and mechanisms adequate to achieve the intended results? What are the main obstacles/challenges to achieve the intended results?  Has there been any change in objectives? What explains any non-achievement of objectives? Are there any unexpected outcomes? 

Impact: Did the assumptions at project purpose (Specific Objectives) level remained true over the implementation period? What are the key achievements of the interventions in terms of policy, practice and behaviour change? What are the intended/envisaged pathways from project results to broader overall development outcomes?

Sustainability: How likely is it that outputs and outcomes of the interventions will be sustained? What actions/conditions have been put in place to sustain changes in channels and mechanisms? Are capacities supported by the interventions likely to be sustained and result in sustained improvements in the future?

Coherence: How compatible was the intervention with other interventions in the country, sector or institution. Internal coherence addresses the synergies and interlinkages between the intervention and other interventions carried out by the same institution/government, as well as the consistency of the intervention with the relevant international norms and standards to which that institution/government adheres.

External coherence considers the consistency of the intervention with other actors’ interventions in the same context. This includes complementarity, harmonisation and co-ordination with others, and the extent to which the intervention is adding value while avoiding duplication of effort.

Assess the usefulness of applied Monitoring and Evaluation systems for achievement, documentation and communication of results.

Scope

MSD wants to maximize the relevance and learning potential of the evaluation. Thus, for this evaluation should be based on a wider investigation of country contexts and specific activity portfolios. This places strong emphasis on the role of the inception phase for specifying the scope, in accordance with the considerations outlined below.

Thematic focus – The project is financed by EU/CSF III the overall objective of which is to increase the contribution of citizens and CSOs in the development and democratization process of the country. Accordingly, important focus areas of this evaluation include:

• Capacity strengthening of CSOs and their networks to engage in policy dialogue and effectively influence Government’s reform processes and nation-building interventions at Regional and Federal levels;
•  Partnership opportunities amongst CSOs (including the media and academia), building on existing networks, supporting emerging networks, and encouraging the building of coalitions and the creation of platforms to exchange knowledge;
•  Support the creation of an enabling environment to ensure the development of a credible, accountable, independent, inclusive civil society sector and undertake strategic engagement with all levels of Government.
•  The chain of results – The project is primarily about contribute towards the national effort in enhancing citizens’ engagement in peace building and improving attitudes, perceptions, and skills about coexistence, trust building and an appreciation of diversity. Thus the evaluation will primarily check the achievements of the high-level results and their indicators in the approved Logical Framework Matrix (LogFrame) of the project. The same can be said for improved participation and advocacy efforts. The evaluation will look for indications of broader development outcomes where possible, but the assessment will primarily be based on the work of Mahibere Hiwot for Social Development as a lead applicant and Hiwot Ethiopia (HE), Young Men Christian Association (YMCA), Save Your Holy Land Association (SYHLA), Eshet Children and Youth Development Organization (ECYDO), as co-applicants of the project.
•  Monitoring and Evaluation information for accountability and learning  The evaluation will among others examine how results and results monitoring are aligned with broader Overall Objectives (with reference to the indicators included in the Logframe of the project proposal),.

 Geographic/Regional focus:

The evaluation will as point of departure consider project activities in Amhara, Addis Ababa, Benshangu Gumuz, Oromia, Sidama, SNNPR, and Tigray regions. The specific locations of the evaluation visit will be decided during the inception phase.   

 Time frame:  The evaluation is expected to cover the activities implemented by MSD during the period from February, 2020 to February 2023. This will allow focusing on activities and results related to the specific project, and analyzing the temporal aspects and their implication for the potential achievement of results.

 NB. In light of the possible supports from other donors for related activities, the evaluation need to be careful in attributing results to CSF III and others. Also, note that the report will be presented and reviewed by the donor and other key stakeholders.

 Approach and methodology.

The evaluation falls in two main parts: As part of the inception phase, a desk study will be undertaken by the Consultant to understand the details of the project, the broader context and develop details of the evaluation question, prepare instruments and plan subsequent stages of the task. The second part is the more in-depth information/data collection and analysis of activities, based on discussions and interviews with, Hiwot Ethiopia (HE), Young Men Christian Association (YMCA), and Save Your Holy Land Association (SYHLA), Eshet Children and Youth Development Organization (ECYDO) and government stakeholders.

•  The evaluation will be based on quantitative and qualitative methods. The specific methodology and analytical framework will be finalized during the inception phase, and will be approved by MSD. The Consultant will ensure triangulation of findings by applying a variety of data collection methods comprising desk research and analysis of existing material; interviews and focus-group discussions with key informants, stakeholders’ workshops and direct observation of program implementation and impact in the field.

 Desk study: The first point of departure will be the project specific documentation available from MSD. In addition, the Consultant will review reports/correspondences held by and to be made available from MSD; collaborating government sources, from resource organizations (Network/Apex organizations), higher learning institutions, the media and the likes. Such reports may include: reviews and profiles of CSO institutional and organizational capacities; the approaches and strategies mostly promoted; internal networking for shared learning and joint actions; their participation and/or contributions in the overall national/regional development processes; status/trends in their partnerships with government and donors; status/trends in the operational environment, and the likes.  

Workshops: The Consultant is expected to present detailed proposals and the final evaluation outputs to MSD, including group-events, where applicable. In the first place, the inception report (output of the desk review) will be presented to the reference point to be designated by MSD or to the Meeting of the Project Coordination Committee. This encounter is anticipated to enable the Consultant to test the evaluation questions and get endorsement on the proposed approaches to finalizing the assignment. At the finalization phase, a second workshop will possibly be organized so as to help validate the main findings, contribute to cross learning between organizations and facilitate buy-in from key stakeholders (dissemination).

Interviews and discussions with key stakeholders: Interviews should be conducted with key informants primarily face to face and otherwise by phone. Key stakeholders include the implementing Partner CSOs/CSOs Hiwot Ethiopia (HE), Young Men Christian Association (YMCA), and Save Your Holy Land Association (SYHLA), Eshet Children and Youth Development Organization (ECYDO), the collaborating governmental departments Regional and Zonal Peace and security Bureaus, Women and Social Affairs offices Youth League offices across all respective regions and Zonal offices  as per Agreements entered, selected project beneficiaries from amongst  youth, elders, women and platform members  and the donor (EUD/TAU). The Interviews will be conducted in semi-structured form. Key areas for discussion will be identified and questionnaires or interview guides will be elaborated. There would be an interview round during the inception phase (with core technical staff directly working on the project and with representative of the donor) to clarify expectations. Following the inception phase, the second and major round of interview will be conducted by reaching the key stakeholders of the project mentioned above, both within the base-city of the project and in implementation areas.

 Guiding Principles: The evaluation process and evaluators must be sensitive to beliefs, manners, and customs of the social and cultural environment in which they will work. Especially, the evaluation team must be sensitive to and address issues of protection, discrimination and gender inequality.

 Fieldtrips:  The Consultant is expected to undertake (at least) Central Gonder, Chagni field trips to Addis Ababa, Assela, Wolita spend at least 10 to 15 days on field trips. The purpose of the fieldtrips will be to collect information on the implementation of the project, both by MSD and ECYDO, YMCA, HE, and SYHLA. The Consultant will have the opportunity to interview field-based staff, project beneficiaries and relevant stakeholders and Partners.

Organisation of the Evaluation

 The evaluation will be carried out by an independent, individual external consultant to be selected through open, competitive and fair selection procedure. (Unless a team of independent, external consultants are specifically required). MSD will be responsible for managing the process and for facilitating initial contact of the Consultant with implementing partners, final beneficiaries and other stakeholders. A reference group (focal group) will be established at the head office of MSD to facilitate and provide smooth coordination of the evaluation task.

Output

The main outputs of the evaluation consist of the following:

Inception report – not exceeding 15 pages excluding annexes. The inception report should include the results of the desk review, the detailed proposal on methodologies and the elaborated analytical framework for the evaluation, survey instruments, timeline, and process progress indicators. MSD (through the reference group) will review/comment and endorse the inception report as appropriate for the Consultant to commence the next stage activities.

Evaluation report, draft and final version, not exceeding 40 pages excluding annexes. The report will be prepared in English. The draft version will be first reviewed and commented by MSD (reference group) before presenting it to the final validation workshop of key stakeholders, likewise, the final version will incorporate feedbacks solicited from the Client as well as from the final validation workshop. The final report must include an executive summary of not more than 4 pages.

In addition to the above, MSD is expected to prepare and present concise summary of the findings of the evaluation during the validation workshop to be organized by MSD.

 Work plan

The preliminary work plan is as follows:

• 15 January 2023 signing of contract, initiation of work.
• 25 January 2023 Inception Phase – desk study of project and other relevant documents; interviews and discussions with core staff of the Client and representative of the donor; design of detailed evaluation methods, instruments; identification of informants and developing detailed schedule of activities.
• 30 January 2023 – presenting and endorsement of the Inception Report by the Client,
• From 15 February to 25 March 2023 – Information collection (from Addis Ababa and Regional sources), including field work as appropriate, along the methodology endorsed, and write-up of the draft report
•  April 2023: submit draft report; plan and prepare for debriefing/validation workshop. Final report has to be delivered to the Client within two calendar week after the above validation workshop.

Competencies

 The evaluation team should have the qualification, knowledge and experience appropriate to the purpose and scope of the evaluation. Required competencies are (need to show proof of evidence from prior assignments):

• First, preferably second, degree in relevant fields such as Development Studies, Peace and security studies, International development, Project Management and related.
• At least 5 assignments completed in the evaluation of projects/programmes; with at least 2 of them with CSO projects;
• The consultant/s shall have knowledge about CSOs and good grasp of the context and challenges of the operational environment;
• Knowledge and experience with evaluation methodologies (both qualitative and quantitative methods).
• Knowledge of English and one or more local languages.

 Preferable competencies:

• International experience in evaluating CSO projects
• Knowledge and experience in of applying gender balanced and participatory approaches to the exercise.

 Background documents

 The following are among the key background documents will be available for this evaluation:

•  The Strategic Plan, and Program document of (MSD);
• The final Full Application set of documents including: The Full Description, LogFrame, Budget and Contracts with their Annexes approved by the EUD;
• All interim and final reports with narrative and financial components submitted to the EUD;
• Mid-term review report of the Project (if any),
• MoUs signed among partners and project implementation agreement with the government partners;
• Baseline survey report;
• Intermediate outputs of the project that include training manuals, publications, toolkits, research reports, etc. produced by the Project,

Lien