1.     Project background and context

Mango is a major food and cash crop in Ethiopia. It provides the much-needed income for millions of resource-poor mango growers and other stakeholders involved along the mango value chain, including women and youth. The major constraints to mango production in Ethiopia include lack of knowledge and skills in improved production practices such proper spacing, pruning, managing soil fertility, limited planting materials/improved varieties, and postharvest handling. Damage due to diseases and insect pests are another constraint. Among pests of mango, the white mango scale Aulacaspis tubercularis and the oriental fruit fly Bactrocera dorsalis, are both invasive species causing 80-100% loss under heavy infestation. In a desperate effort to manage these pests, mango growers have resorted to excessive use synthetic insecticides, which are hazardous to human and environmental health. Besides, limited access to agricultural input supplies, downstream value chain actors such as consumers and processors are constrained by ineffective fruit value chain developments which leads to great price fluctuations and few market provisions. Recently, however, with the expansion of agro-industries, there are indications for value addition and market expansion for fruit sector in Ethiopia.

Mango production in Ethiopia is characterized by fragmented and isolated approaches to addressing specific production constraints in an ad-hoc manner leading to unsustainable production system. There is a strong need for a holistic approach when tackling mango production challenges which entails a better understanding of interconnections between different mango production activities and agro-ecosystems. The current project (since Jan 2020 to Dec 2023) is, therefore, designed to follow an integrated approach that address key mango production challenges including nursery and orchard management, soil fertility, pest management, post-harvest handling and marketing. An integrated mango production is a holistic approach that uses agroecological solutions to achieve sustainable mango farming. An integrated approach examines farmers’ local practices, conduct agronomic appraisal and combine with local innovation and improved technologies to develop comprehensive tactic to building integrated production systems adapted to farmers’ constraints and objectives. This approach will deliver sustainable solution that will improve production practice and fruit quality and safeguard the environment and natural resources and advances the livelihoods of the surrounding communities.

The current project enhances mango production and productivity by reducing fruit losses along the mango value chain through scaling of improved agronomic practices (composting, use of improved planting materials, orchard sanitation), application of eco-friendly IPM technologies for managing key pests of mango and improved postharvest handling. The project also aims to enhance employment opportunities for women and youth along the value chain.

2.     Purpose of the Project

To provide evidence that eco-friendly production of mango is economically feasible and environmentally sustainable.

3.     Project objectives (outcome)

·       to increase an integrated mango production practices by smallholder growers

·       to improve mango market linkages through value chain analysis and

·       to improve awareness among stakeholders in integrated approach to mango production

4.     Purpose of the evaluation

The final evaluation aims to (i) assess the quality of design and implementation arrangements; (iii) to assess progress and gaps in achieving planned outputs and outcomes and risks affecting the implementation of the project; and (iii) to identify lessons and opportunities from project implementation and draw recommendations to sustain project results after the project has been completed. The evidence generated by the evaluation could also help mobilize resources for the second phase of the project or similar project formulation.

5.     Scope and objective of the Evaluation

The evaluation will assess the results achieved during the project implementation periods. It will cover all project intervention villages in Arba Minch. The evaluation aims to generate robust information the Biovision and icipe could use to make decisions, act or add to their knowledge that boosts the project’s sustainability. It has the following specific objectives:

·       To assess the project’s contributions to increased practice of integrated mango production promoted by smallholder growers

·       To evaluate the project’s contributions in creating market linkage between mango producers and private sectors.

·       To examine the contribution of the project in building local capacities of the project area stakeholders in integrated mango production.

·       To identify and document challenges and lessons learned and provide recommendations to enhance the performance of the mango sub-sector by addressing constraints along the value chain.    

To achieve these objectives, the evaluation will address key evaluation questions organized around six evaluation criteria outlined below. The evaluators are welcome to adjust and fine-tune the EQs.

 6.     Audience of the Evaluation

The primary audiences for the evaluation are Biovision, icipe, and local implementing partners.

7.  Suggested Evaluation Questions

The evaluation should address the following key questions:

Relevance: The extent the project meets the priorities and policies of the country and regional government where the project operates and the needs of the mango growers.

·       To what extent is the project responding to the demands and priorities of the targeted districts, farmers, and partners?

·       How well does the project address the needs of farmers to access markets and add value to mango products?

·       How does the project align with the Ethiopian government’s agriculture/horticulture sector priorities? What effect has this had on the success of the project?

Effectiveness:  The extent to which the project has achieved its objectives and results and foster partnerships.

·       To what extent is the project   achieving its output targets?  To what extent are the outputs contributing to the desired outcomes?  Are the activities and outputs of the project consistent with the overall goals and intended impacts?

·       Has the project demonstrated new practices/approaches for integrated mango production?

·       To what extent is the various training provided to local partners, youth groups and farmers effective?

·       To what extent has the project considered postharvest fruit loss and value addition activities (processing)?

·       Has the project enhanced youth groups in business and job creation?

·       To what extent is the project created awareness among farmers and partners experts on the importance of integrated mango production?

·       To what extent is the project fostering partnership (private and public) at different levels? How have these partnerships influenced (positively or negatively) the achievements of the project’s outputs and outcomes?

 Efficiency and value for money:  Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. Value for Money (VfM) is maximizing the impact of each pound spent to improve poor people’s lives.” (DFID, 2011).

·       How well is the project deploying its financial, human, and time resources?

o   Were activities cost-efficient?

o   Were objectives achieved on time?

o   Was the project implemented most efficiently compared to alternatives? What could cost-savings opportunities be implemented without compromising project quality?

·       Has the project bought inputs of appropriate quality at the right price?

·       How well are the outputs from an intervention achieving the desired outcome?

Impact: The positive and negative changes the project produces (directly or indirectly, intended, or unintended).

·       Has the project improved the economic status of the project beneficiaries?

·       Has the project enhanced beneficiaries’ knowledge and skills in an integrated approach to mango production?

·       Have any changes been achieved in the mango sub-sector policy, practices, and attitudes of experts, decision makers where the project has been implemented?

 Sustainability:  Whether the benefits of an activity are likely to continue after the project has been phased-out

·       What actions and strategies have been put in place through the project support to ensure project results sustainability, including institutional, financial, and environmental sustainability?

·       Has the project contributed to institutional and human capacities to sustain project activities and results beyond the project period?

·       What strategies can be used to improve and sustain an integrated agroecological approach to mango production in the intervention area in particular? Are these activities likely to be sustained in the future and why?

·       What is the level of the current state of inclusion of local partners in the project? How can this influence the sustainability of the project?

·       What additional steps can be taken to support project sustainability?

 Cross-cutting issues: gender, vulnerable groups (women and youth), and environmental considerations in its design and implementation?  

·       What measures have been taken to ‘mainstream’ gender in the project? To what extent have women and youth been involved in the project?

·       What environmental benefits have been achieved and measures have been taken to ensure environmental considerations are central to the project, and to what extent have they been successful?

 8.     Evaluation Methods

To answer the evaluation questions, the potential evaluation team/consultants are expected to use diverse methods and tools and information triangulation approaches.  The evaluation will adopt qualitative tools such as meetings, desk review of documents, focus group discussions, key informant interviews, and field observations, in addition to collecting individual data from beneficiaries using semi-structured survey tools. The project log-frame will be used as the basis to understand the contributions made by the project.

 9.     Main Outputs And Deliverables

The following outputs are expected from the evaluator:

A.    Evaluation Inception Report includes the following contents:

·       Executive Summary (brief introduction, methodology)

·       Brief background on the project

·       Purpose, scope, and stakeholders of evaluation

·       Evaluability assessment

·       Review and fine-tuning of the evaluation questions.

·       Methodology and tools

·       Evaluation work plan

·       Annexes:

o   Detailed evaluation methodology (if any), data collection tools.

o   The evaluation design matrix indicates the evaluation criteria, questions, indicators, and how they can be collected and analyzed.

o   A tentative outline of the final evaluation report

o   The ToR

o   Any other information deemed necessary (detailed tables, figures, etc.)

·       List of stakeholders to be consulted.

 B.    Draft Independent Final Evaluation Report:

·       Presentation of draft findings and recommendations, adhering to the content guidelines of the final report annexed below.

 C.    Final Evaluation Report (maximum of 15-20 pages without annexes) including the following content:

·       Executive summary (brief introduction, key findings by evaluations questions, key bullets conclusions, key bullets recommendations)

·       Introduction  

o   Purpose of the evaluation

o   Intended users

o   Scope and objective of the evaluation

o   Methodology (be brief as this is already available in the inception report)

o   Limitations

·       Evaluation questions-key findings

·       Conclusions and lessons learned.

·       Recommendations

·       Annexes:

o   The ToR, detailed evaluation methodology, evaluation matrix, list of meetings attended, persons interviewed, and documents reviewed, data collection tools, and any other relevant materials.

o   detailed tables and figures

D.   Management response action sheet: the evaluator will prepare summary of major issues and recommendations that needs management response (if any), responsible parties, and timeline.  

 10.   Timeframe of Activities and Workplan

The task is expected to be accomplished in Forty (40) days between April and May 2023. The study team/consultants will collaborate closely with the project team. The evaluation process should consist of the following components/activities:

 11.   Management Arrangements

The consultant will be offered a fixed-price contract to accomplishing all the tasks and deliverables listed above. The consultant is expected to budget for his/her professional fees and subsistence costs. The project will provide transport services, accommodation on bed & breakfast. In addition, the project will organize documents and data requested and facilitate meetings with stakeholders.  The consultant will report to Project manager at icipe for all administrative logistics and field work-related issues and to the M&E expert of the project for all technical evaluation matters. The project manager facilitate access to all relevant data and facilitate meetings with stakeholders and ensure the overall quality control of evaluation deliverables.