TERMS OF REFERENCE
Project Evaluation
Enhancement of the Inclusion of People with Disabilities in the DFAT’s Syrian Humanitarian and Resilience Crisis Package through the Strengthening of DFAT’s MEAL Framework and Support to DFAT’s partner
Project Implementation Date: Started: May .2018 - December. 2020
Author: Inclusion Project Manager & Regional Technical Specialist (Humanitarian Action)
Date drawn up: 12th August.2020
Outraged by the injustice faced by people with disabilities and vulnerable populations, we aspire for a world of solidarity and inclusion, enriched by our differences, where everyone can live in dignity.
Handicap International changed its name and became « Humanity & Inclusion ». Humanity & Inclusion (HI) is an independent and impartial aid and development organization with no religious or political affiliations operating in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and vulnerable people to help meet their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights.
HI has offices in 63 countries; for further information about the association: www.hi.org
HI has been operating in Jordan since 2006, implementing projects ranging from the provision of comprehensive rehabilitation services. Directly, through service providers or through a community-based approach; inclusive local development, disability movement strengthening through capacity building of Disabled People’s Organizations and multi-stakeholder dialogue, among others.
In line with the Jordan Response Plan and the 3RP Regional Refugee and Resilience Plan, HI has always sought to integrate resilience and sustainable elements throughout its interventions, by adopting a twin-track approach which seeks to simultaneously address the direct needs of beneficiaries all the while engaging in dialogue with them, with local authorities and local actors to empower them to claim and address their rights to access these services. and to mainstream disability in policies, programs and services.
Since late 2015 and in line with the evolution of the Syrian crisis into a protracted one, HI adjusted its strategy of intervention to progressively shift from direct to indirect service provision through partners. As of July 2017, the Jordan mission comprises a single entity seeking to ensure the continuity between emergency response, resilience and development.
Since the beginning of the COVID-19 outbreak, the Jordanian government has followed the recommendations and updates provided by the WHO. A series of preventive and control strategies at the national levels have been implemented in order to limit the spread of COVID-19 inside Jordan.
Jordanian government has started to implement strict rules and measures relating to travel, education, religious and social events, as well as working within various industries includes INGOs. The turning point in the country's preventive and control measures was dated as the 17th of March 2020 upon declaring the national defence law in order to mitigate the spread of COVID-19 in Jordan. On the 20th of March 2020, a decision for a nationwide curfew was declared with strict rules on individuals' mobility and extreme fines for violations. During the curfew.
The prevention measures applied by Jordan government had an impact on the implementation of the activities at the field level in both parties (HI and DFAT partners), which lead to stop the activities in the period of mid of March until end of May 2020.
2.1 Presentation of Department of Foreign Affairs and Trade
The project due for review is funded by Department of Foreign Affairs and Trade (DFAT) as outlined below and has supported three DFAT’s partners; Oxfam on livelihood project in Mafraq, UNICEF and Caritas on education project in Zarqa to become more inclusive of persons with disabilities within their interventions. Generally, the project focuses on disability mainstreaming to ensure the inclusion of persons with disabilities within the partners response while empowering them to develop and provide tailored support solutions to ensure they not only fully access but participate in targeted DFAT partners’ interventions. The project involves different staff and at different levels in each organization, which has to be taken into consideration in the process of evaluation. The project has three outcomes explained in the table below:
Project Title
Enhancement of the Inclusion of People with Disabilities in the DFAT’s Syrian Humanitarian and Resilience Crisis Package through the Strengthening of DFAT’s MEAL Framework and Support to DFAT’s partner
Implementation date
May 2018 to December 2020
Location/Intervention zones
Location of project supported activities: Mafraq (Oxfam) and Zarqa (UNICEF and Caritas).
Location of main base: Amman for HI, Oxfam, UNICEF and Caritas.
Supported FAT partners
UNICEF, Caritas and Oxfam
Project Budget
AUD$ 1,000,000
The current action will contribute to the enhancement of the inclusion of people with disabilities in DFAT’s Syria Humanitarian and Resilience Crisis Package through the strengthening of DFAT’s MEAL framework and support to DFAT’s partners
Outcome 1:
DFAT partners are capable of removing barriers that hinder people with disabilities' access to their services and participation to the design of new proposals
Output 1.1
Lists of barriers and solutions to improve inclusion are produced for each targeted project; commitment to barriers removal is reflected either in specific action plan or in visible components in their project’s planning
Output 1.2
At least 120 key personnel trained on inclusion; DFAT’s partner show improved ability to implement actions for inclusion (ex. changes to project operational plans, service delivery pathways, etc.)
Output 1.3
Operational tools (checklists, guidance notes and other user-friendly tools) produced for inclusive project cycle management (PCM) for each targeted project
Outcome 1 Activities:
Participatory selection of DFAT partners’ projects to be targeted and identification of disability champions and disability inclusion focal points.
Carry out participatory barriers identification for key processes and activities (key institutional policies and tools, program planning and M&E tools, assessment tools, service delivery practices, communication pathways and materials, physical infrastructure, etc.) and produce a tailored action plan with detailed list of solutions for the removal of barriers.
Training on disability and inclusion to focal points and participatory assessment to establish a baseline along the five dimensions of inclusion.
Continuous technical coaching to focal persons along the implementation of the action plan including provision of support to improve physical accessibility and adopt reasonable accommodations.
Technical support to adapt MEAL systems: systematic collection and analysis of disability disaggregated data, implementation of accountability mechanisms targeting people with disability, capitalization of good practices and lessons learnt.
Facilitate exchanges and peer to peer learning between targeted DFAT partners.
Use good practices and lessons learnt to support target DFAT partners to produce operational guidelines for inclusive PCM.
Outcome 2:
People with disabilities most at risk of exclusion are empowered to fully access and participate to target DFAT partners’ intervention
Output 2.1
150 people with disabilities most at risk of exclusion receive social support through a personalized support plan
Output 2.2
75 people with disabilities receive support to access comprehensive functional rehabilitation services and other services (referral)
Output 2.3
12 champions of inclusion trained and coached to actively engage with target DFAT partners managers and with DFAT representatives
Outcome 2 Activities:
Identification of people with disabilities within target DFAT partners’ target group and assessment of the empowerment needs of individuals and their close environment (caregiver, household, family).
Definition of a personalized support plan for excluded people with disabilities to overcome challenges that hinder their participation in DFAT partners’ intervention.
As part of the personalized support plan, provision of social accompaniment to the person and his/her family to improve self-confidence, overcome family barriers and to activate facilitators at family and close community levels.
As part of personalized support plan for the individual and family, provide referral and support to access other needed services provided either by HI – such as functional rehabilitation services (physiotherapy, occupational therapy, assistive devices and prosthetics and orthotics and community-based rehabilitation), inclusive employment, early detection and intervention for children under 6 years old – or by any other organization from HI’s large referral network (visual and/or hearing aids, cash for health, legal counselling…).
Facilitate emergence of people with disabilities who are interested and willing to become “champions of inclusion” and contribute to make DFAT partner's PCM inclusive.
Training and coaching of champions to contribute to target DFAT partner's barrier identification and removal and to collect good practices for lesson learning and advocacy.
Training and coaching of champions to contribute to the monitoring of disability inclusiveness in DFAT program, through participation in initiatives to measure performance, gather evidence and inform quality evaluations of annual humanitarian aid quality checks.
Outcome 3:
DFAT Syria Crisis Package MEAL framework related to disability inclusion is enhanced and capable of feeding its inclusion advocacy with good practices
Output 3.1
Monitoring framework that reflects disability inclusion in DFAT partners proposals and interventions
Output 3.2
One synthesis report with good and innovative practices to support disability inclusion in DFAT Syria Crisis Package
Output 3.3
40 key personnel from DFAT partners of Syria Package and other humanitarian actors made aware of good practices for inclusion
Outcome 3 Activities:
Participatory assessment of the level of inclusion in DFAT Syria Crisis Package MEAL framework and its tools.
Adapt the quality framework (indicators, guidance to define target and means of verification) to reflect the change in the level of disability inclusiveness of DFAT partners and the impact of inclusion on people with disabilities’ quality of life.
Define a learning framework to be used by DFAT partners and DFAT personnel.
Support to of DFAT Syria Crisis Package MEAL framework to account for the learnings from Outcome 1 and 2.
Support peer to peer exchanges between DFAT partners key personnel at program management and field levels to share good practices and to influence the design of new proposals.
With each target DFAT partner facilitate mid-term and final lessons learned workshops and production of a capitalization reports.
Produce a synthesis report and hold a final workshop to disseminate findings across all DFAT partners of Syria Package and advocate for inclusion.
Why this evaluation at this specific time?
This evaluation is a Final Evaluation of the project. It will be carried out at the end of the project as systematic process and examination the performance of the project. It is part of the project management cycle to evaluate the project in terms of achieving the target outputs and outcomes. In addition, HI accountable to the donor and learning opportunities to HI based on the findings and conclusions from the assessment of the project’s achievements
Who are the main addresses of the evaluation?
HI provided technical support and capacity building to DFAT partners, namely UNICEF, Caritas and Oxfam, on ensuring the inclusion of persons with disabilities in their programs and services. As such, these partners and their beneficiaries will be the main addresses of this evaluation.
What is the dominant motivation for evaluation, Accountability, learning.
The final evaluation is part of the project management cycle to evaluate the project in terms of achieving the target outputs and outcomes. In addition, HI accountable to the donor and learning opportunities to HI based on the findings and conclusions from the assessment of the project’s achievements.
The main purpose of the final evaluation is to provide independent comprehensive assessment for the project: “Enhancement of the Inclusion of People with Disabilities in the DFAT’s Syrian Humanitarian and Resilience Crisis Package through the Strengthening of DFAT’s MEAL Framework and Support to DFAT’s partner”. Aimed to assess the effectiveness, level of change and sustainability of project approach / interventions, the level of participation, and to provide lessons learnt, recommendation to improve the continuation on the future design and strategies for providing inclusive humanitarian action technical support.
Eventually, the final evaluation process will mobilize all involved stakeholders including; HI, DFAT and their partners (UNICEF, Caritas, and Oxfam) with involvement of persons with disabilities among their projects
· To determine if project design / approach contributed to the enhancement of persons with disabilities women, men, girls and boys having safe and equal access to the services provided by partners.
· Assess the quality of HI intervention in providing technical support and capacity building to relation to DFAT’s Partners (UNICEF, Caritas, and OXFAM) on the inclusion of persons with disabilities in humanitarian action (to mainstream disability inclusion and support stakeholders to enhance inclusive services in the field of education and livelihood projects).
· To identify outcomes and/or impacts of the project, including whether the project activities contributed to changes in knowledge, skills, attitude, practices in partners to be more inclusive of persons with disabilities women, men, girls, and boys in their programmes and services.
· Generate lessons learned from the implementation of the project’s activities and achieved outcomes that will be useful for similar projects in inclusive humanitarian action for the future.
· Develop specific recommendation on the project design, approach and implementation based on the result of the evaluation, where HI was providing technical support to DFAT implementing partners (selected by DFAT) without engaging in a structured consortium approach.
· How and to what extent did the project’s outputs and approach enable to achieve the 3 expected outcomes in relation to each target DFAT’s partners (Caritas, Oxfam, target UNICEF Makani partners)?
· To what extent the achieved outcomes contributed to the project’s overall objective of enhancing the inclusion of people with disabilities in the DFAT’s Syria Humanitarian and Resilience Crisis Package?
· What were the key management/operational/Implementation challenges met by HI during the provision of technical support to DFAT partners and the challenges face by the partners when adapting their programs?
· To what extent did the project contribute to a positive change in p the target DFAT Partners’ knowledge, attitude and skills and / or practices to support inclusion diversity of disability, gender, and age within their program and interventions?
· Are the achieved changes/ effects achieved at an acceptable cost? Compared with an alternative approach to accomplish the same objectives (if applicable)?
· To what extent did the project produce positive changes in the DFAT’s partners’ beneficiaries with disabilities lives (through empowerment and inclusive services)?
· To what extent did the project implementation was affected by the situation of COVID- 19 pandemic
· What mechanisms/ actions have been put in place to ensure the sustainability of project results?
· To what extent does the role of Disability Inclusion Focal Point contribute to sustainable inclusive services among the target DFAT’s partners?
· To what extent did the project manage to put in place mechanisms to enable meaningful participation of persons with disabilities of different ages and gender:
a) within the project itself (the provision of technical support by HI to DFAT partners for disability inclusion);
b) within the target DFAT’s partners projects’ lifecycles (especially the design of new projects and MEAL)
c) within the target DFAT’s partners services
· Has there been sufficient clarity in terms of the collaboration and support, between HI and DFAT’s partners receiving disability inclusion technical assistance?
· How and to what extent the approach of mobilizing disability technical expertise in support to projects which had already been designed and were under implementation posed challenges to HI and DFAT cooperation?
· To what extent target DFAT’s partners consider the collaboration with HI for disability inclusion useful? How do they wish to continue such collaboration in their future programming?
Evaluation methods should be rigorous yet at all times proportionate and appropriate to the context of the project intervention and will be discussed with HI steering committee. The project works across various governorates and sites in Jordan, though all at day-trip distance from Amman, site visits will be arranged in line with the methods selected.
The evaluation will be based on primary and secondary data, in particular documentation directly related to the project – proposal, narrative and financial reports, monitoring datasets, surveys, studies, lessons learned, financial information, other documents produced to analyse individual project components and inform decisions regarding the course of the activities. Other contextual and related information by other organizations will also be made available. Finally, key staff from organization supported involved in the project at different stages will be mobilized for the evaluation – in addition to the whole current project team, former senior management and experts, and project’s beneficiaries with disabilities of different ages and gender.
HI values the contributions of the evaluator towards proposing appropriate, innovative, and robust methods of evaluation. Some basic methodological requirements, however, are that:
The evaluator should try to make tools and evaluation processes of the sample as inclusive as possible for persons with disabilities as much as they possibly can.
The evaluation must be a participative and interactive process.
The evaluation should consist of a series of evaluation tasks that shall be implemented throughout the duration of the evaluation. It will allow to build evidence using a logical sequence of activities.
The methodology should use qualitative data collection techniques and analysis.
The proposed method should also describe how cross-cutting issues of gender and age addressed and incorporated throughout the various stages of the evaluation.
A number of persons will be involve in the evaluation process as detailed below:
· Project targeted staff and partners;
Organization
Portfolio
Role
DFAT
DFAT- Second Secretary Humanitarian, and / or Humanitarian Officer
The project funders and periodically review its implementation giving key recommendation.
HI
Inclusion manager, Inclusion Technical Officers. Regional Inclusion Technical assistance specialist
Your day to day interlocutors regarding the evaluation process including fieldwork, meeting, FGDs …etc.
Oxfam
Disability Focal point partnership officer), WASH program Manager, Deputy SWM Program Manager. Beneficiaries with disabilities
Provide information on HI technical support, as they were one of DFAT partners supported by HI in the project.
UNICEF
Makani Project focal point,
Provide information on HI technical support, as they were one of DFAT partners supported by HI in the project.
ICCS (UNICEF partner)
Disability inclusion focal point, social worker, Facilitators, Beneficiaries with disabilities
Provide information on HI technical support, as they were one of DFAT partners supported by HI in the project.
MOSD (UNICEF Partner)
Disability inclusion focal point,
Provide information on HI technical support, as they were one of DFAT partners supported by HI in the project.
Caritas
Protection officers, psychosocial workers, Beneficiaries with disabilities
Provide information on HI technical support, as they were one of DFAT partners supported by HI in the project.
The Evaluation steering committee will be responsible to validate the proposed evaluation methodology after the closure of the selection process. In addition, the steering committee must approve the inception report and evaluation methodology prior to the commencement of any fieldwork or any other substantive work.
The proposed evaluation methods by the evaluator will be reviewed by HI after the closure of the selection process.
The steering committee members are:
Operation Manager;
Regional MEAL Manager;
Inclusion Project Manager;
Logistics Manager;
Regional Inclusion Technical Assistance Specialist.
The evaluation should consist of 8 phases:
Preparatory phase (briefing with HI team, document review, appreciation-review of the evaluation feasibility), sampling, preparation of data collection tools, logistic arrangements.
Kick – off meeting leading to the inception report presenting the proposed evaluation methodology includes the questionnaire completion and tools for the use of data collections.
Fieldwork - data collection with partner organisations and where relevant some of their beneficiaries.
Communication between the steering committee and the evaluator on the midterm report
Development of the final report based on the feedback of all the reviewers.
Organize workshop for debriefing and presentation of the results orally to the steering committee, includes present the step of filling the end of evaluation form as presented in the standard process.
Data analysis and presentation of preliminary findings (meeting with supported partners to present analysis, conclusions and recommendations that will be used later on by HI to present it in a workshop for DFAT partners and other actors).
Validation of the final report on the basis HI quality checklist and according to the steering committee decision.
The following are the HI institutional polices that should be adhered by the external evaluator and his/her team.
Code of Conduct**
Protection of beneficiaries from sexual exploitation, abuse and harassment**
Child Protection Policy**
Anti-fraud and anti-corruption policy**
As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are considered in the technical offer:
o Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures.
o Ensuring a person/community-centred approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for illiterate audiences / sign language / child-friendly materials, etc.).
o Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.
o Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data. **
*These measures may be adapted during the completion of the inception report.
Mention the involvement of stakeholders and beneficiaries in the evaluation:
Partners are involved in the survey and/or qualitative data collection methods as they are the main direct beneficiaries of the intervention.
Consultation of beneficiaries in the construction of the tools should be considered and adhered by the evaluator.
Consultation and involvement of beneficiaries in the data collection, analysis and reporting is recommended.
It is essential that the process of data collection, as well as storage of data, is supported by careful ethical practice, including informed consent, anonymity and confidentiality, no-harm and protection of data and data storage. Informed consent needs to include awareness of the evaluation data collection process and that the evaluation report may be published and publicly disseminated. Extra precaution must be taken in involving project beneficiaries considering the sensitivity of the thematic issues tackled by this project. To protect the anonymity of communities, partners and stakeholders’ names or identifying features of evaluation participants (such as community position or role) will not be made public.
The evaluator should uphold and respect the following ethical principles:
HI’s protection policies (child protection and protection from sexual exploitation, abuse and harassment …etc.)
Integrity (respect of gender sensitivity issues, especially when performing interviews/focus groups, religion and beliefs, and local norms).
Anonymity and confidentiality.
Independence and objectivity.
Veracity of information.
Coordination spirit.
Intellectual property of information generated during and by the evaluation (including report and annexes) will be transferred to HI and donor.
Quality of report and respect for timelines. Should the quality of the report be manifestly below the contract.
The following are expected deliverables of the evaluation:
· An inception report of no more than 15 pages specifying the proposed methodology for answering evaluation questions and a detailed action plan for the evaluation process.
· A presentation document presenting the first results, conclusion and recommendation, to be presented to the steering committee.
· A final evaluation report of approximately 20 – 30 pages maximum and the following contents:
o Table of contents.
o Abbreviations list
o Executive summary (that can be used as stand- alone document)
o Brief on General Jordanian context at the design and implementation phase.
o Introduction that includes evaluation objectives, methodologies, the used techniques, and limitations of the evaluation where relevant.
o Presentation of the evaluation analysis and findings, covering the five focus areas (Effectiveness, Change, Sustainability, Participation, and Cooperation)
o Conclusion and recommendations with a clear relationship between them.
o Report annexes include, The Final Evaluation Term of Reference; The Data collection tools; list of the people met; list of documents and bibliography as well as composition evaluation form.
The final report should be integrated into the following templet
An end-of-evaluation questionnaire will be given to the evaluator and must be completed by him/her, a member of the Steering Committee and the person in charge of the evaluation.
It is expected that the evaluation process will take 2 months for the period from 1st October to 30th November 2020. A detailed action plan will be submitted as part of the inception report. The final report should validate no later than 21st November 2020, and consolidated report to be finalized by 28th November 2020.
The evaluation expert (or team of experts) who will undertake this assignment should have the following skills, experience and knowledge:
Academic background in Disability, Social Sciences or developmental studies with a minimum of a Master’s Degree in the relevant field.
Experience in project evaluation and related methodologies with DFAT-funded projects is a plus.
Demonstrated Experience in conducting participatory (qualitative and quantitative) evaluation techniques.
A wide experience in all aspects of project cycle management.
Experience working with persons with disabilities in general and/or children with disabilities is an asset.
Practical knowledge of rights-based approaches and Inclusion.
Strong analytical and report writing skills.
Qualified persons with disabilities are encouraged to apply!
In the case of external evaluation: Statement of the budgetary modalities that the candidate must detail in the offer: the cost per day; the breakdown of the time spent per evaluator and per stage of work; the ancillary costs (services and additional documents).
In the case of an internal evaluation, detail the costs related to carrying out the evaluation: transport costs, logistical/meeting costs, translation costs, etc.
the overall cost of the intervention including transport costs (international and local), logistics costs, translation costs; with proposals for payment modalities. To give you an idea, a classic final evaluation costs between 15,000 EUR and 19,000 EUR.
Caution: please note that the last payment is conditional on the validation of the final report and not on the sending of the final report. By validation, we mean validation of the quality and under no circumstances of the appreciation of the project evaluated.
Document and resource of information:
· Project proposal.
· Monitoring Box, (PM Box) which include all project activities with the timeframe. In addition, human recourse involved.
· Letters of intent, Memorandums of understanding, letters of cooperation.
· Interim Donor Reports
· HI Operational and monitoring tools, including; inclusion technical assessments, assessment results report, training reports, pictures, attendance sheets and accessibility reports.
· Supported Partners operational and monitoring tools
· Mid Term Lesson leaned Report
Payment will be processed upon receipt of invoice by bank transfer. The tranches are to be agreed at the time of signature of the contract. In all cases, maximum 30% of the agreed price will be transferred only upon HI validation of the final report.
Bids from interested individual consultants or firms should include:
A detailed technical offer that includes the methodology and evaluation plan, and timeframe.
A detailed financial offer that covers all anticipated costs (taxes, travel, accommodation, transportation, insurance, translation, etc.) in Euro or equvalent local currency.
A CV detailing relevant skills and experience of the consultant and her/his team of no more than 3 pages each, including contactable referees. In case of team of experts, the team leader must be clarified.
At least two samples of a relevant previous evaluation preferably for international donor funded project in a similar area or context.
Note: Humanity & Inclusion reserves the right to accept or reject any proposal without giving reasons and is not bound to accept the lowest or the highest bidder.*
This vacancy is archived.