Result of ServiceACROSS Project successfully evaluated. Work LocationHome-Based Expected duration2 Months Duties and Responsibilities1. Introduction The United Nations Human Settlements Programme, UN-Habitat, is mandated by the UN General Assembly to promote socially and environmentally sustainable towns and cities with the goal of providing adequate shelter for all. UN-Habitat works with varied partners including cities, central governments, local authorities, non-governmental organizations, and the private sector in the implementation of the Habitat Agenda and Sustainable Development Goals (SDGs) in particular, Goal 11 of \"Making cities and human settlements inclusive, safe, resilient and sustainable\". This TOR concerns evaluation of the project “the Global Alliance of Cities for Road Safety (ACROS), implemented with the objective of “providing an open platform for cities to be inspired, learn and exchange in order to find the most appropriate local solutions to solve their road safety challenges while contributing to the climate goals.” The project has been funded by the UN Road Safety Fund (UNRSF) and have been implemented with a total contribution of 500,000 USD. The project has been implemented by UN-Habitat Headquarters in partnership with the United Nations Environment Programme (UNEP), the UN Economic and Social Commission for Western Asia (UN ESCWA), the United Nations Economic Commission for Africa (UNECA) and the Institute for Transportation and Development Policy (ITDP). ACROS focused on the following countries in Africa (South Africa, Mozambique, Kenya, Ethiopia, Rwanda, Uganda, Liberia) and Eastern Mediterranean (Jordan, Egypt, Lebanon). The evaluation will assess the entire implementation period of the project which spans from 2023-2025. The UNRSF mandates that UN-Habitat, as the main implementing partner, shall allocate resources to commission an external evaluation of the project achievements, including contributions to the overall impacts of the Fund pursuant to the Project Monitoring & Evaluation Framework.;l; Such evaluations form a part of the reporting required under the operation manual. The evaluation exercise shall be carried out by an external independent evaluator. 2. Project Context Addressing the road safety challenge comprehensively calls for sufficient technical and financial capacity of local governments. The reality shows, however, that “the lack of specialist knowledge of creating safer roads, vehicles and road user behaviour, and for designing and operating well-functioning post-crash systems is a major barrier in many countries. Further, many countries and cities lack expertise in adapting Safe System principles to local conditions” (Global Plan 2021-2030). Many cities, particularly in Africa and the Eastern Mediterranean Region, have been slow in addressing the road safety challenges. Priority is still given towards car-oriented transport policies that facilitate highway development, high speeds and urban sprawl. This is exacerbated by the lack of data on road safety, poor enforcement mechanisms, limited resource allocation from the national to the local level, lack of relevant city policies and action plans, as well as outdated regulations on urban and transport planning. Road safety has become one of the biggest public health and safety challenges which countries in Africa and the Arab region currently face with regional rates of 26.6 and 17.9 deaths per 100,000 population respectively. This situation is particularly alarming for vulnerable groups due to their age, gender, disability or poverty. While the emphasis of interventions often focus on the national level, cities have a critical role to play in complementing these actions through reinforcing national policies and implementing projects on the ground. Cities are responsible for prioritizing active mobility, upgrading public transport, introducing cleaner and safer technology for vehicles, or building public support for road safety through awareness campaigns. The ACROS project recognized that there is a great thirst from cities to exchange lessons learnt and inspiring practices among each other, at a moment when there is no time for further delays. To respond to the aforementioned challenges, this project proposed to establish the Alliance of Cities for Road Safety (ACRoS) that aimed to become a one-stop shop for cities to engage in capacity building, receive technical advice, implement catalytic action and benefit from city-to-city exchange. The main objective of ACRoS was to provide an open platform for cities to be inspired, learn and exchange in order to find the most appropriate local solutions to solve their road safety challenges while contributing to the climate goals. The project intended to have a significant impact on the reduction of road safety fatalities in African and Eastern Mediterranean cities. Anchored in the principles of the “Safe System Approach”, the project took a comprehensive view and proposed the following 3 outcomes: - Outcome 1: Cities are inspired and capacitated to improve their road safety management in alignment with the Safe Systems Approach, and complementary to the national policy framework (Capacity) - Outcome 2: Selected pilot initiatives contribute to improved road safety and reduced fatalities at city and neighborhood level with strong participation of vulnerable groups (Catalytic Action) - Outcome 3: Additional investment in sustainable mobility and road safety projects is secured to scale-up interventions in selected cities (Investments for scale-up) Key stakeholders and implementing partners The Mobility team in UN-Habitat’s HQ has led and coordinated the main activities, supported by UNEP’s mobility team on developing frameworks for active mobility for the African continent, and UNECA and UNESCWA to enhance outreach to other cities and countries in the region. ITDP Africa has been the main external implementing partner supporting project activities through provision of technical expertise. 3. Evaluation Purpose, Objectives and Scope The end of project evaluation aims to serve dual purposes of accountability and learning. It is intended to strengthen accountability by providing UN-Habitat and other implementing partners (UNEP, UNESCWA, UNECA and ITDP), the donor (UNRSF), target countries and cities, and other key stakeholders with an independent appraisal and illustrate useful and credible evidence of what the project achieved in terms of the planned results with the resources used. Also, the evaluation is intended to serve the purpose of contributing to enhanced learning to understand what worked well, what did not, operational experience, opportunities and challenges. Evaluation findings, lessons learned and recommendations are expected to be used and feed into decision-making processes for continuation or scaling up of this or similar projects, as well as improvement of future cooperation agreements. Specific evaluation objectives are: (i) To assess the performance of the project in terms of achieving the planned results at objective, expected accomplishments (outcomes) and output levels. This will entail analysis of actual versus planned results. (ii) To assess impact, relevance, effectiveness, and sustainability of the project. (iii) To assess the adequacy of planning, resources, working arrangements, coordination, cooperation and partnership modalities and management and how they contributed to achieving the planned results of the project. (iv) To assess the extent to which cross-cutting issues such as environmental sustainability, social inclusion issues of gender equality, human rights, and youth were integrated in design, implementation, monitoring and reporting of the project. (v) To identify lessons learned and propose forward-looking recommendations that are strategic, programmatic and managerial for decision-making considerations. In terms of the scope, the logical framework of the project will serve as the basis of the evaluation. The evaluation will cover the planning, implementation and monitoring and reporting of the project for the whole period of the implementation phase (2023-2025). 4. Evaluation Criteria and Questions The following evaluation criteria, aligned with the OECD DAC evaluation criteria, will be applied and outline the key questions: Impact, Relevance, Effectiveness, Sustainability Impact Outlook • To what extend did the project attain the UNRSF goal to reduce road traffic fatalities and injuries in low and middle-income countries? • What positive and/or transformative changes have occurred because of the project? • Have the projects had an impact on the achievement of goals stated in global frameworks such as the Sustainable Development Goals (SGDs), the New Urban Agenda and the UN-Habitat Strategic Plan? Relevance • To what extent was the project and its objectives relevant to the needs and priorities of the participating countries and cities and responded to their development plans? • How did the conceptualisation, design and implementation of project activities influence the impact of the projects? Effectiveness • What results have been achieved and which ones have not been achieved, with the support of the institutional, management set-up, and staff capacity? • How effectively has UN-Habitat and other implementing partners credibly monitored the implementation of the programme, using the indicators of achievements on outcomes to provide evidence on performance and flag any necessary adjustments to improve delivery of the project? • How effectively was the project engaging with countries and cities to achieve desired outcomes? • Were the project indicators well defined and easy to measure, in a way that the project’s impact could be tracked? Have any flaws been identified in the definition of Outcomes/Outputs/Activities? Sustainability • To what extent did activities engage beneficiaries in design, implementation and building ownership of the beneficiaries? • To what extent, if relevant, will the outputs and project’s activities be replicated or scaled up? • Is access to financing for further developments secured? If yes, how; if not, how could this be improved? • To what extent was capacity developed and what mechanisms are put in place to ensure sustainability of the results and benefits achieved? • To what extend have the management structures been efficient in developing partnerships, also with other relevant projects? Cross Cutting • To what extent were gender equality, youth, disability, environment and human rights considerations reflected in project design and implementation? • Were specific strategies or measures implemented to promote inclusion of marginalized or underrepresented groups? • How did stakeholder engagement approaches reflect principles of inclusion and participation? 5. Evaluation Approach and Methodology Approach The end of project evaluation will be carried out by one external independent consultant in collaboration with the relevant stakeholders. The evaluation approach should be as participatory as possible and utilization-focused. The evaluation will be carried out in conformity with evaluation Norms and Standards of the United Nations System and best practices in the evaluation field. A Theory of Change (ToC) will be (re)constructed to illustrate how the project was intended to achieve its objectives and how it actually achieved them. This will involve describing the causal logic linking inputs, activities, expected accomplishments, and the broader results-based management approach of UN-Habitat, as well as identifying the conditions and assumptions required for these causal changes to occur. It will assess the project’s performance against its stated objectives, using the selected OECD-DAC criteria and assess cross-cutting issues. The evaluation will be conducted in an inclusive, consultative, and transparent manner engaging partners and stakeholders throughout the process. Methods A variety of methods will be applied to collect evaluative information, including: Review of relevant documents. The consultants will devote the first two weeks to review relevant documents and draw data related to the evaluation questions. Relevant documentation will include the Partner Agreements, individual project documents, progress reports, project outcomes, available public data (mortality rates, number of crashes, etc.), etc. The document review will inform the evaluator what data is available and provide overview of project design and performance. It will also identify specific issues to follow-up during interviews. Key informant interviews and consultations with key stakeholders. Interviews will be conducted to obtain qualitative information on the evaluation issues to enable the evaluators address the issues of impact, relevance, effectiveness and sustainability of the project. Interviews shall be conducted with stakeholders of the partner organisations and, in case of physical infrastructure implementation, the communities that were involved in the participatory processes and now experience the change. Secondary data analysis. The consultant will review existing data sources if available i.e. National Road Safety Observatories, Ministries of Transport, Road Safety Authorities etc. Field visits. If feasible, field visits will be conducted to observe implementation sites of the pilot initiatives and directly engage with local stakeholders. The evaluation consultant will describe expected data analysis and instruments to be used in the evaluation inception report. Presentation of the evaluation findings should follow a standard format of the UN-Habitat Evaluation report (to be provided). 6. Stakeholders’ participation One of the key determinants of evaluation utilization is the extent to which clients and stakeholders are meaningfully involved in the evaluation process. It is expected that this evaluation will be participatory, involving key stakeholders: UN-Habitat Management, consultants, staff involved in the design, implementation, and reporting, project partners, beneficiaries of the project, and UNRSF as the donor. Stakeholders will be kept informed of the evaluation processes including design, information collection, evaluation reporting and results dissemination to create a positive attitude for evaluation-utilization. The evaluation shall be carried out by an international evaluation consultant. The consultant will be independent of activities to be evaluated and has no stake in the outcome of the evaluation. The evaluation consultant will be responsible for the quality of work and preparation of the specified evaluation deliverables. 7. Evaluation consultant’s skills and experiences The evaluation will be conducted by an independent external evaluation consultant. He/she must have proven experience in evaluating project/programmes and should have knowledge reconstructing Theory of Changes and of Results-Based Management and strong methodological and analytical skills. In addition, the consultant should have: • Advanced academic degree in evaluation, policy analysis, social sciences, urban development, or related fields. • Advanced knowledge and understanding of conducting evaluations and participatory processes and delivering specified outputs, presenting credible findings derived from evidence and putting conclusions and recommendations supported by the evaluation findings. • Advanced professional experience in urban development, urban mobility, road safety or related fields. • Very good inter-personal skills, with commitment to timeliness and high quality. • Proven record of quantitative and qualitative data analysis • Established network with governmental organisations and institutions working on road safety policies. • Excellent writing skills in English are essential as most data collection will be done in English. • Knowledge and understanding of evaluations in the UN system, as well as UN-Habitat mandate and its operations. 8. Evaluation Management and Responsibilities Impartiality and Independence are important principles of the evaluation because they ensure credibility of the evaluation and avoid a conflict of interest. For this purpose, officers responsible for design and implementation of the project shall not conduct the evaluation. This is a decentralised evaluation, commissioned and managed by the project team at the Urban Basic Services Section. The Urban Basic Services Section and the project implementation team will be responsible for the management of the evaluation as well for providing information, documentation required as well as providing contacts of stakeholders to engage with for provision of evaluation information. The independent Evaluation Unit will (upon request) provide guidance on the evaluation process; ensuring that the evaluation is conducted by a suitable evaluation team; providing technical support and advice on methodology; explaining evaluation standards and ensuring they are respected; ensuring contractual requirements are met; review outputs (TOR, Inception Reports; draft and final evaluation reports); The project team is responsible for sharing the evaluation results; supporting use and follow-up of the implementation of the evaluation recommendations. The external evaluation consultant should have both substantive and evaluation expertise. He/she will be responsible for meeting professional and ethical standards in planning and conducting the evaluation and producing the expected deliverables (Outputs). Arrangements for consultation and review will be put in place to maximize the quality and credibility of the evaluation. A Reference Group will be established comprising of representatives from UN-Habitat, the other partner UN agencies, ITDP Africa and UNRSF to serve in advisory capacity, having responsibilities of reviewing and commenting on the main evaluation outputs (draft and final evaluation report) Qualifications/special skillsHe/she must have proven experience in evaluating project/programmes and should have knowledge reconstructing Theory of Changes and of Results-Based Management and strong methodological and analytical skills Advanced knowledge and understanding of conducting evaluations and participatory processes and delivering specified outputs, presenting credible findings derived from evidence and putting conclusions and recommendations supported by the evaluation findings is required. Advanced professional experience in urban development, urban mobility, road safety or related fields is required. Very good inter-personal skills, with commitment to timeliness and high quality is required. Proven record of quantitative and qualitative data analysis is required. Established network with governmental organisations and institutions working on road safety policies is required. Excellent writing skills in English are essential as most data collection will be done in English is required. Knowledge and understanding of evaluations in the UN system, as well as UN-Habitat mandate and its operations is required. LanguagesEnglish and French are the working languages of the United Nations Secretariat. For this evaluation, fluency in written and oral English is required. Working knowledge of other UN official languages is desirable. Additional InformationNot available. No FeeTHE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.