Result of ServiceThe UNSDCF evaluation has two primary purposes: I. Promote greater learning and operational improvement. The evaluation will provide important information for strengthening programming and results at the country level, specifically informing the analysis, planning and decision-making for the next CF programme cycle and for improving UN coordination at the country level. The UNCT, host government and other CF stakeholders can learn from the process of documenting good practices and lessons learned, which can then be shared with DCO and used for the benefit of other countries. II. Support greater accountability of the UNCT to CF stakeholders. By objectively providing evidence of results achieved within the framework of the CF and assessing the effectiveness of the strategies and interventions used, the evaluation will enable the various stakeholders in the CF process, including national counterparts and donors, to hold the UNCT and other parties accountable for fulfilling their roles and commitments. The objectives of the evaluation are to: 1. Assess the contribution of the CF to national development results through evidence-based judgements using evaluation criteria (accountability). 2. Identify factors that have affected the CF’s contribution to achievement of the national priorities; answering the question of why the performance is as it is; and explaining the enabling factors and bottlenecks (learning). 3. Reach conclusions concerning the UN’s contribution across the scope being examined. 4. Provide actionable recommendations for improving the CF's contribution to achievement of the national priorities, especially for incorporation into the new CF programming cycle. These recommendations should be logically linked to the conclusions and findings of the evaluation and should draw upon lessons learned identified through the evaluation. The evaluation will span almost the 3.5 years period of CF implementation from January 2021 until April 2024. It will include an analysis of the extent of CF implementation coverage across the country through a desk review, evaluating the geographical reach achieved by CF interventions. This assessment will examine whether initiatives primarily focused efforts in the capital city or demonstrated a broader geographical dispersion, encompassing regions beyond Tbilisi. The findings of the evaluation will inform strategic planning and future programming. Primary users of the evaluation will include stakeholders involved in CF implementation, such as the UNCT, governmental bodies, civil society organizations, contributing partners, and other relevant stakeholders. Secondary users will include a broader audience, including the academic and research community, media, and the general public. The evaluation will comprehensively review the UNSDCF 2021-2025, including contributions from all sub-outputs by the UNCT, including those of non-resident agencies. It will also examine cross-cutting issues and global UN programming and humanitarian principles such as leaving no one behind, human rights, gender equality and women's empowerment, disability inclusion, youth engagement, environmental sustainability and resilience, and accountability. Furthermore, the evaluation will consider major global and regional developments during the CF implementation, such as the COVID-19 pandemic, Russia’s war in Ukraine and other conflicts, both in its content (e.g., the UNCT’s responsiveness, adaptation, and reprioritization) and its operation (e.g., methods for managing stakeholder participation and inclusiveness). However, it will not evaluate individual programmes or activities of UN agencies. Nevertheless, the evaluation team may suggest specific thematic areas for review after consulting with stakeholders as necessary. The evaluation of the CF will be conducted based on a set of predetermined evaluation criteria, which encompass: Relevance and adaptability, Coherence, Effectiveness, Efficiency, Coordination, Sustainability, Orientation towards impact, and connectedness. The evaluator has the flexibility to adjust these criteria and questions, subject to mutual agreement with the Evaluation Manager. The final evaluation questions and matrix will be established by the evaluation team in the inception report. Below are the initial suggested evaluation questions and the evaluation criteria, which includes OECD-DAC evaluation criteria: Criteria: Suggested evaluation questions Relevance and adaptability: IS THE CF DOING THE RIGHT THINGS AND ADAPTED WELL TO EMERGING NEEDS? 1.To what extent are the CF objectives aligned and been consistent with the needs, priorities, and policies of the government (including alignment to national development goals and targets, national plans, strategies and frameworks). 2.To what extent did the implementation of the CF (joint workplan and agencies programmes) adjust to emerging issues faced by the country during the implementation? Coherence: HOW WELL DOES THE CF FIT? 3.To what extent has the CF strengthened the position, credibility and reliability of the UN system as a partner for the government and other actors, and has served as an effective partnership vehicle? 4.To what extent has the CF strengthened the coherence of support by UNCT members and sought partnerships (with civil society/private sector/local government/parliament/national human rights institutions/international development partners) to enhance achievement of results? Effectiveness: HAS THE CF ACHIEVED ITS OBJECTIVES? IS THE CF DOING IT RIGHT? 5.How effective has the CF been in achieving the results outlined in the results framework? What have been the benefits for the people and institutions targeted by the interventions, including the most vulnerable, disadvantaged, and marginalized population? 6.To what extent has the CF contributed to the promotion of gender equality and women’s empowerment, human rights, disability inclusion and environmental sustainability. Efficiency: HOW WELL HAVE RESOURCES BEEN USED? 7.Has the CF reduced transaction costs for partners through greater UN coherence? 8.To what extent has the CF collectively prioritized activities based on the needs (demand side) rather than on the availability of resources (supply side), and reallocated resources according to the collective priorities and changing needs if/where necessary? 9.How adequate has the CF been in facilitating the effective reallocation of resources to emerging needs and priorities? Coordination: HOW WELL HAS IMPLEMENTATION OF THE CF BEEN COORDINATED? 10.Post UN reform, to what extent have UN agency programs and work plans been effectively and meaningfully derived from the CF both in design and implementation? 11.To what extent did the post reform Resident Coordinator office’s roles and responsibilities enable positive UNCT’s joint convening power and better coherence of the country team? Orientation towards impact: WHAT DIFFERENCE DO CF INTERVENTIONS MAKE? 12. To what extent have UN system activities articulated in the CF driven progress towards, or supported achievement of ToC outcomes? Sustainability:WILL THE BENEFITS LAST? 13. What mechanisms, if any, has the CF established to ensure socio-political, institutional, financial and environmental sustainability? 14. What is the likelihood that progress towards the SDGs is sustained by national partners and stakeholders over time? Evaluation Approach and Methodology The evaluation will methodologically be informed by the UNSDCF Evaluation Guidelines 2021 Engl Revised.PDF The evaluation will use a combination of qualitative and quantitative methods, including document reviews, analysis of quantitative secondary data, individual interviews with key informants and focus groups or other types of discussion to collect data. The Independent External Evaluators will be guided by the UNSDCF Evaluation Guidelines to develop the evaluation methodology. They will also develop the necessary tools to collect data and information to answer the overall evaluation questions. The methodological design will include: an analytical framework; a strategy for data collection and analysis; specially designed tools; an evaluation matrix; and a detailed work plan. Sampling approach: A purposive sampling approach will be used to select programmes (joint workplans; joint programmes; UN agencies strategic plans etc.) that will be covered in the scope of the CF evaluation. The selected programmes components should have sufficient level of transformational intent (depth, breadth, and size) and maturity. See UNSDCF Evaluation Guidelines 2021 Engl Revised.PDF Data collection: The evaluation will use quantitative and qualitative approaches, including literature review, statistics at national and local levels, survey data, semi-structured interviews, direct observation, focus groups and workshops. Quality assurance: The data collected should be subjected to a rigorous quality assurance for validation purposes, using a variety of tools including triangulation of information sources and permanent exchange with the CF implementation entities at Country Office level. Evaluation Matrix[2]: The evaluation team will use the template of the evaluation matrix provided by the evaluation manager to systematically structure and consolidate the data collected for each of the evaluation questions. This matrix will allow them, among other things, to identify the missing data and thus fill these gaps before the end of the collection. This matrix will also help to ensure the validity of the data collected. Participation and inclusion: This evaluation should be conducted using a participatory and inclusive approach[3], involving a wide range of partners and stakeholders. The evaluation team will carry out a stakeholder mapping in order to identify the direct and indirect partners of the CF, specifically targeting United Nations organizations and representatives of the national government. Stakeholders mapping may include civil society organizations[4], the private sector, other multilateral and bilateral cooperation organizations and, above all, the beneficiaries of the program. Theory of change analysis: During the inception phase or early in the data collection phase, the evaluation team will hold theory of change (ToC) meetings with the result or thematic groups to discuss the programmatic changes that could have occurred in the programme result chain during implementation as a response to emerging challenges and needs. These meetings are led by the Evaluation Team and organized by the Evaluation Manager. The Synoptic table for the Analysis of the theory of change is used to support the ToC meetings. The analysis of the CF’s theory of change and the reconstruction of its intervention logic, if necessary, will therefore play a central role in the design of the evaluation, in the analysis of the data collected throughout the evaluation, in communicating results and in developing relevant and practical conclusions and recommendations. Finalization of the evaluation questions and assumptions: The evaluation team will finalize the evaluation questions after consultations with the evaluation steering committee and thematic groups. The final evaluation questions should be a reasonable number, generally not exceeding 15. They should clearly reflect the evaluation criteria as well as the indicative evaluation questions listed in these Terms of Reference. They should also take advantage of the results of the reconstruction of the intervention logic of the cooperation framework. The evaluation questions will be included in the evaluation matrix (see appendix) and should be supplemented by sets of hypotheses that capture the key aspects of the intervention logic associated with the scope of the question. Data collection for each of the assumptions will be guided by clearly formulated quantitative and qualitative indicators, also indicated in the matrix. Evaluation Process and Timeline Inception Phase. This phase will include: ● Preliminary desk review of available sources. The documentation made available to the evaluation team will include but may not be limited to: UNSDCF planning documents, UN agency-specific planning documents linked to the UNSDCF; programme monitoring data covering the timeframe of the evaluation; documentation related to the coordination mechanisms for the UNSDCF; relevant studies, reports or similar documents related to topics addressed by the UNSDCF; policies, strategies and normative guidance that has informed the development of the UNSDCF; relevant national policies and strategy documents. ● Discussions with the Evaluation Team and the Evaluation Steering Committee to: a) understand the spirit of the evaluation questions and refine them; b) understand relevant contextual factors and finetune the methodology accordingly; c) understand the chronology of external and internal events during the UNSDCF period under evaluation and establish an events timeline; d) if necessary, reconstruct and validate the theory of change for the UNSDCF. ● Preparation of the inception report. The evaluation team will have to submit an inception report aligned with the UNEG Norms and Standards. The Inception Report will be subject to quality assurance performed by the evaluation manager, a review conducted by UN stakeholders, and an ethical review. The approval of the inception report marks the completion of the inception phase. It must include: - Methodology, including the methodological approaches showing the reconstructed ToC evaluation matrix, data collection tools and analysis strategy, stakeholders mapping and sampling - Analysis of risks related to ethical issues and identification of mitigating measures. The inception report must be in line with the UNEG guidelines on ethics in evaluation. The inception report will be submitted for ethical review before proceeding with the next phase - Work-plan detailing the work schedule - Outline of the final report. Data Collection Phase. Following the inception phase, a mission to Georgia by the evaluation team will take place to conduct the qualitative data collection and verify the information available in secondary sources. At the end of the mission, the evaluation team will present its preliminary observations and findings to the Evaluation Reference Group and to other UN staff as relevant. Reporting Phase. The evaluation team will prepare a draft evaluation report that will be subject to a quality assurance process based on UNEG Norms and Standards. The report should be no longer than 60 pages excluding annexes and should be in line with the outline presented in Annex 1. Facilitation of use and dissemination phase. The final report will be drafted by the evaluation team based on the comments received. The final report will be shared with the UNCT and relevant stakeholders. Management Response to the Evaluation. In line with the recently released UN Sustainable Development Cooperation Framework Guidance, following receipt of the final evaluation report, the UNCT will develop a management response to the evaluation recommendations. The key deliverables that the CF Evaluation Team is expected to produce are as below. The Team should utilize the templates provided by the CF Evaluation Guidelines. Deliverables 1. Inception Report, containing a preliminary analysis of the CF ToC or, in its absence, reconstruction of the CF ToC; an elaboration of the evaluation approach and methods, including the evaluation design matrix; and a detailed evaluation plan and timeline, including a tentative list of interviews to be arranged or plans for field visits. 2. Preliminary findings report and presentation, in a template to be agreed with the Evaluation Manager. 3. Draft and final CF Evaluation Report, including the annexes. 4. Power point presentation not exceeding 20 slides The Evaluation Steering Committee is engaged from the outset together with the Evaluation Manager to guide the whole evaluation process. The Evaluation Steering Committee ensures the technical and factual quality of the data, analysis and findings. The quality of the deliverables will be assessed in reference to the quality assurance checklist of the CF Evaluation Guidelines. The DCO will provide quality assurance and oversight of and throughout the entire evaluation process. DCO will quality assure the evaluation report at the end of the evaluation process to ensure the soundness, usefulness, and evidence-based elements of the final report. Work LocationGeorgia The evaluation will be conducted mainly in Tbilisi where the key stakeholders are located. However, during the inception phase, in consultation with the UNCT, the evaluation team will identify 2 to 3 regions with adequate programme representation for field visits. Expected durationApril - October 2024 20 Working Days Duties and ResponsibilitiesThe 2021-2025 UNSDCF’s Results Framework is structured around five key outcomes: Outcome 1: By 2025, all people in Georgia enjoy improved good governance, more open, resilient and accountable institutions, rule of law, equal access to justice, human rights, and increased representation and participation of women in decision-making. Outcome 2: By 2025, all people in Georgia have equitable and inclusive access to quality, resilient and gender-sensitive services delivered in accordance with international human rights standards. Outcome 3: By 2025, all people without discrimination benefit from a sustainable, inclusive and resilient economy in Georgia. Outcome 4: By 2025, conflict affected communities enjoy human rights, enhanced human security and resilience. Outcome 5: By 2025, all people, without discrimination, enjoy enhanced resilience through improved environmental governance, climate action and sustainable management and use of natural resources in Georgia. Considering the programmatic thematic covered by the CF (5 outcomes), the evaluation team (ET) will be made up of 3 independent external evaluators including an International Team Leader (TL). The TL shall demonstrate extensive evaluation expertise and at least sound expertise in at least one of the programmatic areas. The Evaluation Team Leader leads the entire evaluation process, working closely with team members. He/she will conduct the evaluation process in a timely manner and communicate with the Evaluation Manager on a regular basis and highlight progress made/challenges encountered. The Evaluation Team Leader will be responsible for producing the inception report, cover one thematic outcome and deliver the draft and final evaluation reports. The other 2 National team members will demonstrate multidisciplinary expertise to adequately cover the 4 other CF outcomes. Efforts will be made to ensure gender representation within the ET. The ET will actively engage in gaining an in-depth understanding of both the CF and the country context. They will assess the ToC for better alignment with the CF, select and adapt evaluation questions, and propose the most appropriate methods for data collection and analysis. In consultation with the UNCT and the Evaluation Manager, the team will select a sample of stakeholders for interviews, ensuring a well-described methodological approach to sampling. The Evaluation Team will collect and triangulate data, drafting the inception, draft, and final evaluation reports, including a summary of the Performance Rating. The team will revise deliverables as needed based on comments and lead stakeholder, briefing, and debriefing workshops as needed. ET is expected to make presentation of their findings to the UNCT/RCO, collect comments and incorporate them into the Final Evaluation Report. Deliverables Estimated working days for national Team Member Inception phase 7 days Initial desk review ToC meeting Secondary desk review Inception report Field phase data collection 5 days First draft 5 days Preliminary findings presentation 1 day Final report 1 day Stakeholder workshop 1 day Total Working Days 20 days Qualifications/special skillsEducation: • Advanced University Degree (Master or PhD) in the field of political science, governance, public administration, development studies, law, human rights or other relevant field is required; Working experience: • Minimum five years of relevant professional experience is required. • Strong data collection and analysis skills; Prior experience in working with multilateral agencies is required. • Strong experience and knowledge in the UN programming principles including leaving no one behind LNOB, human rights, gender equality and women's empowerment, disability inclusion, environmental sustainability and resilience, and accountability is required. • In-depth knowledge and strong research record of country socio-economic development and environment context is required. Competencies: • Excellent timely delivery and time management. • Process management skills such as facilitation skills and ability to negotiate with a wide range of stakeholders. • Technical competence in undertaking complex evaluations which involve use of mixed methods. • Knowledge of UN role, UN reform process and UN programming at the country level, particularly UNSDCF. • Experience in translation and interpretation. LanguagesFluency in English and Georgian languages are required. Additional InformationInterested candidates shall submit documentation demonstrating why they are the best-suited to carry out the above task. This should include: • Latest CV and sample of previous works. • A brief outline of the strengths and expertise highlighting directly relevant experiences to the assignment, including experience and knowledge in the UN programming principles (LNOB, human rights, GEWE, environmental sustainability and resilience, and accountability) • Financial Proposal No FeeTHE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.
This vacancy is archived.