Back National consultant for Final Evaluation of PTIB Project

Background

UNDP’s “Partnerships for a Tolerant, Inclusive Bangladesh” project is a multi-year initiative to understand and prevent violence and extremism in Bangladesh. The project responds to the UN Secretary General’s 2015 call for every country to develop a coherent and contextual strategy for the prevention of violent extremism (PVE), while reaffirming the need for inclusive institutions, transparent politics, and a commitment to fundamental human rights. The project also adapted over the past year to respond to the Secretary General’s Strategy and Plan of Action on Hate Speech (2019). The project builds local research capacity, and uses cutting-edge evidence to help promote authentic and resonant Bangladeshi narratives of diversity and tolerance.

This project is a major component of UNDP’s Democratic Governance Portfolio in Bangladesh, which supports Bangladesh’s achievement of key targets under Sustainable Development Goal 16 (SDG 16): “Peace, Justice and Strong Institutions”.  The PTIB Project is set to run until May 2021, and presently operates with substantial support from six governments (USA, UK, Denmark, Norway, Japan and the Netherlands), with a total budget of USD 5.1 M over 3 years.

The project addresses complex issues of identity and inclusion by promoting a digital literacy model. “Digital literacy” is defined by the PTIB project as a citizen’s ability to distinguish between trustworthy information and untrue or manipulative content online. This involves a critical engagement with Bangladesh’s online environment, helping citizens to identify and question any misleading or incendiary online content they may encounter. In the internet age, UNDP believes that digital literacy is just as important as being able to read and write - if Bangladeshis are able to engage more skeptically and safely in cyberspace, UNDP expects they will become more resistant to divisive, exclusionary and violent rhetoric. Specifically, a focus on promoting digital literacy and safe use of online platforms (like Facebook) can empower some of Bangladesh’s most vulnerable women to identify and speak out against incitement and violence that might disproportionately affect them. This model is fully evidence-based, and relies on superior monitoring and analysis of online trends, as well as an adaptable communications strategy and programme design. The project accomplishes this in three ways:

• By improving knowledge and insight on the drivers of violence in Bangladesh;
• By supporting Bangladeshi narratives of inclusivity and tolerance, online and offline, with a particular focus on youth, women, and vulnerable groups;
• By supporting and helping to sensitize Bangladesh government agencies to emerging standards of best practice and practical means of promoting social inclusion and tolerance.

 The project supports three data collection teams, which supply different kinds of regular information:

 1) The Bangladesh Peace Observatory (BPO): The BPO to catalogues, aggregates, maps and visualizes nationwide statistics on violent incidents using a combination of open data and media reports. Based at the University of Dhaka and supported by UNDP, the BPO produced targeted analyses and updates, identifying new avenues for research. A variety of qualitative and quantitative methods are used to help identify which dynamics push at-risk individuals from alienation to intolerance, and how this process might be peacefully interrupted.

2) Social Media Mapping: UNDP Bangladesh has commissioned specialist internet mapping companies to investigate and analyze extremist or incendiary narratives on popular online platforms like Facebook and Twitter. UNDP Bangladesh is using this information to identify and promote peaceful counter-narratives on key themes.

3) The Cox’s Bazar Analysis and Research Unit (CARU): UNDP’s in-house monitoring team, which tracks violent incidents in regions affected by the Rohingya emergency, and produces weekly updates for international and Bangladeshi partners. Since the COVID outbreak, CARU has also been producing additional weekly updates on the social and economic implications of COVID-19 spread in Bangladesh, including an overview of violent incidents, demonstrations, and noteworthy incidents

Social media campaigns under the hashtag #digitalpeacemovement have reached 17m Bangladeshis across the country, promoting inclusive perspectives and a spirit of national collaboration. UNDP Bangladesh’s highly successful hackathon series, the “Digital Khichuri Challenges”  have become an established brand in Bangladesh with a reach of over 12m people, while many of the platforms which emerge from these hackathons have developed an even greater reach. Numerous UNDP outgrowth events from the Digital Khichuri Challenges, including the “Peace Talk Cafe” event series are also popular and attract considerable public attention and interaction. In 2019, UNDP launched an online Digital Literacy Challenge – a quiz which would help Bangladeshis to question and identify untrustworthy sources of online information. The quiz was taken by 1.5m respondents.

UNDP’s engagement with the Dhaka Police’s Counter Terrorism and Transnational Crime (CTTC) Unit has helped to integrate core PVE and human rights concepts in the policing discussion in Bangladesh. Joint research between the police and the Bangladesh Peace Observatory has given academics access to key data for the first time, producing five joint research publications on the sources and expressions of extremism. Findings have been built into national police policies, including the draft National Counter-Terrorism Strategy (developed by the CTTC with USAID and UNDP support), which emphasizes a preventive approach built on community participation.

Evaluation Purpose:

Purpose:

The purpose of the final evaluation is to assess achievements to date, document lessons learned, and provide recommendations to UNDP and its partners to develop next phase of PTIB in Bangladesh.

Timing:

The final project evaluation is proposed to be conducted in August 2020 as the PTIB is scheduled to end on 31 May 2021.  The timing has been agreed with the project advisory board. The timing is also meant to ensure that the evaluation results will support UNDP and its partners in developing a next phase of PTIB.

Utilization:

The primary users of the evaluation results will be UNDP, but the evaluation results will equally be useful to relevant GoB ministries, development partners and donors so on.

In addition, the evaluation aims at critically reviewing and identifying what has worked well in the project, what challenges have been faced, what lessons can be learned to improve future PTIB  programming. The evaluation will also generate knowledge for wider uses, assess the scope for scaling up the current programme, and serve as a quality assurance tool for both upward and downward accountability. UNDP will take in consideration all useful findings, conclusions and recommendations from the evaluation, prepare a systematic management response for each recommendation, and implement follow-up actions as per UNDP Evaluation Resource Center guidance/policies.

In the view of the above, UNDP is seeking for one national consultant to support the final evaluation of PTIB. The evaluation will work under the overall supervision of the Deputy Resident Representative, UNDP Bangladesh and day to day collaboration with the Project Manager PTIB. The national consultant will report to the international consultant.

 Objectives of the assignment:

The final evaluation will focus on measuring development results generated by the PTIB, based on the scope and criteria included in this term of reference. The unit of analysis or object of study for this evaluation is the PTIB, understood to be the set of components, outcomes, outputs, activities and inputs which are described in the PTIB Project document and M&E Framework.

 The objectives of this evaluation are to:

• Assess to what extent PTIB has contributed to address the needs and problems identified during programme design;
• Assess how effectively PTIB has achieved its stated development objective and purpose;
• Measure how efficiently the PTIB outcomes and outputs have progressed in attaining the development objective and purpose of the project;
• Assess both negative and positive factors that have facilitated or hampered progress in achieving the project outcomes, including external factors/environment, weakness in design, management and resource allocation;
• Assess the extent to which the application of the rights-based approach and gender mainstreaming are integrated within planning and implementation of the PTIB project
• Identify and document substantive lessons learned, good practices and also opportunities for scaling up the future PTIB project in Bangladesh;
• Provide forward looking programmatic recommendations for the PTIB project;

The evaluation will focus on five key evaluation criteria: relevance, efficiency, effectiveness, potential impact, and sustainability. The evaluation should provide credible, useful, evidence-based information which enables timely incorporation of its findings, recommendations and lessons into decision making processes of UNDP and key stakeholders as well as assess the potential of the next phase of the project. The evaluation will cover the time span from 15th May 2017 (the beginning of the PTIB) to date. In order to meet the above objectives of the evaluation, the International Team Leader will work together with a national consultant throughout the assignment.

DETAILS OF THE TOR IS AVAILABLE AT FOLLOWING LOCATION

LINK to the TOR: https://drive.google.com/file/d/1byq4Vi9xz9IgOXzI4HrQmU39xwhW8NCV/view?usp=sharing

Duties and Responsibilities

Scope of Work and Timeline: 

The scope of work for the national consultant of this evaluation will include but not be limited to:

• Take responsibility for relevant areas in the development and finalization of the inception report that will include elaboration of how each evaluation question will be answered along with proposed methods, proposed sources of data, and data collection and analysis procedures;
• Designing of tools and data collection;
• Data collection, analysis and interpretation;
• Take responsibility for relevant areas in the draft evaluation report;
• Provide inputs in the final evaluation report;
• Utilize best practice evaluation methodologies;

Responsibilities with timeline (proposed) furnished below:

Phase

Scope of work of consultant

Number of Days

Timing

Inception Phase

This phase is meant to ensure that the evaluation team is fully prepared before undertaking data collection. It includes:

• Desk review of existing documents, including project document, TAPP, strategies developed by the project, reports and documents developed by the project and write-ups on the project initiatives
• Consultation with key external stakeholders
• Drafting of the inception report, including evaluation methodology, timeline, evaluation matrix, and data collection tools
• Development of data collection tools (i.e. KII checklists and short questionnaires)

05 Days

Within 1 week of signing contract

Data Collection Phase

(to be conducted virtually)

• A briefing session by UNDP and the project management
• Initial introductory meeting/workshop with the stakeholders and partners
• Key informant interviews with the stakeholders
• Debriefing to the UNDP CO and the stakeholders on the key findings

07 Days

Within 3 weeks of signing the contract

Reporting Phase

• Aggregation of findings from desk review and stakeholders interview
• Drafting of the evaluation report and evaluation brief
• Review by UNDP and stakeholders for quality assurance
• Incorporation of comments and revision of the report
• Submission of the final report

08 Days

Within 4 weeks of signing the contract

Competencies

Evaluation Questions:         

The evaluation questions define the information that must be generated as a result of the evaluation process. The answers will provide key basis to the intended users of the evaluation in making informed decisions, taking actions or adding knowledge.  Some of the tentative questions can be as follows:

Relevance: The extent to which the objective, purpose and outcomes of the intervention are consistent with the needs and interest of the people and the needs of the country.

 Effectiveness: Extent to which the outcomes of the development intervention have been achieved

• To what extent has the project achieved the objectives and targets of the results framework in the Project Document?
• Compared to 2017, to what extent do key stakeholders now better  prevent violent extremism and counter incitement of hate and violence in Bangladesh? To what extent are any changes linked to PTIB interventions?
• What factors contributed to the achievement or non-achievement of the PTIB outcomes and outputs?
• To what extent and in what ways has ownership - or the lack of it - by the implementing partner impacted on the effectiveness of the PTIB?

 Efficiency: Extent to which resources/inputs (funds, time, human resources, etc.) have been turned into results.

• To what extent were the PTIB outputs delivered in time to ensure high quality?
• To what extent has PTIB ensured value for money?
• To what extent was resource mobilization efforts successful? Was funding sufficient for achievement of results? (funding analysis)
• To what extent and in what ways has ownership - or the lack of it - by the implementing partner impacted on the efficiency of the PTIB?
• To what extent was there any identified synergy between UNDP initiatives/projects that contributed to reducing costs while supporting results?
• How well did project management work for achievement of results?
• To what extent did project M&E systems provide management with a stream of data that allowed it to learn and adjust implementation accordingly?

 Sustainability: Probability of the benefits of the intervention continuing in the long term

• To what extent will the PTIB achievements be sustained? What are the indicators of sustainability for these achievements, e.g., through requisite capacities (systems, structures, staff, etc.)?  What are the challenges and opportunities?
• To what extent are policy and regulatory frameworks in place that will support the continuation of PTIB?
• To what extent are the institutional mechanisms in place to sustain impacts of PTIB’s
• To what extent have development partners committed to providing continuing support?

 Evaluation of Cross-Cutting Issues:

Leave no one behind and gender aspects will be considered well in evaluation questions as well the evaluation process. Gender analysis, including gender disaggregated data need to be incorporated in the evaluation.

 Leave no one behind:

• To what extent have the research and monitoring of PTIB been inclusive in terms of capturing the situation of the most vulnerable and marginalized part of the Bangladesh population, vulnerable for incitement of hate and violence.
• To what extent has PTIB civil society and youth engagement been able to include and reach the most vulnerable and marginalized part of the Bangladesh population, vulnerable for incitement of hate and violence.

Gender Equality:

• To what extent has PTIB and other national stakeholders’ capacity been strengthened in better promoting and protecting women’s rights.
• To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?
• Is there gender marker data assigned to this project representative of reality?
• To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

Way forward

• Have any good practices, success stories, lessons learned, or transferable examples been identified? Please describe and document them.
• Based on the achievements to the date, provide forward looking programmatic recommendations for UNDP PTIB next phase.

Methodology    

It is strongly suggested that the evaluation should use a mixed method approach – collecting and analyzing both qualitative and quantitative data using multiple sources in order to draw valid and evidence-based findings and conclusions and practical recommendations. The evaluation team is expected not only to conduct specific surveys to collect quantitative/qualitative data but also is highly encouraged to review all relevant reports providing quantitative data collected by PTIB. However, the evaluation team is expected to propose and determine a sound evaluation design and methodology (including detailed methodology to answer each evaluation question) and submit it to UNDP in the inception report following a review of all key relevant documents and meeting with UNDP and PTIB. Final decisions about the specific design and methods for the evaluation will be made through consultation among the  PTIB, UNDP,  the consultants and key stakeholders about what is appropriate and feasible to meet the evaluation purpose and objectives as well as answer the evaluation questions, given limitations of budget, time and data. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with stakeholders. All stakeholder meetings will be organized virtually for primary data collection given the current COVID-19 pandemic in Bangladesh.

Methods to be used by the evaluation team to collect and analyze the required data shall include but not limited to:

• Desk Review: This should include a review of inter alia

• Project document
• Result Framework/M&E Framework
• Project Quality Assurance Report
• Annual Work Plans
• Annual Reports
• Highlights of Project Board meetings
• PTIB social media engagement review (to be completed by 30th July 2020)
• Studies relating to the country context and situation

• Semi-structured interviews with key stakeholders including UNDP, PTIB, Government partners, UN colleagues, development partners, CSOs, youths, so on:

• Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed

• Key informant interviews with relevant stakeholders from government agencies, donors, UN Agencies, youth groups and CSOs supported by PTIB.
• All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments of individuals

• Analysis of PTIB’s funding, budgets and expenditure generated from Atlas.
• Analysis and interpretation of qualitative and quantitative data available from various credible sources.
• Data review and analysis of monitoring and other data sources and methods: ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources

Data and evidence will be triangulated with multiple sources to address evaluation questions. The final methodological approach including interview schedule and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between UNDP, stakeholders and the consultants.

Gender and Human Rights-based Approach

As part of the requirement, evaluation must include an assessment of the extent to which the design, implementation, and results of the project have incorporated gender equality perspective and rights-based approach. The evaluators are requested to review UNEG’s Guidance in Integrating Human Rights and Gender Equality in Evaluation during the inception phase[1].

In addition, the methodology used in the final evaluation, including data collection and analysis methods should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. Detailed analysis on disaggregated data will be undertaken as part of final evaluation from which findings are consolidated to make recommendations and identify lessons learned for enhanced gender-responsive and rights-based approach of the project.

These evaluation approach and methodology should consider different types of groups in the PTIB project intervention – women, youth, minorities, and vulnerable groups.

Expected Deliverables

The International Team Leader will be responsible for ensuring the following outputs/deliverables to UNDP Bangladesh as per the agreed work plan:

Inception Report:

The consultant(s) will commence the evaluation process with a desk review and preliminary analysis of the available information provided by UNDP. Based on the ToR, initial meetings with the UNDP and the desk review, the consultants should develop an inception report which will be around 5 pages in length and will elaborate evaluation methodologies, including how each evaluation question will be answered along with proposed methods, proposed sources of data, and data collection and analysis procedures. The inception report will include the evaluation matrix. UNDP and PTIB will review the inception report and provide useful comments for improvement. This report will serve as an initial point of agreement and understanding between the evaluation team and UNDP/PTIB;

Draft Evaluation Report:

The evaluation report will contain the same sections as the final report and shall follow the structure outlined in Annex 3/ Evaluation Report Template and Quality Standards (Page 49-53) of Section 4/ Evaluation Implementation of UNDP Evaluation Guideline (2019) [2]. The draft report will be reviewed by the PTIB and UNDP. The draft report will ensure that each evaluation question is answered with in-depth analysis of information and back up the arguments with credible quantitative and/or qualitative evidences.

Presentation/Debriefing/Audit Trial:

A meeting will be organized with key stakeholders including UNDP and PTIB to present findings, conclusions and recommendations.

Final Evaluation Report:

The final report will incorporate comments and feedbacks from the stakeholders including the feedback provided during the Presentation/Debriefing meeting. Other relevant documents (i.e. data collection tools, questionnaires, datasets, if any) need to be submitted as well.

Evaluation Brief:

A concise summary of the evaluation report will include findings, conclusions and recommendations using plain language targeting wider audience. This concise summary will be not more than 4 pages.

Price Proposal and Schedule of Payments

Consultants must send a financial proposal based on Lump Sum Amount. The total amount quoted shall be all-inclusive and include all costs components required to perform the deliverables identified in the TOR, including professional fee, travel costs, living allowance (if any work is to be done outside the IC´s duty station) and any other applicable cost to be incurred by the IC in completing the assignment. The contract price will be fixed output-based price regardless of extension of the herein specified duration. Payments will be done upon completion of the deliverables/outputs and as per below percentages:

 The expected outputs, deliverables and payment schedule is as follows:

Deliverables/ Outputs

Estimated duration

Target Due Dates

Payment Schedule

Review and Approvals Required

Submission of Inception Report, including a methodology note and evaluation matrix (based on meetings with the UNDP, the desk review and preliminary analysis of the available information provided by UNDP)

5 days

5th August 2020

25%

PTIB/ Deputy Resident Representative, UNDP Bangladesh/ Head of DG Cluster, UNDP Bangladesh/ M&E focal point, UNDP Bangladesh

Submission of Final Evaluation Report (including Evaluation Brief, Data Collection Tools, Questionnaires, Datasets (if any))

15 days

31st August 2020

75%

Total days consultant wise            

20 days

 

 

 

Travel:

No travel will be required for this assignment. It will be homebased, all meetings and interviews will be virtual online. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and Individual Consultant, prior to travel and will be reimbursed.

 [1] http://www.uneval.org/papersandpubs/documentdetail.jsp?doc_id=980

[2] Evaluation Report Template and Quality Standards of UNDP Evaluation Guideline (2019),

Section 4 : Evaluation Implementation, available at http://web.undp.org/evaluation/guideline/

Required Skills and Experience

Implementation Arrangement, Supervision and Performance Evaluation:

This evaluation is commissioned by UNDP Bangladesh. The Deputy Resident Representative, UNDP Bangladesh will be responsible for managing the evaluation throughout the entire process and PTIB will provide necessary support in day to day operation of evaluation. The International Team Leader will work under the overall supervision of the Deputy Resident Representative, UNDP Bangladesh. The International Team Leader will lead the evaluation mission together with a National Consultant. The consultant will report to and work under supervision of the international consultant.

Evaluation Team Composition and Required Competences

The evaluation team will be comprised of one team leader (an national consultant) and a national consultant. The presence of an national consultant is deemed desirable given the complexity and sensitivity of some of the issues concerned, and therefore to safeguard the independence and impartiality of the evaluation.

Qualifications:

The qualifications below are for the National consultant

• A masters’ degree or equivalent peace & development, international relations, social sciences, political economy or other relevant fields.
• At least 10 years of working experience in the field of democratic governance, experience in peace building and/or PVE preferred.
• Experience in conducting evaluations or assessment of policies and programs in peace building and social cohesion.
• Possess strong analytical and writing skills, with the ability to conceptualize, articulate, write and debate about governance issues.
• Experience in implementing a range of qualitative and quantitative data collection tools and methods in project evaluation.
• Knowledge of current issues and innovation in results-oriented monitoring, including trends, principles and methodology.
• Good knowledge of UN and/or UNDP’s mandate and socio-political context in the region.
• Advanced level of proficiency in both written and spoken English.

  Corporate Competencies:

• Demonstrates integrity by modeling the UN’s values and ethical standards (human rights, tolerance, integrity, respect, and impartiality);
• Promotes the vision, mission, and strategic goals of UNDP;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

 Functional Competencies:

• Consistently approaches work with energy and a positive, constructive attitude ;
• Strong interpersonal and written and oral communication skills;
• Strong analytical skills and strong ability to communicate and summarize this analysis in writing
• Has ability to work both independently and in a team, and ability to deliver high quality work on tight timelines.

 Skills:

• Strong leadership and planning skills
• Excellent written and presentation skills (English)
• Strong communication skills
• Ability to work in the multi-cultural team environment and to deliver under pressure/meet deadlines
• Ability to network with partners on various levels
• The necessary computer skills with competence in MS office package

 Evaluation Ethics

______________________________________________________________________________________________________

The evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. [1] The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

Evaluation of the proposals:

Evaluation Method and Criteria

Individual consultants will be evaluated based on the following methodology:

 Cumulative analysis

The award of the contract shall be made to the individual consultant whose offer has been evaluated and determined as a) responsive/compliant/acceptable; and b) having received the highest score out of set of weighted technical criteria (70%). and financial criteria (30%). Financial score shall be computed as a ratio of the proposal being evaluated and the lowest priced proposal received by UNDP for the assignment.

Technical Criteria for Evaluation for National consultant (Maximum 70 points)

 

Criteria

Weight

Max. Point

Technical

70%

70

A masters’ degree or equivalent peace & development, international relations, social sciences, political economy or other relevant fields, or Bachelors but with 5 years of additional experience than below.

5%

5

At least 10 years of working experience in the field of democratic governance, preferably Peace Building and/or PVE

25%

25

Experience in conducting evaluations or assessment of policies and programs in peace building and social cohesion

25%

25

Knowledge of current issues and innovation in results-oriented monitoring, including trends, principles and methodology.

5%

5

Good knowledge of UN and/or UNDP’s mandate and socio-political context and human rights situation in the region.

10%

10

Financial

30%

30

Total

100%

100 points

Only candidates obtaining a minimum of 49 points (70% of the total technical points) would be considered for the Financial Evaluation.

Financial Evaluation (Total 30 marks)

All technical qualified proposals will be scored out 30 based on the formula provided below. The maximum points (30) will be assigned to the lowest financial proposal. All other proposals received points according to the following formula:

p = y (µ/z)

Where:

• p = points for the financial proposal being evaluated;
• y = maximum number of points for the financial proposal;
• µ = price of the lowest priced proposal;
• z = price of the proposal being evaluated.

 [1] UNEG, ‘Ethical Guidelines for Evaluation’, June 2008. Available at http://www.uneval.org/search/index.jsp?q=ethical+guidelines.

Document required:

Interested individual consultant should submit the following documents/information:

• A technical proposal (i) explaining their understanding of the work and (ii) provide a brief methodology on how they will conduct the work (500 words max);
• A financial proposal (including fees, international travel costs if applicable and per diems). Template can be downloaded from the link below: http://www.bd.undp.org/content/dam/bangladesh/docs/Jobs/Interest%20and%20Submission%20of%20Financial%20Proposal-Template%20for%20Confirmation.docx
• Personal CV or P11, indicating all past experience from similar projects/ fields, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references; P11 can be downloaded from the link below: http://www.bd.undp.org/content/bangladesh/en/home/operations/jobs/

Please combine all your documents into one (1) single PDF document as the system only allows to upload maximum one document.