July 21, 2020

Terms of Reference For Final Evaluation of the Regional Project: “Men and Boys as Partners in Promoting Gender Equality and the Prevention of Youth Extremism and Violence in the Balkans – Young Men Initiative – YMI II”

1st October 2017 – 31st December 2020
Contract no: 8305-00/2017
Individual experts, companies or evaluators teams can apply 
Deadline for application: August 16th, 2020, 4pm CET.

About CARE in the Balkans

CARE’s mission in the region is to contribute to the socio-economic development of the Western Balkans by working in Bosnia & Herzegovina, Serbia, Kosovo, Macedonia, Montenegro and Albania. CARE in the Balkans as a regional office consists of the Headquarters in Sarajevo, Bosnia and Herzegovina (B&H), and Program Offices in Belgrade, Serbia, Pristina, Kosovo and Skopje, Macedonia

CARE’s work in the Balkans started in 1992, when it provided humanitarian support to people affected by war. In the late 90ies, CARE shifted its focus in the region from humanitarian post-war assistance and rehabilitation to socio-economic development engaging in interventions directed at conflict prevention and peacebuilding, sustainable livelihoods, gender equality and the prevention of gender-based violence. 

CARE in the Balkans places the quality of its program at the center of its mission, which is reflected in its regional strategy through two main program directions: Gender Equality and Social and Economic Inclusion. The goal of the Social and Economic Inclusion Program is to strengthen capacities and create opportunities for marginalized, socially excluded and poor to integrate into society and access rights. CARE’s Gender Equality Program aims to empower women and girls vulnerable to violence, discrimination and poverty, to reach better life opportunities and social justice by equally engaging men and boys as allies in that process. CARE’s engagement and contribution lies in strengthening sustainability of key regional, national and/or local civil society organizations and networks promoting gender equality and diversity, in the context of social inclusion and non-violence.

Background 

The Men and Boys as Partners in Promoting Gender Equality and prevention of Youth Extremism and Violence in the Balkans or Young Men Initiative II (YMI II) project builds upon CARE´s comprehensive and programmatic effort to fight interpersonal and gender based violence (GBV) as well as to improve gender equality in the region and address preventative issues related to youth extremism and violence. 

The YMI II project has a budget of 1 million euros, started on 1 October 2017 and it will last until 31st December 2020. The project is being implemented in Albania, Bosnia and Herzegovina, Kosovo*and Serbia, and targets youth, especially young men vulnerable to violence and anti-social behaviors. It strengthens key skills, knowledge, attitudes that support improved behaviors around gender equitable norms and non-violence. In total, at least 56 800 people would be directly reached through the project: 150 teachers, 5 000 students, 150 university students, 50 200 youth, 1 100 parents/citizens, 100 community members and 100 movement leaders. (Indirectly more than 68 000 people).

The expected project’s impact is to enable positive and peaceful societies for young people in Serbia, Kosovo*, Albania and Bosnia and Herzegovina, that support gender equality and decrease interpersonal violence and its extremism. The project’s outcome is to improve Youth NGOs and government capacities to integrate Program Youth – Life Skills Educational Curriculum in the current educational and youth policy strategies and strengthen governmental and civil society efforts related to the inclusion of youth vulnerable to extremism. 

The YMI project started in 2006 with a qualitative research on violence and anti-social behaviors in young men, and over the years has grown into a comprehensive program targeting a wide variety of stakeholders: young people, parents, teachers, educational workers, university students, journalists, etc. Since 2013, YMI has received financial support from the Austrian Development Cooperation for two implementation phases: YMI I (2013-2016) and YMI II (2017-2020), being the latter the object of this evaluation.

YMI II focuses on scaling up and mainstreaming the Gender Transformative Life Skills program for youth – in short Program Y – via targeted advocacy through a close cooperation with relevant stakeholders and by building coalitions and engage parents and citizens in supporting gender equality focused life skills education program. In addition, a new comprehensive model, Program Y plus is being piloted, by working with a selected group of youth vulnerable to violence and extremism. Supported by the project, the partners have launched campaigns targeting youth and selected communities, creating awareness about hate speech and promoting the positive role of all community members in addressing extremism and violence. The project mainly builds on the use of social media, but also involves popular media and arts to reaching and engage vulnerable groups. The project has continued to support the efforts of five CARE’s local partner organisations to become national resource centers regarding gender equality, youth and violence prevention. 

The project proposes the following “theory of change”: IF students learn to recognize harmful gender norms and are provided safe spaces to practice questioning these constructs, THEN there is a greater likelihood of internalizing new ideas in the support of gender-equitable, healthy and non-violent behaviors. But this process of change cannot happen only at the individual level. It has to be supported by certain broader influences and structures, such as positive peer groups and role models, and the existing policy environment.: 

In order to accomplish its main outcometo improve Youth NGOs and government capacities to integrate Program Y (Youth – Life Skills Educational Curriculum) in current educational and youth policy strategies and strengthen governmental and civil society efforts related to inclusion of youth vulnerable to extremism – the YMI II project targets the following specific outputs:

Output 1: Life Skills program (Program Y) scaled up in accredited countries with the Ministries of Education and supported in its promotion and capacity building for use in secondary schools.

Output 2: In selected communities, a Program Youth “Plus model” is developed, piloted and tested to address issues of vulnerability to violence and extremism amongst selected youth.

Output 3: In targeted countries, social movements activated and/or supported to engage/encourage citizens; including parents, youth and a diverse civil society, with the focus on encouraging governments to actively promote/address gender equality, life skills and health education in schools. 

Output 4: Social norms campaign developed for targeted audience promoting peaceful masculinities, gender equality and addressing hate speech and intolerance.

The project has already conducted a baseline and an end line assessment targeting Output 1. The key findings from these assessments are expected to be taken into consideration in the project final evaluation. The Output 1 related end line assessment report is based on a quantitative research carried out in May 2020 using self-administered questionnaires in 6 high schools, with 1 201 respondents, and evaluates the change in knowledge, attitudes and behavior related to gender-based violence, gender equality, and healthy lifestyles amongst high school students.

Additionally, three case studies are planned to be carried out during May-August period: i) a case study related to the Program Y Plus model for working with prevention on youth extremism and radicalization (Output 2), ii) a case study on Program Y scaling up (Output 1), and iii) a case study on project adaptation and response to COVID-19 pandemic. The findings of these studies should be taken into consideration in the final evaluation report. 

Project outcome and outputs, as well as project indicators can be found in the project document and the project logframe. Additionally, all relevant information regarding the YMI can be found at youngmeninitiative.net website, program-y.org website and its resources page.

Purpose of the Final Evaluation

The main purpose of the final evaluation is to determine the appropriateness of the project design and the proposed Theory of Change, to assess the sustainability of the processes initiated through the intervention and to provide evidences in order to gain more knowledge and learn for future interventions in this area.

Objectives of the Final Evaluation

The focus of the final evaluation would be on outcome and impact level. The final evaluation will assess the relevance, sustainability and (the road to) impact based on OECD-DAC evaluation criteria and will provide findings, conclusions and recommendations. 

The evaluator(s) has to analyze the following:

  1. the design and coherence of the project including the design of the log frame matrix theory and present the underlying Theory of Change and its assumptions.
  2. the extent to which the project has already achieved its objectives and results, focusing on impact on target groups/partners, beneficiaries and other affected parties. 
  3. the extent to which cross-cutting issues such gender equality and nondiscrimination, participation and inclusion, transparency and accountability and empowerment of marginalized groups were applied.

The results of final evaluation will serve to inform CARE, the donor and the involved stakeholders about the findings and recommendations drawn from the implementation of the YMI project and to enable learning, improved design, planning and implementation of future CARE projects in gender transformative programming. The findings and recommendations will be particularly relevant to inform a possible continuation of the project with financial support from the  Austrian Development Cooperation.

Subject and focus (scope)

The final evaluation will focus on extracting lessons learned and identifying main challenges which will help with determining how future projects in this area could be made more relevant, impactful and sustainable. The results chain (log frame, theory of change) and project design are key aspects of this evaluation. 

The evaluation shall address 36 months of project implementation (from 1st October 2017- until 30th September 2020), covering four target countries (B&H, Serbia, Kosovo* and Albania) and main project beneficiaries (partner organisations, teachers, youth, movement leaders).

In order to assess the project design and log frame, evaluators are expected to take into consideration existing evidence as the end line assessment and case study on Output 1, case study on Output 2 and case study on COVID-19 project response; as well as the rest of the monitoring documents which will be provided by CARE. Existing evidence will cover findings on behavioral and change of attitudes among high school students, analysis of work with youth at risk and social institutions, analysis of scaling up process with ministries and schools and covid-19 adaptations in all target countries. It is expected that evaluator(s) take these findings into consideration and conduct further analysis to complete the tasks requested in this call.

Evaluator(s) must cover particular OECD-DAC evaluation criteria and provide findings, conclusions and recommendations. Priority will be given to relevance, impact and sustainability of the intervention in order to get recommendations and findings for the future continuation of the project. Furthermore, it is expected to include efficiency and effectiveness only on outcome level. The rest of the criteria are not expected to be prioritized having in mind given time frame, budget and purpose of the evaluation.

Beside the analysis of the level of success of defined OECD/DAC criteria (see main evaluation questions section) the evaluator(s) is expected to assess the role of the Be a Man Clubs, regional cooperation and peacebuilding as well as project adaptation and response to COVID-19 pandemic.

The final evaluation process is going to be conducted during the last phase of the project implementation, in the period from August 24th – October 31st(the detailed evaluation schedule is described in the Timetable section below). 

Main evaluation questions 

1. Relevance

  • To what extent the project responded to international standards and anti-discrimination priorities 
  • To what extent has the project responded to the needs of the local communities and the relevant sectoral policies of the target countries and proven to be feasible
  • Have the activities implemented been relevant for achieving the outcomes defined by the project?

2. Impact

  • Has the long-term impact as defined by the project document been achieved (or it’s on the path to be achieved)? To what extent have all stakeholders/target groups as defined gained new knowledge and changed their behavior? 
  • Are there any positive/negative unintended results of the intervention?
  • To which extent other mediating factors (e.g., social support networks, family social-economic status, etc.) have also influenced the attitudes and behaviors of young men and women?
  • To which extent campaigns and/or programs implemented by the project have increased awareness of key target groups about promoting peaceful masculinities, gender equality and addressing hate speech and intolerance?
  • To what extent the project adapted to the COVID 19 pandemic with the specific focus on gender equality impact?

3. Sustainability

  • What actions/results can be maintained after this intervention is over?
  • To what extent can the Program Y accreditation be integrated in Program Y Plus ensuring sustainability?
  • To what extent 6 project partners have improved organizational capacities for self-sustainable functioning as well as lobbying and advocacy (beyond project duration)?
  • To what extent are the key actors (partners and other stakeholders) prepared to take over the responsibility and carry on with the implementation of the developed strategies and commitments (ownership and leadership) taking into account institutional level?

Evaluation/review questions can be modified and others can be included.

Evaluation approach and methods

The evaluation will be conducted from combination of desktop research, focus group discussions, and group and individual interviews as finally agreed in inception report phase. Depending on potential restrictions (relation to COVID-19), the evaluator(s) will be invited to observe field visits and events organized by CARE and partners in accordance with technical proposal and agreed plan.

The Evaluator(s) will use the following methods:

  1. Desk review of available project documentation: Project proposal set (narrative, log-frame and supporting docs), Baseline and Endline Assessment Report, YMI Case Studies, selected Most Significant Change stories, YMI project web sites with documentation and reports (www.youngmeninitiative.net, www.program-y.org, www.pazisex.net and YMI Facebook, Instagram and YouTube pages), CARE’s Annual Narrative Reports, partners’ grants progress reports and monitoring tables, ADA (Austrian Development Agency) Country Strategy, CARE Gender Policy Guidelines. 
  2. Focus group discussions with 6 partner organizations management teams (min. 1), with implementing teams in each country (min. 4), and with beneficiary groups in each country, including youth from secondary schools, teachers, university students and movement leaders (min. 4).
  3. Individual or group interviews with representatives of public bodies (ministries of education, social welfare and youth, universities, centres for social work), schools´ staff (pedagogues and teachers), young men and women, university students, parents, community members, etc. based on inception report and agreed plan. (minimum 2 per country, 8 in total).
  4. Field research in one locality in each project country related to the implementation of the partners’ organizations grants in the framework of all outputs. If feasible, participatory observations in the target localities (school workshops, Be a Man Club activities, university students’ workshops, street campaign performances, etc.) Details will be provided by CARE Project Manager. (minimum 1 city per country, 4 in total).

All data collected need to disaggregated by sex.

The Guidelines for Project and Programme Evaluations developed by the Austrian Development Agency need to be considered throughout the entire evaluation process. 

The Evaluator(s) will produce the following deliverables (details can be found in Deliverables section of the ToR)

  1. An inception report in English (max 20 pages) which will include the detailed evaluation approach and methodology, detailed work plan and supporting documentation to outline the process of evaluation.
  2. A high-quality evaluation report in English using DAC Evaluation Criteria and Standards as framework with a structure mirroring the criteria as well as the achievements of the project purpose and the expected results, containing an introductory executive summary and a concluding lessons learned/recommendation chapter.
  3. High-quality evaluation report summary in English. The summary should be based on introductory executive summary of 5 pages max, and a concluding lessons learned/recommendation chapter.  

Indicative workload:

Desk Research3 days 
Inception Report4 days  
Field Research 1 – visiting partners/countries and if necessary, follow up online interviewsA maximum of 8 days (no more than 2 days per country) including focus groups, in depth interviews and individual and group consultations)
Field research 2 – individual interviews during partners meeting (TBC)3 days (individual interviews and focus groups with key staff from 6 partners organizations and with CARE staff) during partners meeting in Serbia or B&H in September 2020
Drafting Evaluation Report8 days (data analysis and report writing)
Incorporating Comments and Final evaluation report submission2 days 
TOTALUp to 28 days 

Timetable

TimetableOutputComments
21st July– 16th August 2020Call for proposalsCARE will announce open call for Final Evaluator (s) 
24th  August-14th September 2020 (up to 7 days)Desk research and Inception Report The Evaluator will draft the Inception Report based on an initial desk research. The Inception Report will be presented through online call and sent by e-mail to Project Manager. Only once the draft Inception Report has been approved by CARE, can data collection in the field start
14th September– 10th October 2020 (up to 8 days)
Field research 1-visiting partners/countries and if necessary, follow up skype interviewsField Research will be conducted during the ongoing CARE partners’ national and regional activities, as agreed with CARE and stakeholders (participatory observation and main target population interviewing – individually, in depth and/or focus groups interviews). Field research will be conducted in four countries and at least in 1 locality per country (if possible), where actions related expected results have been implemented. 
Second half of September 2020 (TBC) (up to 3 days)
Field research 2 – individual interviews and focus groups during partners meetingCARE will organize partners meeting in Serbia or B&H. 1 and ½ of days of meeting will be dedicated to the final evaluation. Partners meeting will be organized at the beginning of September 2020 (depending on COVID-19 restrictions). The Evaluator(s) will conduct individual (face-to-face) interviews and focus groups with executive directors and the YMI project coordinators of 4 partner organizations (Banja Luka, Belgrade, Pristina and Tirana) as well as CARE staff. Also, the Evaluator(s) will conduct two additional interviews with directors of NGO Status M, Zagreb and NGO Synergy, Kosovska Mitrovica who are partners in the regional component of the YMI project supported by Oak Foundation (all travel and lodging costs will be covered by CARE). 
10th – 23rd  October (up to 8 days)Draft Final Evaluation ReportThe Evaluator will deliver the draft Report including Evaluation Summary Report to CARE by 23rd of October 2020. 
October 27th -31st 2020 (up to 2 days)Incorporating Comments and Final evaluation report writingCARE and ADA will review and provide its comments to the Evaluator by 27th October 2020.
October 31st, 2020FINAL Evaluation ReportThe final report will be delivered to CARE in electronic format, in English, by October 31st 2020 04 pm CET as latest.
The Evaluator(s) will be engaged in the period 24th August – 31st October 2020 for a maximum of up to 28 working days. The final products will be delivered to CARE by October 31st 2020.

Note:

In case of field visits, the Evaluator(s) will be able to conduct interviews with partners’ representatives, identified stakeholders and CARE’s project staff during regular project activities with logistical support from CARE Project Manager. Travel and food/refreshment costs will be covered via the contracted consultancy fee.  Only the travel and lodging costs related to the partners meeting to be conducted in September 2020 (Field Research 2) will be covered by CARE. Due to the COVID-19 pandemic, CARE recommends that the Evaluator(s) put in place all necessary measures to minimize the risks and follows the recommendations given by authorities in target countries and local communities.

It is expected that the evaluation field research should be conducted in at least one selected site or location per country, in all four target countries (Banja Luka, B&H; Belgrade, Serbia; Pristina, Kosovo and Tirana, Albania). If the travel is restricted due to state of emergency such as in case of a recurrence of the pandemic and lock down – it will be done online in cooperation with CARE and partners.

The Evaluator(s) will be paid in two separate installments: 

  • First payment will be made upon submission the receipts of first invoice and consultancy time sheet (CARE will provide templates) with indicated tasks performed and dates of tasks to be implemented after the approval of the inception report and no later than 7 days after its submission. The first payment will include travel advance for Field research. The total payment will correspond to a maximum of 7 working days and travel costs. 
  • Second and final payment will be made upon submission of the receipts of the second invoice and consultancy time sheet with indicated tasks performed and dates of tasks to be implemented after evaluation report is approved, no later than 7 days after the submission day. 

The payment for the work defined in the Contract will be made upon the work completion and the final product delivered to CARE, with all the necessary documentation as per CARE’s administrative procedures.

The deadline for application is August 16th, 2020 4pm CET.

Coordination/Responsibility

The Evaluator(s) is expected to commit to the agreed timetable and outputs:

  • Prepare a work-plan in agreement with CARE
  • Conduct desk research and deliver Inception report;
  • Conduct Field researches based on agreed work plan;
  • Submit draft and final evaluation report based on agreed timeline and work plan;
  • Maintain timely communication with CARE Project Manager;
  • Coordinate logistics (organization of meetings, etc.) in agreement with CARE.

CARE commits itself to:

  • Coordinate with the Evaluator(s) and agree on the work plan
  • Provide requested documentation to Evaluator(s)
  • Provide logistical support to the Evaluator(s)

The Evaluator(s) will report to Emilija Milenković Project Manager, [email protected] 

The Evaluator(s) will be responsible for leading and coordinating the whole process of the final evaluation based on inception report and including writing the final evaluation report. 

Required qualifications

The Evaluator(s) must meet following criterions:

  1. More than 5 years of comprehensive professional experience in conducting evaluations 
  2. Excellent oral and writing skills in English and in at least 1 local language (Albanian, B&H languages). 
  3. Ability to work in team and under a pressure.
  4. Prior evaluation experience of working in the Balkans region.
  5. Prior experience with projects targeting young men and women beneficiaries and/or gender mainstreaming and/or educational policy is considered an asset.
  6. Knowledge of the national/regional and EU public policies targeting prevention of gender-based violence and promoting gender equality is considered an asset.

The consultants must not have been involved in the design, implementation or monitoring of this project.

Applications

Individual experts, companies or evaluation teams are eligible. The candidates (individual, company or teams) are expected to include in the application: 

  1. Cover letter 
  2. Technical Offer covering all aspects and tasks required in the ToR. The technical offer must include:
    • Approach, methods and design including tentative work plan and division of responsibility (if team of consultants is applying) suggested by the applicant (3 pages)
    • Professional profile of the evaluator(s) (CVs of all individuals included in the consultancy team)indicating contacts on reference persons from at least three former employees/contractors; 
    • Former evaluation reports or other analytical outputs produced by the applicant.
  3. Financial Offer including cost breakdown. 
    • The consultant(s) shall suggest the global price in EUR (per position if evaluation team applies) and the number of days predicted for the work. 
    • The global price must include all administrative costs (such as but not limited to backstopping costs, insurance, reports, communication costs, rent for Contractor’s facilities), as well as a breakdown of travel, allowances and accommodation expenses for Field Research 1 (as specified in the Timetable section) .
    • Prices must in gross amount (free of all duties, taxes and other charges, including VAT). The amount of VAT may be shown separately.

Selection:

The applications will be assessed as follows:

  • Financial offer 30%
  • Technical offer 70%

Offers should be submitted by 16th August 2020 4pm CET. to the following e-mail address:  

Procurement Team, CARE International Balkans, e-mail address: [email protected] 

The selected will be notified in writing by  20th August 2020.

Deliverables

Following documents have to be submitted via e-mail or other softcopy formats:

  1. Inception Report
    • The Inception Report should include the detailed evaluation approach and methodology suggested specifying data collection methods and related sampling strategies and potential limitations, risks and related mitigation strategies, an evaluation matrix which will outline how each evaluation questions will be answered (e.g. sources, indicators),a work plan with division of labor in the evaluation team (if applicable), and draft data collection instruments (in an annex).  
    • The inception report should not be longer than 20 pages without annexes (ToR, overview of documents used, interview guidelines, etc). Arial or Times New Roman, line spacing 1.15, font 11.
  2. Final Evaluation Report: 
    • Should be no longer than 30 pages without annexes (Arial or Times New Roman, line spacing 1.15, font 11) 
    • Should be structured in the following manner: Title, Index, list of abbreviations, Executive summary, Background, Introduction, Methods, Evaluation findings, Conclusions, Lessons learnt, Recommendations, Annexes
    • Should be based on OECD/DAC Evaluation Quality Standards as framework, as required by the donor.
    • Should present disaggregated data based on sex and, when feasible, ethnicity. 
    • Should contain an executive summary (5 pages max)
    • Where feasible, an overview based on country/locality differentiated data should be provided.
    • Should provide an overview of most important highlights (such as for instance best and bad practices, success stories, challenges, lessons learned) in separate boxes and illustrating the findings with quotations.
    • Available monitoring tables developed through the project should be attached to the evaluation report after checking for consistency.
    • Should respect the templates provided by CARE  
    • The report must fulfill the requirements listed in the ToR.
  3. Evaluation Summary Report: 
    • Should be no longer than 5 pages (Arial or Times New Roman, line spacing 1.15, font 11) 
    • Should be structured according the Final Evaluation Report chapters – Introductory Summary, Methods, Lessons Learned, Findings, Conclusions and Recommendations. 
    • Should present disaggregated data based on sex and if feasible ethnicity. 
    • Providing an overview of most important highlights (such as for instance best practices, success stories, challenges, lessons learned) in separate boxes and illustrating the findings with quotations.
  4. Results-Assessment Form for Final Project Evaluations that can be downloaded on the following link 

All deliverables should be written in English. 

  • Categories

  • Recent Posts

  • Archives