Terms of References for the external final Evaluation of RSF project “Strengthening Press Freedom Centers in Ukraine”

 

1/ General information

1.1/ Introduction and context

Since the Russian full-scale invasion of Ukraine on 24 February 2022, the Ukrainian media landscape has been under pressure and has suffered the consequences of the war. Indeed, the capacity of journalists and media outlets to provide reliable information has been hindered over the past two years by continuous psychological, physical and financial pressure. Journalists and media outlets are still being targeted and constantly face additional challenges and threats.

Reporters sans frontières (RSF – Reporters Without Borders) has undertaken actions since the beginning of the full-scale war to ensure that journalists and media outlets can pursue their work and to guarantee access to independent and reliable information in Ukraine.

In 2022, RSF opened two Press Freedom Centers in Kyiv and Lviv to provide on-site support and assistance to journalists and media. The Centers have been essential to enable journalists and media to continue covering the war and its consequences by providing equipment, training, psychological support, etc.

RSF is also raising awareness about the situation in Ukraine, and particularly about the violations of press freedom and crimes committed against more than 100 journalists and 14 media facilities. RSF has been actively contributing to the fight against the impunity of these crimes, and denouncing restrictions on press freedom and disinformation related to the war.

 

1.2/ Reporters sans frontières

Founded in 1985, Reporters sans frontières (RSF) defends the right to reliable information. Its mandate is based on article 19 of the Universal Declaration of Human Rights, which states that: « Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers ».

RSF strives to ensure that all human beings benefit from information that enables them to know, understand and form an opinion on the issues facing the world and their environment. To achieve this, the organisation is developing a holistic strategy, with 360° activities, to bring about global change. RSF acts on four levels: press freedom, relations between the public and journalists, the information market and the information space.

RSF is creative in creating systemic initiatives that address the causes of problems: the Journalism Trust Initiative (JTI) and the Partnership on Information and Democracy (I&D).

RSF has an international secretariat in Paris, thirteen sections and offices around the world, more than 150 correspondents, 4 representatives and local partners in a wide range of countries. RSF is a registered association in France and has consultative status with the United Nations and UNESCO.

 

2/ The project

The project to evaluate is entitled “Strengthening Press Freedom Centers in Ukraine”, funded by the Delegation of the European Union in Ukraine (EEAS).

The project contract was signed on 12 October 2023. The project began on 1 June 2023 and will end on 30 November 2024 (18-month implementation period).

The total budget for the project is 1,450,240.15€, of which 1,300,000.00€ is funded by the European Union and 150,240.15€ are covered with other sources of funding.

The project has already been externally evaluated as part of a ROM review mandated by the European Union in July 2024.

The RSF team working on the project are: the Project and MEAL department, the International Coordination department, the Advocacy and Assistance department, the Eastern Europe and Central Asia (EECA) Desk, the Communication department, the Editorial department and the Journalism Trust Initiative (JTI) department.

In the framework of this project, RSF cooperates with two types of organisations (in addition to service providers) :

• local organisations defending press freedom, to ensure the relevance and implementation of some project activities;
• other international non-governmental organisations with which RSF coordinates support to Ukrainian journalists.

This project is a continuation of a previous RSF project in Ukraine funded by the European Union (10 March 2022 – 9 March 2023). The aim of this project is to consolidate and complement the actions implemented in the first project. A final external evaluation of this first project was carried out, which enabled RSF to draw lessons learned and take them into account in designing and implementing its second EU-funded project in Ukraine.

2.1/ Objective of the project

The project aims to contribute to fostering access to independent and reliable media coverage of the crisis in Ukraine and the region.

2.2/ Target countries

The target country is Ukraine.

2.3/ Target groups

The project is designed to address the needs of journalists and media professionals – both ukrainians and foreign – covering the war in Ukraine as well as Ukrainian media outlets.

The final beneficiaries also include the general public that will benefit from reliable information and could therefore be able to better identify and avoid disinformation.

2.4/ Objectives and planned activities

The specific objective of the project is to contribute to providing and promoting a safer environment for journalists, media professionals and media working in the coverage of the crisis in Ukraine and the region.

The project includes three output:

Output 1: Journalists, media professionals and media covering the war in Ukraine are better equipped and assisted/supported to work in the current conflict context

• Activity 1.1: Provide journalists covering the crisis with the necessary working equipment to continue their work
  • Task 1.1.1: Identification and assessment of potential additional needs in equipment or goods.
  • Task 1.1.2: Provision and distribution of protective equipment (bullet proof vests and helmets) and other equipment and goods (IFAKs, emergency preparedness kits including possible chemical and nuclear protection…)
• Activity 1.2: Implementing training programs for journalists and media covering the war in Ukraine
  • Task 1.2.1: Capacity building of journalists in crisis situations: Training programs in physical security and in digital security, including cybersecurity audits to media (7 sessions of 3 days)
  • Task 1.2.2: Psychological support to journalists : Training programs in psychological security and access to individual counselling
  • Task 1.2.3 : Psychological support to journalists : Residency programmes
• Activity 1.3: Provide direct financial support to media outlets covering the crisis in Ukraine
  • Task 1.3.1: Identifying relevant media (meeting RSF’s criteria) in need of support in Ukraine in collaboration with other local organisations
  • Task 1.3.2: Media Support Fund – Financial Support to media outlets
• Activity 1.4: Develop the Journalism Trust Initiative (JTI) Emergency Protocol
  • Task 1.4.1: Deploying the JTI emergency protocol
  • Task 1.4.2: Supporting a Ukrainian certifier in becoming an official JTI

Output 2: Crimes committed against journalists, media professionals and media in Ukraine are documented and legally addressed

• Activity 2.1: Press freedom monitoring and research
  • Task 2.1.1: Daily monitoring of the situation
  • Task 2.1.2: Updated map of attacks against journalists and media outlets
  • Task 2.1.3: Investigation and inquiry into the abuses committed
• Activity 2.2: Advocacy and strategic litigation
  • Task 2.2.1: Strategic litigation (complaints and filings)
  • Task 2.2.2: Contribution to ongoing legal proceedings
  • Task 2.2.3: Advocacy activities (Multilateral and bilateral) – participation to international and other meetings

Output 3: Public opinion is informed about and mobilised against press freedom violations and aware of RSF’s actions.

• Activity 3.1: Production, broadcasting and promotion of a documentary resulting from the investigation
  • Task 3.1.1: Production and broadcasting of the documentary
  • Task 3.1.2: Organise one or more major debate events with the projection of the documentary film
• Activity 3.2: Inform general public and raise public awareness on the evolution of the situation and the abuses committed against journalists
  • Task 3.2.1: Define and implement specific communication campaigns
  • Task 3.2.2: Publication of regular press releases and reports on the evolution of the situation and the abuses committed against journalists.

3/ The assignment

3.1/ Intended use of the evaluation and stakeholders

The purpose or intended use of the evaluation is for RSF to identify and build on the strengths and room for improvement of its project, for the designing and implementation of future similar projects in Ukraine or other crisis-contexts.

The main stakeholders who will use the results of the evaluation are : the Project officer, the MEAL Officer, the global RSF team in general, and the donor. The evaluation is to be designed, conducted and reported to meet the needs of these stakeholders. During the inception phase, the evaluator and RSF will agree on who will be responsible for keeping the various stakeholders informed about the evaluation process and results.

3.2/ Evaluation scope

The evaluation scope is limited to the activities implemented in the scope of the project in  Ukraine between 1 June 2023 and 30 November 2024.

3.3/ Evaluation objectives:  criteria and questions

The objectives of this evaluation are to:

• Provide RSF with a critical analysis of the quality of the project implementation which will serve as a basis for RSF internal learning process ;
• Suggest areas of improvement for the designing, implementation, and monitoring of RSF’s projects and activities in Ukraine ;
• Suggest areas of improvement for synergies with local organisations and other projects in favour of the media landscape in Ukraine ;
• Suggest areas of improvement for taking gender into account for the possible renewal of the project ;
• Assess the sustainability and measure the preliminary impacts of the project ;
• Provide the donor with an assessment of the effectiveness of the project

The evaluation questions are:

Relevance and coherence

• Is the project’s intervention logic relevant?
• Are the different means of action deployed adapted to the political, security and cultural context of the target country and its evolution?
• Are the activities carried out and their observed results compatible with the expected impact and effects?
• Is the program complementary with the activities already carried out by other organisations and does it bring added value?
• How efficient was the synergy with international and local organisations? How could it be improved?

Effectiveness and efficiency

• Have the resources mobilised (human, material, financial) for the program activities been adequate?
• Did the reinforcement, reduction and modification of activities enabled the project to adapt to changes in context and/or overcome obstacles encountered?
• Did the intervention logic of the project allow it to reach a greater diversity of actors (profession, location, etc.)? To what extent?
• To what extent were the project results achieved?

Impact and sustainability

• Has the project already had a positive impact on the Ukrainian media landscape (journalists, media professionals, media outlets)?
• Did the psychological support activities answer the needs of media professionals and have a positive impact on the continuation of their journalistic activities?
• Did the intervention reach media and journalists with an extended and diverse audience? In particular, audiences reached by disinformation on the war?
• How sustainable is the project?
• How can gender be further taken into account for future projects in Ukraine or crisis situations?

Questions are expected to be commented on, developed and completed in the tender by the tenderer and further refined during the inception phase of the evaluation.

3.4/ Evaluation approach and methods

The evaluator is expected to describe and justify an appropriate evaluation approach and data collection methods in the proposition. The evaluator is to suggest a methodology that can provide credible answers to the evaluation questions. The evaluation design, methodology, methods for data collection and analysis are expected to be fully developed and presented in the inception report.

The evaluation of the project should be carried out in a participatory manner and be representative of all the stakeholders involved. It is expected that the evaluator will think through and facilitate the whole evaluation process, taking careful consideration of how the evaluation will be used. Applicants are therefore expected to present in their tender how the stakeholders are to participate in and contribute to the evaluation process. They must also present a methodology and data collection methods that create a space for reflection, discussion and learning among RSF teams.

Recommendations will be formulated to enable RSF to consolidate the strengths of its project implementation, and to take into account the various observations and recommendations that may have been identified by the evaluation for a possible renewal of the project.

A gender-responsive approach/methodology, methods, tools and data analysis techniques should be used.

In cases where sensitive or confidential issues are to be addressed in the evaluation, evaluators should ensure an evaluation design that does not put informants and stakeholders at risk during the data collection phase or the dissemination phase.

All the documents required for the evaluation of the programme will be made available to the evaluator.

3.5/ Organisation and evaluation management

The evaluation will be supervised by the MEAL officer with the support of the project officer. Relevant documentation and contact details will be provided by them.

3.6/ Evaluation quality

The evaluation shall conform to OECD/DAC’s Quality Standards for Development Evaluation. In this regard, the applicants shall specify in the tender how they intend to ensure quality during the evaluation process.

3.7/ Time schedule and deliverables

It is expected that a timeline and work plan is included in the tender and further detailed in the inception report. The timeline and work plan must allow flexibility in implementation. The evaluation shall be carried out between 2 October 2024 and 31 January 2025. The timing of field visits (if any), surveys and interviews need to be set by the evaluator in coordination with RSF during the inception phase.

The table below details key deliverables for the evaluation process:

 

Deliverables	Deadlines
Kick-off meeting	Between 2 and 4 October 2024
Draft inception report	Between 14 and 16  October 2024
Inception meeting (presentation of the draft report and discussion of RSF feedbacks)	Between 21 and 23 October 2024
Data collection	Between 24 October and 15 December 2024
Debriefing meeting	Between 16 and 20 December 2024
Draft evaluation report	6 January 2025 at the latest
Meeting to discuss RSF feedback	17 January 2025
Possible additional feedback on new version(s) of the draft report	Between 20 and 31 January 2025
Final evaluation report approved by RSF	31 January 2025 at the latest
Restitution with a PowerPoint presentation	Beginning of February 2025

 

The inception report will form the basis for the evaluation process and shall be approved by RSF before starting to implement the evaluation.

The inception report should be written in English. The report will include:

• An updated timeline and work plan based on the documentation review and the kick-off meeting.
• Updated methodology, evaluation questions, data collection tools, etc.
• A list of stakeholders who will be contacted and the dates of meeting.

 

The final report shall not exceed 30 pages and be written in English. The executive summary shall not exceed 5 pages and be written in English. The report, taking into account feedback from RSF, will include:

• An executive summary including the main conclusions and recommendations resulting from the evaluation (listed by order of priority) .
• A main report describing the context, objectives and methodology of the evaluation, the limitations, the detailed findings and results of the evaluation in relation to the objectives and evaluation questions, and the evaluator’s conclusions and recommendations (listed by order of priority).
• Appendix (list of documents, list of people interviewed, etc.)

The restitution in English should be based on a PowerPoint presentation to present the key findings and recommendations to RSF. The restitution should last 1 hour, with a 30-minute presentation to allow time for questions and answers.

3.8/ Evaluation team qualification

Preference will be given to consultants with relevant expertise and experience who propose a participatory evaluation methodology.

The following skills will be sought:

• Specific expertise and experience related to the project :
  • Experience in evaluating projects implemented by NGOs and/or international organisations in crisis context
  • Expertise in journalism and/or media ecosystem
  • Knowledge of the Ukrainian context
• Skills and significant experience in project evaluation (methodology, interviewing, analysis, report writing, etc.)
• Experience of evaluation of EU-funded projects ;
• Fluent spoken and written English
• Knowledge of Ukrainian and/or French is an asset.

It is important that the skills of the individual evaluation team members are complementary. It is highly recommended that local evaluation consultants are included in the team, as they often have contextual knowledge that is of great value to the evaluation.

The evaluators must be independent from the evaluated project and activities, and have no stake in the outcome of the evaluation.

3.9/ Financial and human resources

The maximum budget available for this evaluation is €20,000 all taxes included. This amount must include all the costs required to carry out the evaluation.

The assessment can be carried out remotely or the evaluator can decide to travel to Ukraine for the evaluation. In the event of a mission, the costs must be part of the total budget and the evaluator will be required to arrange the logistics including any necessary security arrangements.

4/ Submission of the offer and selection 

Consultants interested in the evaluation assignment should include the following documents in their application:

• A technical proposal detailing the understanding of the evaluation stakes (with a critical analysis of the terms of reference), the proposed evaluation methodology, as well as the implementation schedule considered.
• A CV describing education and experience.
• A list describing previous evaluations or consultancies. Please give details of similar evaluation/consultancy contracts: donor and organisation that implemented the project, budget and duration of the project concerned, budget and duration of the evaluation/consultancy, main results, etc.
• A detailed financial proposal (estimate) with the total budget all taxes included.

Proposals must be submitted in English. Incomplete applications will not be considered.

Full applications should be sent by email to the following addresses by 8 September 2024 at the latest:

• Charlie Troncy, MEAL officer : ctroncy@rsf.org
• Chloé Vernet, Project officer : cvernet@rsf.org

Proposals will be evaluated according to the following criteria:

• Technical and financial proposal
  • Understanding of the terms of reference
  • Evaluation questions
  • Proposed methodology
• Consultants’ expertise and experience
  • Lead Consultant
  • Associate consultant(s)
  • Complementarity of the team as a whole
• Budget and timetable
  • Proposed budget
  • Proposed duration, timeline and work plan
• Added value: specific tools, expertise, language etc.

The interviews with pre-selected applicants shall be organised from 9 September.