Published in April 2023
Independent High-Level Evaluation of ILO’s COVID-19 response 2020-22
The COVID-19 pandemic forced the International Labour Organization to find new ways to support its constituents and contribute to the global United Nations response. This evaluation report assesses its institutional and policy response to the first two years of the pandemic.
Photo: Health workers in Indonesia during the COVID-19 pandemic, June 2020. © ILO
About the evaluation report
The ILO has a history of dealing with natural, economic and global health emergencies, but the magnitude of the coronavirus (COVID-19) pandemic is something new. The Organization’s governance and resources were faced with unprecedented challenges, which forced it to find new ways to support its constituents and contribute to the global United Nations response.
The Independent high-level evaluation of ILO’s COVID-19 response 2020-22 focuses on two dimensions – institutional and policy – of the Organization’s response to the first two years of the pandemic, from March 2020 to March 2022. It also presents key findings and an overall assessment, using the following evaluation criteria: relevance, coherence and design, effectiveness, efficiency, sustainability and impact.
The ILO was forced to develop new policies to address this new world, under a four-pillar framework:
- Stimulating the economy and employment
- Supporting enterprises, jobs and incomes
- Protecting workers in the workplace
- Relying on social dialogue for solutions.
To develop and promote these policies, the Organization produced more than 170 COVID-19-related publications, performed rapid assessments of the pandemic’s effects on 47 countries, and conducted surveys and other activities.
As the pandemic remains a challenge, the high-level evaluation also provides lessons learned and recommendations going forward.
The evaluation was undertaken by ILO’s Evaluation Office with support from a team of external consultants. The exercise benefitted from substantial preparatory work, including guidelines on how to conduct evaluations during the pandemic, a protocol on gathering evaluative evidence during the pandemic, and a two-staged synthesis review of COVID-19 evaluative evidence gathered from completed project evaluations.
The high-level evaluation considered two “evaluative dimensions” of the ILO’s response to the COVID-19 pandemic:
- Institutional readiness, adaptability and capacity to deliver timely support in a responsive manner; and
- Policy action at national, regional and global levels.
Mixed methods were used to provide an evidence-based assessment of the ILO’s response to the crisis. As part of data collection, the evaluation team conducted document reviews; 354 interviews with constituents, staff, funding partners and other ILO partners; surveys; and 14 country and thematic case studies.
ILO's institutional response to the pandemic
When the worldwide COVID19 pandemic struck in early 2020, the ILO was forced to rapidly adapt its current management structure and resources to confront this unprecedented challenge.
Overall, management responded well to the uncertainty and unpredictable change, and its crisis management approach demonstrated agility, flexibility and a willingness to continuously adapt.
Navigating the crisis
Although the ILO had begun preparing its response well before 11 March 2020, when the World Health Organization declared COVID-19 a pandemic, none of the Organization’s policies, plans or procedures in place at the time could fully prepare it for the global scale and impact of the crisis.
A Global Management Team was created to complement the work of the existing Crisis Management Team, which was comprised of senior management. Case studies were carried out in several country offices, and constituents expressed overall satisfaction with the Organization’s response, though some reservations were expressed by both staff and constituents.
Governance during the crisis
ILO governance structures – unique in the United Nations system in their application of the principles of social dialogue through the Organization’s tripartite nature (governments and employers’ and workers’ organizations) – were challenged in unprecedented ways, demanding speed, adaptability and new virtual processes.
In the early stages of the pandemic, the usual governance processes were circumvented to some extent, but ILO senior management stressed that maintaining social dialogue and continuing to engage with constituents remained the highest priorities.
Constituents felt that the new governance and management processes introduced during the pandemic that proved to be efficient and effective should be maintained in the post-pandemic world.
Resourcing the crisis response
The crisis required the ILO to make rapid adjustments to its planned expenditures. It quickly established procedures to adapt regular budget and development cooperation funds to new circumstances, while maintaining accountability.
The Organization briefed its funding partners on the situations faced in the field and discussed how projects might be adapted. It also mobilized new voluntary contributions from development partners. Human Resources Management helped the Organization to adapt quickly, establishing new systems, equipping staff to work remotely and ensuring workplace safety.
Despite these steps, lack of budget flexibility and inadequate speed were sometimes reported as obstacles to a rapid response.
Support to constituents and social dialogue
When the COVID-19 pandemic struck, the ILO acted quickly to identify and address the challenges faced by constituents.
The ILO Bureau for Employers’ Activities (ACT/EMP) helped employers’ organizations face the institutional-level challenges of maintaining membership, communicating with and delivering services to members, and engaging in advocacy.
The ILO Bureau for Workers’ Activities (ACTRAV) connected with workers’ organizations across the globe to collect and disseminate national trade union responses to the pandemic. The International Training Centre of the ILO (ITCILO) played a central and much expanded role in building constituent capacity to meet the many new challenges faced during the pandemic.
Contribution to the UN response
Thanks to its normative role regarding labour and employment, and its capacity to produce authoritative labour market data, the ILO possesses a unique level of relevance to inform the United Nations system’s pandemic responses at national and international levels.
However, this positive contribution has not translated into increased UN funding, and has caused the Organization’s resources to be stretched thin. Particularly in the context of UN reform, with its premium on inter-agency collaboration and joint programming, the major challenge going forward will be for the ILO to deliver on its substantial new commitments as part of its overall UN COVID-19 response.
Monitoring the ILO’s response
When the pandemic hit, the ILO had to find a way for its established planning, monitoring and reporting systems to adapt quickly and track the Organization’s COVID-19 response.
Developments in the systems aimed to better align, within the results-based management system, the programmatic and financial dimensions of the response. Ultimately, however, the ILO’s tracking of its COVID-19 response through this system was deficient. Details of results were often imprecise or poorly reported.
Even so, results from the external ex post quality appraisal of evaluation reports from 2020–21 showed that overall quality ratings for evaluations in 2021 were above those from 2019.
Perspectives from staff and constituents
A survey of staff members rated the overall preparedness of the ILO’s institutional response to the COVID-19 pandemic at 5.62 on a 6-point scale. It also asked staff to identify the top three successes and obstacles concerning the Organization’s response. A survey of constituents and partners gave the ILO a rating of 5.9 for overall preparedness.
However, participation in the latter survey was heavily skewed towards workers’ organizations, with 62.2% of responses, in contrast with employers’ organizations representing 27.7% and government agencies just 8.4%. This survey also asked participants to name the Organization’s top three successes and obstacles.
ILO's policy action during the pandemic
Senior ILO staff began considering possible policy responses to COVID19 as early as January 2020. When the World Health Organization declared the pandemic in March of that year, the Organization responded swiftly to the crisis by providing data, statistics and knowledge products to its constituents.
Evolution of ILO’s policy response framework
In 2020, during the earliest stages of the COVID-19 pandemic, the initial ILO response was framed through its “four-pillar” policy and the Centenary Declaration. The next step was to inspire global action through a true tripartite agreement.
Over the following months, there were continuing consultations with constituents on these “building blocks”, which culminated in the adoption by the International Labour Conference of the Global call to action for a human-centred recovery from the COVID-19 crisis that is inclusive, sustainable and resilient. A staff survey provides insight into perceptions of the relevance of actions taken in various policy areas.
Improvements to knowledge and guiding policy
The ILO responded swiftly to the COVID-19 crisis by providing data, statistics and knowledge products to its constituents. From March 2020 to October 2021, it produced more than 170 COVID-19-related policy publications and launched a COVID-19 Information Hub.
Publications aligned with the four-pillar policy framework: (a) stimulating the economy and employment; (b) supporting enterprises, jobs and incomes; (c) protecting workers in the workplace; and (d) relying on social dialogue for solutions. This required a high level of innovation and efficiency in delivery. Despite some reservations, staff and constituents had an overall positive perception of the relevance of these knowledge products.
Inclusive economic growth and employment
The ILO is a global leader in analysing and communicating actions that promote inclusive economic growth and employment, both in the immediate crisis response stage and in the long-term recovery.
Rapid assessments of the country-level economic impact of COVID-19 were conducted in more than 47 countries. A tool was developed and applied in 14 countries to assess reskilling and upskilling needs. Both the Global Call to Action and the ILO’s four-pillar policy framework defined areas of policy action designed to minimize the damage caused by the pandemic on the quantity and quality of jobs, and to promote a broad-based, job-rich recovery.
Protection of all workers
The action areas defined in the Call to Action and the four-pillar framework were designed to protect workers’ fundamental rights, health and safety, and working conditions affected by COVID-19.
Some of the ways the ILO addressed these goals included: (a) reinforcing international labour standards; (b) preventing child labour and forced labour; (c) protecting wages and working conditions in emerging and non-standard work; (d) protecting informal economy workers and other vulnerable groups; (e) protecting migrant workers and refugees; (f) taking collaborative action on the future protection of migrant workers; and (g) making protection efforts promoting gender equality, diversity and inclusion.
Universal social protection
The pandemic has exposed and exacerbated pre-existing labour market and structural socioeconomic inequalities within and across countries, disproportionately affecting the most vulnerable populations.
Actions taken by the ILO to address this problem include assessing social protection gaps and needs; developing social protection responses through social dialogue; repurposing social protection projects and developing new interventions; mobilizing and delivering emergency cash transfers; formulating policy responses to the COVID-19 crisis; strengthening the governance and financial sustainability of social protection systems; increasing capacities to integrate social protection in comprehensive policy responses; developing capacities and knowledge products; and targeting vulnerable groups, including migrants and informal workers.
Alignment with United Nations and the Sustainable Development Goals
In 2015, the United Nations, through its 2030 Agenda for Sustainable Development, established 17 Sustainable Development Goals and 169 Targets, with the aim of promoting peace and prosperity.
As part of those efforts, because of its role regarding labour and employment, the ILO has become a leader in the UN system’s global pandemic response.
Some of the actions the Organization has taken in this regard include contributions to the financial architecture of the global pandemic response, financing the ILO’s work with UN and multilateral partners, staffing the ILO’s work with those partners, and enhancing collaboration and coordination with them.
Country Programme Outcomes and financial databases
The analytical strategy developed by this high-level evaluation for the analysis of Country Programme Outcomes (CPOs) consists of two main phases: phase 1, which refers to the analysis of data on the ILO’s Decent Work Results dashboard to identify reports on COVID-19; and phase 2, which refers to the financial analysis of expenditures associated with the selected CPOs and global products that reported on COVID-19 responses.
According to the 2020-21 Programme Implementation Report, the ILO exceeded the target set for the biennium by 3%, with the achievement of 896 results in 151 Member States and 2 territories.
Perspectives from staff and constituents
A staff survey suggested that a majority believed the ILO was taking the necessary steps in designing and implementing COVID-19 recovery actions. Respondents also felt its actions were relevant to its core mission and cross-cutting principles. They were less supportive in rating coherence and collaboration between policy areas, and in their evaluations of the effectiveness of the ILO’s work in implementing projects and programmes.
Constituents tended to rate ILO efforts positively, but the small sample size of respondents requires caution in interpreting these results. There was also a high level of “don’t know” responses to constituent survey questions.
Evaluation's key findings and lessons learned
This chapter summarizes the findings of the high-level evaluation. Ratings for the criteria cover the ILO’s response at both the institutional and policy action levels, specifically in the areas below.
The ILO’s overall response to the uncertainty and unpredictable change the pandemic brought about was highly relevant. Even as ILO management was forced to make quick decisions, this enabled the Organization to adapt to a dramatically altered operational landscape, and constituent engagement through social dialogue remained its highest priority.
However, even as the ILO elevated its profile within the United Nations system, its contributions at the country level were somewhat limited. To continue to be relevant in shaping the future of work, its actions will need to remain responsive to continuous, unpredictable change, not just to the damage left by the pandemic.
The pandemic played a catalytic role in improving the ILO’s collaboration and policy coherence, helping it to break out of its perceived “silo mentality”.
However, the walls of the silos have not completely crumbled. To address this, the Organization set up structured mechanisms to encourage and reward cross-departmental interactions, notably through the increased use of virtual meetings.
The ILO worked hard to ensure that these changes cohered with its emphasis on social dialogue, the Centenary Declaration and the centrality of international labour standards. To this end, new mechanisms for coherence and collaboration with other UN agencies and multilateral partners emerged.
The ILO was partially successful in adapting operational planning and reporting systems to track and measure the effectiveness of its COVID-19 response. Although it carried out an arduous, line-by-line review of Country Programme Output reports, there was still some subjectivity involved in this, and important details might have been lost.
Despite these deficiencies in monitoring and reporting, and the fact that decisions related to adapting systems were made in an environment of great operational uncertainty, the work in support of inclusive economic growth and employment enabled the effects of the pandemic on national labour markets to be better understood.
Despite unprecedented circumstances, the ILO managed the crisis in an efficient and timely manner, quickly reinventing its service delivery model, defining a coherent policy framework and asserting its position as a global authority on the pandemic’s effects on the world of work.
At the governance level, the Organization was able to adapt its long-established mechanisms for decision-making and constituent engagement, and achieved some new efficiencies in the process. It established procedures to support budget flexibility while maintaining accountability.
However, despite the Organization’s impressive output in its response to the pandemic, the high-level evaluation was unable to accurately evaluate its cost-effectiveness.
Impact and sustainability
Measuring the impact and sustainability of the ILO’s policy actions will require more time and commitment, but many countries are already more alert to the need for action. There is also a new impetus for cooperation between the ILO and other UN agencies, multilateral partners and international financial institutions.
At an institutional level, the ILO has emerged from the initial crisis phase of the pandemic with experience in radically and quickly adapting its operations. The Global Accelerator on Jobs and Social Protection for Just Transitions could also have transformative impacts, but these will require strong partnerships and substantial financing.
- New work practices can enhance headquarters interaction with the field.
- The pandemic forced the ILO to produce agile and innovative responses in service delivery.
- The crisis response showed that leadership can improve organizational coherence.
- Digital delivery of ILO services offers the opportunity to expand reach and scale, but the digital divide persists.
- Monitoring and reporting of crisis response actions need to be improved.
- The pandemic will have an enduring effect on the ILO’s service delivery approach, but in-person missions still bring benefits.
- The pandemic has highlighted additional dimensions of occupational safety and health, such as mental health.
Ratings cover the ILO’s response at both the institutional and policy action levels.
The ratings are based on the assessments of the five evaluation team members, the Synthesis Review of project evaluation reports completed in the period, and the findings of the staff and constituent surveys.
Recommendations and Office response
This high-level evaluation focuses on two dimensions of the ILO’s response to the COVID-19 pandemic for the two-year period of March 2020 to March 2022: how well the ILO adapted at an institutional level, and how well it refocused its policy work to meet the changing needs of constituents.
Based on the lessons learned during this exercise, the evaluation team presented eight recommendations to guide the Organization going forward.
Continue to strengthen the capacity of the tripartite constituents to enhance and adapt their services to contribute to the development of effective global, regional and national post-pandemic recovery policies and actions.
Develop an Organization-wide crisis response strategy encompassing both headquarters and the field.
Expand and mainstream more broadly the approach to cross-departmental teamwork demonstrated in the pandemic and continue the efficient and effective management and governance practices that were introduced.
Enhance the ILO’s capacity to monitor, report and evaluate crisis response actions that are developed and implemented outside the normal programming cycle.
Strengthen the institutional capacity of governments to respond to systemic crises through universal social protection.
Continue to strengthen the constituents’ capacities to sustain international labour standards and fundamental principles and rights at work for workers, even during a crisis, and develop inclusive, gender-responsive policies for the protection of workers in insecure forms of work.
The ILO should more clearly integrate a just transition into its post-pandemic employment and skills development strategies and actions,and use its experience and expertise to implement approaches with maximum potential for impact. It should pursue financing and delivery partnerships with organizations with resources to help bring a just transition to scale.
The ILO should review its current capacity to deliver on the whole-of-government approach and new models of development financing, focusing on the scale and distribution of workload implied by its agreements as part of the UN COVID-19 response (including with both UN and other multilateral organizations), and devise a prioritized and specific plan to meet the resource requirements, including at the country level.