Evaluation of quality apprenticeship programmes
The issue: Why evaluation of apprenticeships is necessary
The active participation of various stakeholders is essential for the success of apprenticeships. To secure their continuous support and participation in apprenticeships, it is important to be able to provide concrete evidence of the outcome and impact of programmes, and the net benefits to different stakeholders.
A tracer study can help to answer a number of questions about the transition of apprenticeship graduates to the labour market, such as: How many are hired by their training employers directly after training? How quickly can apprentices find a job if they do not remain with the apprenticeship employer? What level of income can the apprenticeship graduates obtain? How many of them choose to continue with their education? What situation are do graduates find themselves in, half a year or one to two years after finishing their apprenticeship? These data, taken together, provide a comprehensive evaluation of the outcomes of an apprenticeship programme.
The steps for conducting a tracer study are as follows:
- Plan and design a tracer study.
- Formulate research questions.
- Develop a questionnaire.
- Send out the questionnaire and collect data.
- Analyse the data and interpret the results.
For further details, see Tool "Guide to tracer studies".
For example, potential apprentices would like to know the outcome of the programmes in terms of employment rates, income levels and long-term career prospects. They would also like to be able to assess the quality of the training offered by various enterprises and TVET providers. For these purposes, tracer studies can provide a very useful tool for gathering the relevant information.
Enterprises would like to be able to weigh the costs and benefits of apprenticeships while governments’ interest may focus more on the social and economic outcomes of apprenticeships (an example from South Korea is given in box 6.3). In the United States, registered apprenticeship programmes are evaluated using an apprenticeship outcomes performance matrix, which lists indicators for employers, apprentices or workers, and partner organizations (tool "Tool 5.2.9 Apprenticeship outcomes performance matrix"). This tool seeks to determine “whether using apprenticeship as a talent development strategy has been valuable to employers; created opportunities for workers to enhance their skills, earn industry-recognized credentials, and increase earnings; and helped workforce, education, and community partners achieve their goals” (United States Department of Labor, n.d.). For policy-makers, evaluation at the system level may also involve benchmarking of their national apprenticeship system against internationally recognized good practices.
Therefore, evaluation of the apprenticeship system and programmes is essential to collect information on performance which provides evidence to inform stakeholders’ decisions regarding the changes needed to improve and strengthen the apprenticeship system and programmes. Stakeholders would also be able to decide whether a particular apprenticeship programme should be scaled up, limited or stopped, and how it could be improved.
While the evaluation of the system and specific programmes can be carried out separately, the evaluation findings at the two levels are inextricably linked. Very often, the implementation challenges at the programme level stem from underlying gaps or weakness in the broader policy environment.
Before developing apprenticeship programmes, practitioners need to be fully aware of the design features, training and working conditions for apprenticeships prescribed in the law and the regulatory framework. Therefore, policy-makers and practitioners should cooperate and carry out regular evaluation of the apprenticeship system and programmes, developing comprehensive evaluation mechanisms and methodologies for that purpose. They should share evidence and data gathered from a wide array of evaluation mechanisms and methodologies, in order to obtain a holistic perspective of the strengths and weaknesses of the apprenticeship system or programmes.
Box 6.3 Economic and social outcome analysis of apprenticeships in South Korea
In South Korea, apprenticeship programmes are evaluated based on both economic and social outcome analysis.
Economic outcomes are examined through cost–benefit analysis from the employers’ perspective. The total cost includes personnel expenses (e.g. apprentices’ income, in-CT allowance), operating expenses (e.g. teaching materials, training equipment) and apprentice selection and recruitment expenses (e.g. advertisement and promotion). The total benefit comprises, among other factors, apprentices’ enhanced productivity, reduction in the cost of new worker recruitment or retraining and government incentives.
Social outcomes are categorized and assessed on three levels:
- National level – Four outcomes, namely, improvement in youth employment rates, reduced age of entry into first job, reduced period spent seeking employment, alleviated mismatch between supply and demand.
- Organizational level – 11 outcomes, including company’s strengthened competencies in the implementation of on-the-job training and the quality of internal training and improvements in work performance, adaptability and satisfaction of apprentices.
- Individual level – Seven outcomes, including improvement in employability and job performance as well as adaptability of enterprises and enhanced psychological well-being due to increased job security and welfare provision.
Source: Kang, Jeon and Lee, 2017.