Structuring and standardisation of vocational orientation
Evaluation-based design of a central action area for vocational education and training
Carolin Kunert, Guido Kirst
Using the example of the BMBF programme “Supporting vocational orientation in inter-company vocational training centres and comparable VET centres” (BOP), this article explores the question of what a funding programme can contribute to the development of a central action area for VET, and the importance of a supporting evaluation in this context. The key data of the programme and evaluation are outlined in the introduction, and the article concludes with the presentation of the most significant outcomes of the evaluation and the consequences of these for the shaping of vocational orientation.
The action area of vocational orientation
Career choice decisions made by young people are governed by complex interdependent factors that influence one another. Alongside the interests and abilities of the individual, important roles are played by aspects such as the current respective situation on the labour and training market, reactions from within the social environment and the structuring of vocational orientation provision (cf. BIBB 2018, p. 448). At the same time, career choice decisions are (no longer) a one-off event. They usually need to be scrutinised and adjusted on several occasions over a lifetime. The objective of vocational orientation is, therefore, to foster young people’s career choice competence. To this end, it is important that due consideration is accorded to the various influencing factors in orientation and decision making processes. Inclusion of practical or business-related provision in school-based vocational orientation has in particular now become part of the quality standard (cf. e.g. BRÜGGEMANN 2015).
The programme “Supporting vocational orientation in inter-company vocational training centres and comparable VET centres” (“BOP”), which was instigated by the Bundesministerium für Bildung und Forschung (BMBF) [Federal Ministry of Education and Research], has taken on a structure-defining role as a beacon in the development of an action area for vocational orientation (cf. RATSCHINSKI et al. 2017, p. 120). This will be explained below.
Key data of BOP and of the supporting evaluation research
The BOP was initially introduced as a pilot project in 2008 before being established on a firm footing in 2010, earlier than planned. The programme provides for a two-day analysis of potential for school children, which takes place in Year 7 or Year 8. It also includes so-called “workshop days”, i.e. a period of two weeks in Year 8 during which pupils explore at least three occupational fields (out of a possible total of 18). The core objective of BOP is to strengthen pupils’ career choice competence and thus support them in arriving at well-considered and self-determined options.
The evaluation results from the pilot phase of the BOP indicated a good level of acceptance of the programme by all stakeholders involved. They also enabled the conclusion to be drawn that the practical occupational field explorations were a useful supplement to the vocational orientation provision that had previously existed. However, the initial evaluation failed to provide a response to a number of questions of policy and technical relevance. For this reason, BIBB commissioned further and much more extensive evaluation research in 2012 with the particular aim of investigating the impacts exerted by BOP on the career choice competence and learning motivation of the pupils.
This assignment was carried out between the start of 2013 and the end of 2017 by an investigation team comprising InterVal GmbH, the University of Hanover and qualiNETZ GmbH. Technical support was provided by the Vocational Orientation Programme Office at BIBB (cf. Information Box).
Research design of the evaluation
Evaluation research for the BOP programme encompassed the following investigation modules.
- Process data analysis from the BOP application portal (applications from two rounds, interim evidence on a one-off basis)
- Observations of the implementation of analyses of potential and workshop days (8 locations)
- Qualitative interviews with pupils at the participating locations with a control group (4 occasions, as of the time of the last survey n = 174)
- One-off qualitative surveys of education and training providers, network partners, teaching staff and parents at the locations involved
- Annual written provider questionnaires on various main topic focuses (full survey of around 400 participating vocational training centres per year)
- Written surveys of schools on the implementation of the BOP and vocational orientation (n = 257)
- Quantitative longitudinal survey of pupils with a control group (3 occasions, as of the time of the last survey n = 1,408)
- One-off cross-sectional survey of classes of school leavers with a control group (n = 2,179)
Even while the evaluation was still ongoing, the Vocational Orientation Programme Office made use of the findings which emerged by passing on work materials to the vocational training centres conducting the programme via vehicles such as workshops. These became an object of debates centring on changes to guidelines and also informed discussions between the Federal Government and federal states as part of activities undertaken within the Educational Chains Initiative 1. After the conclusion of the evaluation, the results are continuing to play an important role in the structuring of the BOP and in consultations between the Federal Government and federal states regarding the design of other programmes and further vocational orientation measures. The most significant results will be presented below together with example explanations of how these outcomes have contributed to the shaping of the action area or may do so in future.
The BOP as a structure-defining beacon programme
The evaluation confirms that the functional chain contained within the programme (analysis of potential – workshop days – company-based work experience for pupils – career choice decision, see Figure 1) has helped provide a structure to the vocational orientation process right across the country (cf. RATSCHINSKI et al. 2017, pp. 119 ff.). Programme interventions act as part of this functional chain to exert a positive influence on career choice competence and therefore also on the career choice decisions of most of the young people.
One of the fundamental ideas underlying the BOP was to create a “time-forward displacement of vocational orientation by closely linking school teaching and practical experience” (INBAS/IWAK 2010, p. 18) in order to reduce the number of young people without any prospect of a training place and have fewer persons in the so-called transitional sector. Although there had certainly been various federal state initiatives which had previously acted as prototypes for the BOP, the analysis of potential, the workshop days or comparable instruments did not become established as a fixed component of vocational orientation in almost all states until the BOP had been disseminated nationally and agreements had been put in place between the Federal Government, the federal states and the Federal Employment Agency within the scope of the Educational Chains initiative. Since 2012, a constant average of more than 160,000 school pupils per year have been financed using BOP funding. By way of comparison, the corresponding figures for 2010 and 2009 were around 100,000 and just over 42,000 respectively.
Agreements between the Federal Government, the federal states and the Bundesagentur für Arbeit (BA) [Federal Employment Agency] have now been concluded with 13 federal states. Analyses of potential or comparable competence assessment procedures form a constituent part of all agreements. In twelve federal states, endeavours are being undertaken to establish blanket coverage for all forms of school, in some cases including upper secondary schools. Comprehensive introduction for all school types took place in North-Rhine Westphalia with effect from the 2016/2017 school year. Workshop days or occupational field explorations have been incorporated into eleven Agreements.2
In order for vocational orientation to succeed, it is of crucial importance for schools to be closely involved since the law stipulates that they bear overall responsibility in this area. The programme has helped to ensure that vocational orientation takes place at an early stage, from Year 7, whilst the Federal Government-federal states-BA agreements are increasingly allowing the programme to be introduced at all forms of school. The BOP instruments are creating opportunities for timely consideration of issues relating to vocational orientation by school pupils, by teachers and by parents and guardians. At the same time, they are structuring the activities undertaken by the pupils and the support services provided by schools and external partners.
Schools and vocational centres normally work together over a period of several years. In more than half of cases, this leads to the development of ideas for joint projects which extend beyond cooperation within the BOP programme (cf. RATSCHINSKI et al. 2017, p. 54).
The influence exerted by the BOP on the development of a national vocational orientation system has, however, not merely been limited to a structural impact. The relevant instruments at federal state level are usually also aligned to the quality and implementation standards of the BOP. The Federal Government-federal states- BA agreements make specific reference to the quality standards set by the BMBF for the carrying out of analyses of potential.
The BOP was thus accorded the status of a beacon programme on the basis of the outcomes of the evaluation. However, the results also show that there is even greater scope for enhancement of the career choice competence of young people if qualitative further development of the programme takes place. Two examples of how it was possible to demonstrate that a particularly strong influence was exerted on career choice competence are presented below.
Shaping communication – the significance of preparatory and reflective discussions
Within the scope of the five-year evaluation, the quality and extent of communication with pupils during preparation, implementation and assessment of the BOP instruments emerged as significant success factors at a very early stage.
Carrying out an analysis of potential as soon as in Year 7, for example, was shown only to be useful in circumstances where the school had introduced and addressed the topic beforehand. The aims of the analysis of potential and of individual assignments also need to be explained in comprehensible terms to the young people taking part. In the case of the observation of the implementation of analyses of potential at eight BOP locations, however, it was revealed that some pupils had no precise idea of the purpose of the process, did not know which competencies were being monitored in the exercises and were unaware of the significance of these for their vocational orientation (cf. RATSCHINSKI et al. 2017, p. 32). Nevertheless, the quality of feedback interviews is the main crucial factor. The qualitative interviews with the young people signalled that they often perceive feedback from the analysis of potential to be one-sided and a “recommendation on the part of the member of staff carrying out the observation” (ibid., p. 56). Self-reflection which stimulates learning processes, however, requires prior understanding of the roles of self-assessment and external evaluation. Participants also need to be able to process their own experiences and arrive at their own conclusions. For this reason, the evaluation team recommends, for example, that the analysis of potential should end with an outcome which pupils have developed for themselves. This would also make it easier for the young people to use such a result as a point of reference for later decisions made in the career choice process (cf. ibid., p. 123).
A particularly important role is also played by process-support communication at the interfaces to the school and the family. These interfaces are only capable of being influenced to a very limited extent by the programme itself because of the large number of stakeholders. Nevertheless, the vocational training centres were encouraged both to pass on information and to provide assistance on how teachers and parents can integrate this data into their own respective context in an individual and efficient way. For this reason, the support provided by the Vocational Orientation Programme Office at BIBB has been ascribing increasing significance to the related topics of “communication, feedback and reflection” cf. the sample activities listed in the Information Box).
New support services for the BOP – two examples
- Seminar provision
A two-day seminar on the topic of “Feedback and reflection in the BOP” was designed in 2017. This workshop is aimed at educational professionals who conduct reflective discussions with the young people. Its aims are to raise awareness of the particular relevance of this element, to impart the necessary pedagogical attitudes, and to expand methodological competences for the structuring of feedback and reflection. The seminar has already been hosted four times. The concept is currently undergoing revision, and contents for a support brochure are being prepared. The intention is for this workshop to become part of standard training provision for educational professionals involved with the BOP. Federal state-specific adaptations are also being planned.
- Video diaries
Use of video diaries during the workshop days was initially introduced on a pilot basis in the summer of 2017. The idea is that daily entries to a video diary will support young people in discovering the strengths and interests they have and in identifying which occupational activities could suit them. In addition, the video diary serves as a basis for the reflective discussion that takes place following the workshop days. Because of positive experiences at the pilot location, various audiovisual and explanatory materials have been developed for both educational professionals and pupils. These will also make it possible for BOP providers to deliver the project independently in future (i.e. without media education support).
Reaching more target groups – internal differentiation and diversity of provision
Participation in the programme is not in itself sufficient to guarantee that the measure will actually achieve its objectives. Effects depend on how pupils accept provision at an individual level (cf. RATSCHINSKI et al. 2017, pp. 54 ff.). There are numerous possible approaches for further development of the programme.
The spectrum of occupational fields offered may in itself be decisive for whether the service is viewed as being appropriate or not. Girls, for example, are less likely to find the BOP provision appealing (cf. Figure 2). Boys benefit the most, particularly those who already know that they wish to enter training upon completion of schooling. If the programme is to live up to its objective of serving all school pupils equally, then this is an area in which development is still required.
Career choice continues frequently to take place in a gender-specific way (cf. e.g. BMBF 2018, pp. 40 ff.). Pupils are in particular need of support if they wish to explore occupational fields which are not typical of their gender. In many cases, vocational training centres hosting the workshop days allocate one of the occupational fields in a gender non-typical way in the hope that young people’s enthusiasm may be fired once they have had the chance to familiarise themselves with it. However, observations conducted as part of the evaluation revealed an opposite effect in the form of significant motivational problems in these groups. The evaluation team therefore recommends that the task assignments in the analysis of potential should be used as an initial vehicle to raise awareness for strengths that are gender non-typical.
However, the greatest potential for further development of the programme in line with requirements is to be found in the structuring of the measure itself. This makes it even more important to also appeal to young people who may be aspiring to achieve a university of applied sciences/general higher education entrance qualification after completion of lower secondary school rather than wishing to proceed to training (yet). One way of resolving this, for example, would be for the occupational fields covered during the workshop days also to address school-based training programmes and professions for which a higher education qualification is required alongside the dual training occupations. The general conditions of the programme could also be altered for schools which offer upper secondary level education.
The further outcomes that will emerge from the evaluation are of interest in this regard. Diversity of assignments and different individually matching degrees of difficulty of tasks are also revealed to exert a positive influence on the effect of the BOP programme. The more strongly in agreement the young people are with the statement “The difficulty of the tasks was exactly right for me”, the greater will be the development in partial dimensions of career choice competence (cf. BIBB 2018, p. 434). Deployment of state-of-the-art technology and the closest possible proximity to operational practice help to increase interest on the part of the pupils. Overlaps in tasks between the analysis of potential and exercises covered in the subjects of business studies and technology should be avoided so that pupils perceive that participation in the BOP will provide added value.
Just how such an individualisation of assignments can be realised within the scope of a highly standardised programme represents a challenge which the Vocational Orientation Programme Office is already tackling together with the BOP providers.
Conclusions and outlook
The evaluation of the BOP confirms that it is exerting an effect as a “beacon programme” and shows that a nationally structured support scheme within a particular action area, in this case vocational orientation, can succeed in initiating medium and long-term structural changes at federal state and local government level.
The evaluation also highlights the particular challenge that is inherent in the high degree of standardisation which the implementation of a funding programme of this magnitude inevitably brings in its wake. Although the standardisation of the BOP guarantees high recognition and comparability of the instruments in terms of content, this being an important prerequisite for its structure-defining function, standard elements can induce “uniform” implementation and thus reduce the intended effect of the measure, at least at an individual level. This is particularly revealed with regard to the requirements level of tasks and gender-sensitive design of vocational orientation.
Strengthening of the communicative elements and an internal differentiation of assignments are two major recommendations from the evaluation with regard to how the individualisation and therefore also the effectiveness of the programme can be strengthened. These and further recommendations will enable the evaluation to make an important contribution to further development of the BOP. Because of its beacon function, the programme will also provide significant impetuses for quality development within the action area of vocational orientation in overall terms.
- 1 The Educational Chains Initiative represents a joint commitment on the part of the BMBF, the Federal Ministry of Labour and Social Affairs and the BA to join forces with the federal states in order to dovetail successful funding instruments and create a holistic, nationally valid and coherent support system for vocational orientation and the transitional sector. For this purpose, the Federal Government, the federal states and the BA conclude state-specific agreements relating to the transition from school to work.
- 2 All Federal Government-federal states-BA agreements may be viewed at www.bildungsketten.de/de/388.php (retrieved: 30.07.2018).
BIBB (Hrsg.): Datenreport zum Berufsbildungsbericht 2018. Informationen und Analysen zur Entwicklung der beruflichen Bildung. Bonn 2018 – URL: www.bibb.de/datenreport-2018 (retrieved: 30.07.2018)
BRÜGGEMANN, T.: 10 Merkmale »guter« Berufsorientierung. In: BRÜGGEMANN, T.; DEUER, E. (Hrsg.): Berufsorientierung aus Unternehmenssicht. Bielefeld 2015, S. 65–79
BUNDESMINISTERIUM FÜR BILDUNG UND FORSCHUNG (BMBF) (Hrsg.): Berufsbildungsbericht 2018. Frankfurt/M. 2018
INSTITUT FÜR BERUFLICHE BILDUNG, ARBEITSMARKT- UND SOZIALPOLITIK GMBH (INBAS); INSTITUT FÜR WIRTSCHAFT, ARBEIT UND KULTUR ZENTRUM DER GOETHE-UNIVERSITÄT FRANKFURT/M. (IWAK) (Hrsg.): Evaluierung des Berufsorientierungsprogramms in überbetrieblichen und vergleichbaren Berufsbildungsstätten. Evaluationsbericht. Offenbach/Frankfurt/M. 2010
MINISTERIUM FÜR ARBEIT, GESUNDHEIT UND SOZIALES DES LANDES NRW (MAGS) (Hrsg.): Kein Abschluss ohne Anschluss – Übergang Schule – Beruf in NRW. Zusammenstellung der Instrumente und Angebote. Düsseldorf 2018
RATSCHINSKI, G. u. a.: Endbericht Evaluation des BMBF-Programms zur »Förderung der Berufsorientierung in überbetrieblichen und vergleichbaren Berufsbildungsstätten«. Berlin 2017
Research Associate in the Vocational Orientation Programme Office at BIBB
Head of the Vocational Orientation Programme Office at BIBB
Translation from the German original (published in BWP 5/2018): Martin Kelsey, GlobalSprachTeam, Berlin