Bundesinstitut für Berufsbildung (BIBB)

"Education controlling incorporates all planning steps, starting with a needs analysis and extending to the implementation of measures all the way to measuring results, and activities to ensure that what has been learned is transferred to day-to-day practice" (WEIß 2005, p. 33). The term 'Controlling' is in Germany often translated with 'test' and, even more often, with 'check'. However the English term 'to control' means 'to steer something' or 'to guide something'. In light of this, one has to agree with BÖTEL/KREKEL: "Education controlling should not be understood as checking continuing education" (1999, p. 5). In fact, education controlling entails the "cycle-based meshing of planning, measurement, evaluation and correction in the continuing education and training field" (see BEICHT/KREKEL 1999). The following diagram illustrates this concept:

Education controlling is a concept that provides a flow and phase model for managing continuing vocational training (see Diagram 1). At the same time, day-to-day practice in the respective organisation is much more complex and follows this model only in part or scarcely at all in many cases: "Management cannot be described as a mechanism. Rather, it is an objective-driven approach that is inherent to action and which manifests itself during action in a wide variety of forms that do not exhibit a pattern or follow a template." (ROBAK 2004, p. 341).

This indeterminateness is not a sign for deficits on the part of the bodies and persons responsible for education and training. Rather, it is an indication of the complexity and flexibility of the work that takes place in organisations - work that often does not follow the management concepts found in textbooks. Today, this work is conducted in highly intermeshed contexts that are dependent on external developments. ROBAK (2004) speaks of "activity honeycombs" which not only provide a frame for programme planning but at the same time are also generated by programme planning. The following examples from enterprises were gathered in the course of various BIBB projects and are provided here to illustrate this ⁰¹:

• Enterprise A, a supplier to the automobile industry, conducted an in-depth training needs analysis on soft skills with the aim of boosting the culture of cooperation in the company. Plans are developed, preliminary talks held, providers chosen, etc. Just before the first workshop is to be conducted, new machines are installed in the company and new products are manufactured because of a sudden change in an external customer's requirements. The cost-intensive training on the new equipment is conducted in improvised crash courses. The workshops on soft skills are postponed indefinitely.
  
• Enterprise B budgets a specific amount every year for continuing training. This amount is based on the company's financial situation. The training department has to work with this budget, irrespective of any findings from needs analyses and regardless of how large or small the training needs are among the company's workforce that year.
  
• Due to an acute deterioration in the general economic situation, management at enterprise C decides to eliminate all training that is conducted outside the company. As a result, the persons responsible for training at enterprise C have to develop at short notice a new plan for the selection of nearly all providers and instruments and for conducting the training. Improvising, they look for and find solutions in order to be able to conduct at least part of the continuing training at enterprise C themselves or have it conducted in their branches by internal or external trainers.

These examples illustrate the complexity of continuing vocational training. In the wake of increased cost awareness, the persons in charge of training in the respective enterprise must make efficient use of the company's resources and justify spending on continuing training (see BEICHT/KREKEL 1999). When it comes to planning, conducting and evaluating continuing vocational training, they are extremely dependent on the overall economic situation in their enterprise, on external customers/cooperation partners, management decisions, and rivalries between different managers or departments. Education controlling's phase model offers an important point of reference for education management and programme planning. Due to various interdependencies and short-term disruptive factors, the persons in charge of training must be able to adapt their plans on a flexible basis and shouldn't follow projections unwaveringly.

Education controlling must be supplemented by professional action on the part of the training managers in the particular enterprise to ensure that it is possible to deal competently with unforeseen developments. Deviations from education controlling's phase model are not necessarily a sign that professionalism is lacking. Indeed, they can represent a professional, flexible response to a company's changeable realities. Professionalism manifests itself in how new requirements and situations are handled and in the finding of practicable solutions. This occupational competence on the part of training managers - which phase models cannot fully reflect - has not been the focus of much scientific study to date (see ROBAK 2004).

Education controlling was the subject of numerous publications in the late 1990s. It was a catchword that could be found in nearly every article and paper on continuing vocational training during those years. This concept and the instruments associated with it were the source of great expectations - and many fears. The number of publications on this subject peaked around the year 2000 and interest declined noticeably in the years thereafter (see PRÜSTEL/WOLL 2008, KÄPPLINGER 2009). But how do things look with continuing vocational training practice today? Does the waning interest in research in this area reflect a decline in the importance of education controlling in day-to-day practice? Or is education controlling much more widespread today as the result of the often proclaimed economisation of continuing education and increased cost awareness? A comparison of the 1/1997 RBS survey with the 1/2008 RBS survey does not confirm the assumption that education controlling has become less important or the expectation that this concept is being used much more widely today. The findings signal a remarkably high degree of continuity in day-to-day Vocational Education and Training practice (see Chart 1).

The extensive use of education controlling increased by nearly two percentage points between 1997 and 2007. By comparison, the use of elements of education controlling and non-use of education controlling together fell by nearly two percentage points. Thus at first glance the shifts that took place between 1997 and 2007 appear to be only marginal. Approximately one out of every two enterprises in Germany uses elements of education controlling or makes intensive use of education controlling. However, based on the information provided by the companies surveyed, we do not yet know what enterprises understand by education controlling or how they actually practise it.

It must be remembered when examining the latest RBS survey that 56.7% of the enterprises surveyed reported that they had been active in continuing vocational training in 2007.⁰² Out of this group, 14.3% make intensive use of education controlling and 59% use at least elements of it. Thus some three-quarters of all firms that are involved in continuing vocational training use education controlling at least in part. Only 5.3% of those enterprises that are not involved in continuing vocational training said that they use some elements of education controlling at most. In such cases, the individual enterprise may, for example, have conducted a training needs analysis and feels it has no need for training and consequently does not provide continuing vocational training. An enterprise that provides little or no continuing training correspondingly does not make use of education controlling as a rule. Thus, education controlling goes hand-in-hand with the degree to which an enterprise is involved or not involved in continuing training. This was probably also the case in 1997 although the survey back then did not ask about the individual company's continuing training activities.

Similarly to the general level of involvement in continuing training, the use of education controlling depends greatly on the size of the respective enterprise (see Chart 2, page 4).

The share of large companies that use education controlling is noticeably larger than the corresponding share of small enterprises. Large companies also make more intensive use of education controlling. For example, 32.3% of large companies with 500 or more employees make intensive use of education controlling and 52.5% use elements of it. By comparison, these figures are only 12.1% and 32.4% among small enterprises with fewer than ten employees. Thus, 84.8% of large companies use education controlling while only 44.5% of small enterprises do. Looking at medium-sized enterprises, these figures are 70.1% (50-499 employees) and 42.5% (10-49 employees). This is understandable since few small firms can afford to use education controlling on a continual and comprehensive basis. The reason: Education controlling requires personnel and financial resources that few small firms have available internally due to their size. At most, such firms can purchase education controlling services through external cooperation partners such as continuing training centres or management consultancies.

Although small and very small enterprises are less active in continuing training, nearly one out of every two of these companies report that they use education controlling in part (32.4%) or extensively (12.1%). Thus, although education controlling appears to be particularly important for continuing vocational training in medium-sized and large companies, quite a few smaller firms indicate that they also make use of it. Consequently we must guard against associating education controlling only with large companies. Instead, education controlling should be differentiated in conceptual terms based on company size in order to do justice to the differences in the conditions in the respective enterprises.

A comparison of the use of education controlling in 1997 and in 2007, broken down by company size, reveals noticeable changes:

The most striking change on balance was that education controlling has become much more widespread among enterprises with a medium-sized or large workforce (see Chart 3). For example, extensive use increased by 21.7 percentage points among large companies between 1997 and 2007. A marked increase of 10.9 percentage points was also observed among companies with 50 to 499 employees. By contrast, use of education controlling by small enterprises (10-49 employees) declined slightly (-3.5 percentage points).

Opinions are apparently divided on education controlling. Depending on whether one is examining large companies or small enterprises, education controlling has either become more important or its importance remains unchanged. Large companies with more than 500 employees and medium-sized enterprises used education controlling much more in 2007 than in 1997. By contrast, its use has declined slightly among small enterprises. Three times more large companies than small enterprises reported that they use education controlling "extensively". Many small enterprises are able to apply just some elements of education controlling at most (see Chart 2). Reasons for this include practical (cost) considerations.

This raises the research-and-development (R&D) question of whether and how lean, selective and pin-pointed education controlling could be designed for small and medium-sized enterprises in particular and whether and how it could be linked with external training Counselling services (see LÖFFELMANN 2008). Plans for education controlling have perhaps depended too much to date on the entire controlling cycle being conducted according to the textbook - something that is sensible and understandable in theory but probably does not dovetail well with the demands of most smaller enterprises. For example, learning that takes place in the workplace is very important in many small enterprises.

The discussion over trends in continuing in-company vocational training has often attributed great importance to forms of learning that take place in the workplace and are not organised as courses (see European Commission 2000, p. 9 f.). For example, it is argued that in-company courses and seminars are edged out by forms of learning that take place in the workplace because the latter hold the promise that what is learned will be put to direct use in the work process. Empirical studies indicate however that learning that takes place in courses and learning that takes place in the workplace tend not to be in competition with one another but rather are complementary to one another (see, for example, BEHRINGER/DESCAMPS 2009).

The 1997 RBS survey did not cover forms of learning that take place in the workplace. The 2008 RBS survey on the other hand asked two questions about such forms of learning for the first time in order to ascertain whether firms apply their education controlling system to this area as well. These questions were modelled after the CVTS survey: "Are forms of learning in the workplace that do not involve courses (informational events, job rotation, self-directed learning) included in your education controlling activities?" and "What has your experience been with this?" The findings draw an ambivalent picture (see Chart 4).

One-third of all enterprises has not gathered any experience to date in applying education controlling to different forms of learning in the workplace. A total of 16.6% of all companies think that this type of controlling would be easier and 26.7% say it would be just as easy to conduct as controlling for forms of learning that take place in courses. It might be something of a surprise that 43.4% of the enterprises surveyed came to a positive or neutral assessment, given that continuing education researchers point to considerable methodological problems in connection with assessing learning that takes place in the workplace (GRÜNEWALD et al. 1998). There is the possibility that education practitioners take a pragmatic approach to education controlling for forms of learning that do not take place in courses and therefore occupy themselves less with methodological problems. Thus, 'only' 23.6% report negative experiences because they consider learning that takes place in the workplace to be more difficult to assess (14.8%) or as impossible to assess (8.8%).

All in all, it appears that education controlling is being applied to more than courses; it is also frequently used for forms of learning that take place in the workplace. This is an important finding when one considers that research literature has not directed intensive attention to this subject to date (see PRÜSTEL/WOLL 2008). The question of how enterprises could specifically apply education controlling to learning in the workplace would be an interesting question for more in-depth interviews. And it would be interesting to follow how education controlling could be set up at conceptual level for forms of learning that take place in the workplace.

Enterprises were also asked in both 1997 and 2008 about those elements of education controlling that they used "regularly", "in part" or "not at all". This question drew on the concept developed by BEICHT/KREKEL (1999, pp. 46-47) which assigned a total of 15 controlling elements to three sub-categories:

a) Identification of objectives / Planning

The respective enterprise:

• Ascertains continuing vocational training needs.
• Draws up an annual continuing vocational training plan.
• Brings its continuing training objectives systematically into line with its strategic objectives of the whole enterprise.
• Coordinates continuing vocational training objectives with the providers responsible for the particular training measure / with the instructor.
• Coordinates continuing vocational training objectives with participants and systematically selects external continuing training providers.

b) Evaluation

• Participants assess the respective measure while it is being conducted.
• Participants orally assess the measure upon its completion.
• Participants assess the measure in writing upon its completion.
• The company measures the extent to which the measure achieved the targeted objectives and assesses the individual participant's job performance following the measure.

c)  Cost recording / Assessment of benefits / Knowledge transfer

The respective enterprise:

• Conducts a subjective evaluation of the benefits of the continuing vocational training measures.
• Assesses the benefits on the basis of objective criteria.
• Records the continuing training costs.
• Implements measures to ensure that the knowledge gained during the training is transferred to day-to-day practice.

An examination of the 'regular' use of education controlling yields the following findings for these sub-elements of education controlling (listed by frequency, see Chart 5):

The recording of continuing training costs is the most frequently-cited element of education controlling. Its use increased by 7.6 percentage points between 1997 and 2007. By contrast, at 24.9% and 13.7%, subjective and objective assessments of the benefits of continuing training are conducted noticeably less often. As a result, the gap between the recording of costs and the recording of benefits has grown since 1997 (see BEICHT/KREKEL 1999, p. 51). The objective assessment of the benefits declined by 13.5 percentage points and the recording of costs increased by 7.6 percentage points. This was particularly the case for large companies (see KÄPPLINGER 2009). Surprisingly, small enterprises show greater interest in a good cost-benefit balance than large companies do (ibid.). According to BEICHT/KREKEL (1999, p. 51), this is because the benefits of continuing vocational training can be observed better and more directly in small enterprises than in large companies. The different 'configurations' of the respective social partners could offer another explanation. With an eye to possibility that corresponding entries could be made in the individual's personnel file, personnel representatives in large companies are often reserved when it comes to employers documenting the benefits of training or using a 'skills balance sheet'. By contrast, many small enterprises do not have any personnel representatives and dealings between employer and employee are more personal. One could also allow the interpretation that continuing training is negotiated differently in large companies. Many large enterprises have an annual continuing training budget that is a standing budget item. Although the continuing training budget might be increased or reduced, the way it is distributed among the departments or employees is an important focus of internal negotiations. On the other hand, in small enterprises decisions on continuing training are reached more directly between the proprietor or managing director and the workforce. It is likely in this connection that cost-benefit questions arise directly and in concrete terms.

It is also striking that assessing the training measure while it is still in progress and upon its completion has become more important (10 percentage points in both cases). The use of 'happy sheets' (training evaluation forms to be completed at the end of a course) has become more widespread. This is not very surprising given that they are easy to use. It is however somewhat surprising that measures for ensuring the transfer of knowledge to day-to-day practice and measuring the extent to which objectives were achieved are to be found at the lower end of the scale and are declining in importance. It is possible that enterprises baulk at the methodological problems or the cost and effort involved. Looking at employee participation, it is good news that the three areas associated with this ("Participants assess the measure", "Company coordinates the measure's objectives with the participants" and "Participants orally assess the measure upon its completion") ranked third, fourth and fifth with 31.7% to 34.8%. Apparently, rather than going over their employees' heads, most enterprises are actively incorporating their employees into their education controlling activities.

A closer examination of these average values also reveals considerable differences based on company size. For example, marked differences can be seen when firms with 500 or more employees are compared with very small enterprises with fewer than ten employees. Large companies use most of the elements of education controlling much more frequently than very small enterprises do. For example, 76.1% of large companies survey their employees in writing following a continuing training measure. This figure is only 8.1% for very small enterprises. Similarly large gaps of 50 percentage points or more can be observed in areas such as "Draws up an annual continuing vocational training plan" and "Brings continuing training objectives into line with corporate objectives". However some elements of education controlling are more widespread among very small enterprises than among large companies. This is the case with "Measures whether and to what extent the measure achieved the targeted objectives" (7.7% in very small enterprises vs. 1.3% in large companies), "Company assesses the participant's job performance" (25.2% vs. 10.0%), "Company subjectively assesses the benefits" (28.2% vs. 23.7%) and "Company assesses the benefits on the basis of objective criteria" (16.1% vs. 9.5%). It is clear however that small enterprises are particularly interested in the benefits of continuing training and concentrate their education controlling activities on this aspect.

Looking at the year 2007, BIBB asked the RBS reference companies how important they felt education controlling would be in the future (see Chart 6). The majority of the companies surveyed (91.1%) assumed that education controlling will be just as important in the future (56.1%) or even more important in the future (34.9%):

When asked which factors are key to good continuing training, 20.5% of the enterprises surveyed said that the existence of an education controller constitutes such a factor. In addition to this specialised, 'division-of-labour' approach which is illustrated by the position of education controller, it can be assumed that many companies understand education controlling more as an integrative process task to be performed by different parts of their organisation (immediate superior, personnel department, etc.). This shows that the use of education controlling to organise continuing training work is of considerable importance, at least in some parts of the economy (and particularly in large companies). Should the trend seen over the last ten years continue, education controlling could become standard among large companies. Only 8.9% of all enterprises think that education controlling will become less important in the coming years. Thus, education controlling is definitely a topical subject when one looks at day-to-day education practice. One should not be misled by the relatively small number of publications on this subject in recent years.

All in all, education controlling has become slightly more widespread since 1997. Approximately one out of every two German enterprises reports using at least some elements of education controlling. The pattern seen here: The larger the company, the greater the likelihood that it uses education controlling in part or on a regular basis. However, many small enterprises also make use of elements of education controlling and are, for example, more interested in documenting the benefits of continuing training than large companies are. Large enterprises and small enterprises differ greatly in how they practice education controlling. There are indications for a need to develop specific, selective and segmented controlling elements for small enterprises. To date, controlling literature has understood education controlling to be a regular, across-the-board procedure. A lean and easy-to-practise version of education controlling that is purposely incomplete has yet to be developed for small and very small enterprises.

Overall, enterprises feel that education controlling will gain in importance in the years ahead. It is therefore surprising that the number of relevant publications has declined in recent years. Given the growing gap between recording the costs of continuing training and recording its benefits, it would appear advisable to introduce a "fresh wind to the perception of benefits and the management of benefits" (see von LANDSBERG/WEIß 1995, p. 3; QUENZLER 2008) because continuing training will otherwise be seen primarily as a cost factor and not sufficiently as something that provides benefits. Looking at continuing training involvement, this could lead to a decline in the amount of continuing vocational training that enterprises make available.