Bundesinstitut für Berufsbildung (BIBB)

Methods for selecting trainees

Young people differ from one another in terms of their occupational interests and abilities. At the same time, occupations and jobs also differ with respect to the requirements and possibilities they entail. In light of this, it is wise 0 with a view to training successful future employees and avoiding trainees dropping out 0 to select 'suitable' applicants whose abilities are in line with the demands that will be placed on them in the particular occupation. For this, companies that provide in-house vocational training ('training companies') have a wide range of diagnostic tools at their disposal to determine applicant aptitude and suitability. As shown by a company survey conducted with 1,228 enterprises in 1999 regarding trainee selection methods, nearly all companies carry out a preliminary screening of applicants on the basis of application documents and school certificates. This is almost always followed by personal interviews with the individual applicants whereby one third of the companies surveyed conduct tests (see PRUSIK 2003).

However, using diagnostic methods to determine a candidate's suitability makes sense only when it can be proven that this would substantially increase the likelihood of finding suitable applicants. This in turn requires studies in which statistical correlations between the results from selection procedures (predictors) and performance measures (criteria) are determined on the basis of a larger sample. Only in this way is it possible to assess how reliably later occupational performance can be predicted on the basis of selection procedures. The grades recorded on the individual's vocational school certificate or earned on the interim or final examination conducted by the competent body are often used in the research field and in actual recruitment practice as criteria for predicting whether the individual will complete initial vocational training and pass the final examination. The following report examines the question whether youths who earned better grades in secondary school and performed better on recruitment tests actually turn out to be the more successful trainees as measured by the grades they earn on their interim and final examinations. For this, it was possible to draw on a data set which contains usable data from some 400 trainees. This data covers the youths' school grades, the results of recruitment tests that are geared to the particular occupation and the grades subsequently earned on the interim and final examinations for the individual's initial vocational training. Using this information, it is possible to make statements regarding the practical value of school grades and recruitment tests for predicting an individual's later performance during vocational training.

Top of the page

Previous findings regarding the prediction of success in vocational training

In their meta-analysis, BARON-BOLDT/SCHULER/FUNKE (1988) brought together previous studies on the reliability of school grades for predicting an individual's later success in vocational training. This work analysed 13 studies with a total sample size of 2,064 cases and examined the final school grades of school leavers from lower and intermediate secondary schools to determine their reliability in predicting grades on final examinations in vocational training. The authors determined a correlation of r=.293 between the average school grade and the final examination for initial vocational training and established that youths with good average school grades also tended to earn better grades on their final examination. Their analysis did not reveal any marked differences between the individual 'training occupations' (recognised occupations which require completion of formal vocational training). At the same time, the average school grade had a greater predictive power than the individual subject grades. Looking at subject grades, the grade for Mathematics was the best predictor of the individual's grades on the final examination. Furthermore the average grade had a stronger correlation with the theoretical part of the final examination than with the practical part of the final examination.
 
There have been scant findings to date regarding how reliable recruitment tests are for predicting an individual's success in vocational training. Using a sample of 91 trainees for the occupations Biology Laboratory Assistant and Chemical Laboratory Assistant, FUNKE (1986) examined the correlation between success in vocational training and the results of a battery of recruitment tests for numerical and figural skills and likewise determined a moderate positive correlation.

On the whole however, the data base for predicting the performance of youths in Germany's 'dual' vocational training system (which combines part-time vocational schooling with practical work experience) is quite sparse. For this reason, further empirical studies are urgently needed.

Top of the page

Information regarding the data set

The data set used as the basis for this report was compiled through a collaborative undertaking with the University of Stuttgart and the U-Form-Verlag publishing house. It contains information on trainees in vocational training programmes for various commercial and industrial/technical training occupations such as Industrial Clerk, Milling Machine Operator and Bank Clerk. However, complete information on all predictors and criteria is available for only a part of the set 0 some 200 to 400 youths. Table 1 shows the descriptive values of the predictors and criteria used.

Table 1  Predictors and criteria

Predictors	Average	Standard deviation	n
Average grade in school	2.69	0.58	1,123
Grade in Mathematics	2.82	0.90	1,061
Grade in German	3.0	0.72	1,072
Grade in English	3.06	0.82	887
Score on the recruitment test (in %)	68.9	13.72	1,547
Criteria
Grade on the interim examination	2.57	0.796	472
Grade on the final examination	2.5	0.729	367
Grade on written part of the final examination	2.7	0.872	340
Grade on practical part of the final examination	2.35	0.761	263

The average grade recorded on the leaving certificate issued by the individual's general secondary school was calculated either on the basis of all subject grades or 0 when these were not available 0 the grades from the three main subjects German, Mathematics and English. The results from the recruitment tests used are available in the form of percentage values for the number of correct answers. On average, candidates answered two-thirds of the questions on the recruitment test correctly. The grades from the interim examination, the overall grade from the final examination and the grades from the written and practical parts of the final examination were used as criteria for assessing the individual's success in vocational training.

The recruitment tests used here are tailored to the respective occupation and were developed by U-Form-Verlag especially for the purpose of trainee selection. These tests consist of three to five subtests with questions and assignments from the areas: general education/economics, numbers/tables, mathemathics, spelling, German, English, comprehension of technical tests, technical thinking, and interest in technical matters. Different subtests are conducted, depending on the targeted occupation. The level of difficulty of some of these subtests varies.

Top of the page

Predictive power of school grades and recruitment tests

Correlations were calculated to determine the importance of school grades and recruitment tests in predicting an individual's performance on interim and final examinations. It should be noted in this connection that a negative correlation between the results of the recruitment test and the grades earned on an interim examination or final examination actually indicate a positive correlation because in the case of school grades, a lower numerical value stands for good performance and in the case of exams, a good result is expressed in high point values.

Table 2 Correlations between predictors and criteria

	Average school grade	Grade in Mathematics	Grade in German	Grade in English	Recruitment test
Grade on interim examination	.412** (352)	.237** (310)	.122* (318)	.183** (289)	-.266** (402)
Grade on final examination	.442** (264)	.370** (205)	.201** (213)	.291** (186)	-.315** (302)
Grade on written part of final examination	.468** (236)	.355** (189)	.218** (196)	.288** (169)	-.312** (280)
Grade on practical part of final exam	.437** (222)	.330** (182)	.218** (189)	.296** (162)	-.143* (205)

** The correlation is significant at the level of 0.01 (one-tailed).
* The correlation is significant at the level of 0.05 (one-tailed).
The size of the respective sample is shown in parentheses.

Looking at Table 2, all of the correlations between school grades, the results of the recruitment test and the grades earned on the interim and final examinations are statistically significant. The closest correlations can be observed between the average school grade and the different criteria. Here, the strongest correlation of r=.468 was determined for the grade on the written part of the final examination. The individual grades in the main subjects Mathematics, German and English also exhibit significant correlations with the criteria. These correlations do not however reach the level of correlation seen for the average grade. At the same time, the individual grades consistently predict the results of the final examination better than they predict the results of the interim examination. This could be due to the fact that in some training occupations little value is attached to the interim examination and youths do not put forth a maximum effort on their interim exam. As in the meta-analysis by BARON-BOLDT/ SCHULER/FUNKE (1988), the grade in Mathematics is the best individual predictor for a trainee's subsequent performance on the examinations conducted during initial vocational training. In contrast to the findings from the meta-analysis however, this study did not find any distinct differences in the predictors' correlations with the written part of the final examination compared to their correlations with the practical part of the final exam. So in the data set used in this study, school grades predicted the two sections of the final examination equally well. The correlations between the recruitment test and the criteria for success were not as strong as they were between the average school grade and the success criteria. A slight difference in the correlation with the written part and the practical part of the final examination could be seen, with the correlation with the practical part of the final exam being markedly weaker.

The data set contained the grades not only from secondary school pupils with qualification to enter a university / university of applied sciences but also the grades of secondary school pupils who have a leaving certificate from an intermediate secondary school. Consequently, final grades must be seen in relation to the type of secondary school leaving certificate earned. In keeping with this, the correlations listed in Table 3 are broken down by the type of leaving certificate. The number of youths with a leaving certificate from a lower secondary school was too small in this data set for a separate analysis.

Table 3  Correlations between predictors and criteria, broken down by level of education

	Average school grade		Grade in Mathematics		Grade in German		Grade in English		Recruitment test
Level of education completed	Qualification to enter a university of applied sciences	Intermediate secondary school	Qualification to enter a university of applied sciences	Intermediate secondary school	Qualification to enter a university of applied sciences	Intermediate secondary school	Qualification to enter a university of applied sciences	Intermediate secondary school	Qualification to enter a university of applied sciences	Intermediate secondary school
Grade on interim exam	.358** (183)	.358** (183)	.208** (163)	.207** (128)	.131* (163)	.106 (136)	.248** (157)	.058 (116)	-.203** (225)	.386** (124)
Grade on final exam	.426** (159)	.470** (94)	.365** (138)	.388** (66)	.184* (139)	.253* (73)	.277** (133)	.289* (52)	-.257** (193)	.387** (69)
Grade on written part of final exam	.443** (141)	.488** (84)	.365** (131)	.352** (57)	.231** (131)	.226* (64)	.276** (125)	.294* (43)	-.285** (176)	-.320** (66)
Grade on practical part of final exam	.455** (127)	.440** (84)	.332** (123)	.335** (58)	.195* (123)	.301** (65)	.275** (117)	.382** (44)	-.085 (122)	-.220 (45)

 * The correlation is significant at the level of 0.05 (one-tailed).
** The correlation is significant at the level of 0.01 (one-tailed).
The size of the respective sample is shown in parentheses.

As Table 3 shows, some predictors exhibit marked differences depending on the type of school leaving certificate. In the case of individuals with a leaving certificate from an intermediate secondary school, the results of the recruitment test exhibit stronger correlations with the grades from both the interim examination and the final examination than they do for individuals who have earned qualification to enrol in a university / university of applied sciences. The average grade earned at school (average school grade) by youths with a school leaving certificate from an intermediate secondary school is also somewhat better at predicting how well the individual will satisfy the criteria for the success of vocational training. The degree of the correlations between the grades for the individual subjects and the criteria is for the most part comparable, with the exception of the final grades earned by intermediate secondary school pupils in the language subjects, which exhibit noticeably stronger correlations with the practical final examination than in the case of upper secondary school leavers who had earned qualification to enrol in a university (or university of applied sciences). These different correlations could conceivably be simply due to sampling errors; however social and communication skills 0 which are particularly important in the practical final examination 0 perhaps count more in the final grades in the language subjects taught in intermediate secondary school.

As the results listed in Table 2 and Table 3 show, later performance on vocational training examinations can be predicted not only on the basis of the recruitment test but also on the basis of the final grades earned in secondary school. Here, the average grade from secondary school has the most predictive power for upper secondary school leavers with qualification to enrol in a university (or university of applied sciences) as well as for pupils with a school leaving certificate from an intermediate secondary school. This raises the question whether recruitment tests provide enterprises information that supplements the average grade earned in school and offers an additional means for ensuring that they choose appropriate applicants. To test this, hierarchical regressions were calculated for the grade earned on the interim examination and on the final examination. In the first step of this calculation, only the average grade earned in school was included. In a further step the calculation used the average grade together with the results from the recruitment test. As can be seen in Table 4, R2 =17.8 per cent (p<.01) of the variance of the grade from the interim exam can be explained by the average grade earned in secondary school. A further 6.1 percentage points of the variance can be explained through the addition of the recruitment test which improves the reliability of the prediction. The situation is similar with the prediction of the grade on the final examination (Table 5). In this case, R2 =16.1 per cent (p<.01) of the variance can be explained by the average school grade alone. Adding the recruitment test increases this by a further 7.6 percentage points. Therefore, in both cases the recruitment test can contribute to a somewhat greater degree of certainty regarding the soundness of recruitment decisions. 

Table 4 Hierarchical regression for the prediction of the grade on the interim examination

Top of the page

Multimodal applicant selection a sensible approach

The results from our study are largely consistent with the findings from earlier research projects. Thus an individual's success in vocational training (as measured by the grade earned on the interim examination and the final examination) can be best predicted using the average grades earned in secondary school.
It is not surprising that the average grade is a better means of predicting success in vocational training than the grades for the individual subjects, since an average grade reflects the assessment of several teachers over a longer period and should consequently provide a comprehensive picture of the individual pupil's performance. As a result, the average grade probably reflects not only academic performance but also general aspects of the individual's work conduct such as diligence, motivation and level of perception.

In contrast to previous findings, our data does not reveal any marked differences between the accuracy of the predictions of the written part of the final examination and the accuracy of the predictions of the practical part of the final examination. Only the recruitment test exhibits significant correlations, albeit only with the written part of the final examination and not with the practical part. This can be explained by the fact that additional competences which are not part of the recruitment test play an important role in the practical examination. As the results of the regressions show, a combination of the average school grade and a recruitment test can improve the accuracy of the prediction of an individual's performance in vocational training.

Based on these findings, it cannot be disputed that the use of grades from the individual's school leaving certificate have a certain relevance in the selection of training place applicants.

The inclusion of recruitment tests which entail specific technical/occupation-related content that is of relevance to the particular occupation can also be considered useful based on the findings presented here since these tests further improve the reliability of predictions of an individual's training performance compared to the use of just the average grade. However, this increase in the degree of reliability is quite small. For these reasons it would be desirable to follow up the use of recruitment tests with an empirical evaluation to examine whether it is possible to predict later performance or even predict other criteria for successful performance in vocational training and/or in working life 0 such as job satisfaction 0 using such tests.

Data concerning training place applicants who were rejected by a company and their possible subsequent training performance in another enterprise is not available. This lack of data must be viewed as a qualification in terms of the interpretability of the data. Furthermore, the findings permit statements at group level only and under no circumstance for individual youths. Consequently it cannot be generally concluded that all youths who earned poor grades in general secondary school can be expected to be poor performers in vocational training. Therefore based on these findings, it is not possible to make a general recommendation to reject youths with poorer school records when they apply for a training place. Rather, these youths should particularly be asked in a personal interview what their interests and motivation are because empirical studies (see, for example, NICKOLAUS et al. 2010; METZNER/FRINTRUP 2005) have shown the importance that interest and motivation (to achieve) have for later performance during vocational training. It would also be advisable during vocational training to tap into the trainees' interest in the occupation they are learning, instil in them enjoyment of their work through a wide variety of tasks and collegiality and, in doing so, foster and maintain their intrinsic motivation on a long-term basis.
 

Top of the page

Literature

BARON-BOLDT, J.; SCHULER, H.; FUNKE, U.:
Prädiktive Validität von Schulabschlußnoten: Eine Metaanalyse. In: Zeitschrift für Pädagogische Psychologie, 2 (1988) 2, pp. 79-90
FUNKE, U.:
Die Validität verschiedender eignungsdiagnostischer Verfahren bei Lehrstellenbewerbern. In: Zeitschrift für Arbeits- und Organisationspsychologie, 30 (1986) 2, pp. 92-97   METZNER, F.; FRINTRUP, A.:  Erfolgsprognose von der Stange: Ausbildung im Schatten von Pisa. In: Personalwirtschaft: Magazin für Human Resources, (2005) 8, pp. 17-20
NICKOLAUS, R. u. a: Erklärungsmodelle zur Kompetenz- und Motivationsentwicklung bei Bankkaufleuten, Kfz-Mechatronikern und Elektronikern. In: SEIFRIED, J.; WUTTKE, E.;   NICKOLAUS, R.: Lehr-Lernforschung in der kaufmännischen Berufsbildung. Ergebnisse und Gestaltungsaufgaben. Stuttgart 2010, pp. 73-87
PRUSIK, C.: Implizite außerfachliche Auswahlkriterien von Betrieben bei der Einstellung von Auszubildenden. Neuhofen 2003

Top of the page

Last modified on: December 8, 2011