Employment effects for people with disabilities after participation in vocational training programmes: A cohort analysis using propensity score matching

1Introduction

People with disabilities are highly disadvantaged in terms of labour force participation compared to people without disabilities. Besides the positive (economic and health-related) effects of employment for people with disabilities, like e.g. social security contributions, feelings of belonging and societal purpose [1, 2], there is additional evidence for organisations that hiring people with disabilities can have positive effects on, e.g. firm profitability, work culture and competitiveness [3].

In this context, vocational rehabilitation (VR) is designed to develop and restore the ability to work and increase employment prospects for people with disabilities in Germany. However, there are very few studies that confirm that VR improves employment chances [4, 5], though VR and especially vocational training programmes are very cost-intensive for the welfare state and time intensive for the participants.

1.1Study focus

In the following sections, we define our understanding of disability, describe the quite complex German system of VR and characterise vocational training programmes. In this context, we give some background information regarding our two comparison groups.

1.2VR process: Financers, eligibility and programme determination

In Germany, the (financial) responsibility for VR is shared by mainly three federal institutions: the Statutory Pension Insurance (SPI), the Statutory Accident Insurance and the Federal Employment Agency. The following analyses focused on the FEA, which is mainly responsible for people who have been in employment covered by social security contributions for less than 15 years and for others who cannot be financed by other institutions.

1.3Eligibility for VR and recognition procedure

Eligibility for VR is connected to the definition of disability found in Section 2 [1] of Book IX of the German Social Code:

“[...] a person has a disability if they have a physical, psychological, intellectual or sensory impairment that, in interaction with attitudinal and environmental barriers, is highly likely to impair their equal participation in society for longer than six months. A person has an impairment if their physical and health condition falls short of that typical for their age. A person is at risk of disability if such an impairment is to be expected.” (8, p. 36)

The medical or psychological service of the FEA first must assess the individual need for VR. These disability-specific needs are known as the VR status. The person granted VR status is called a vocational rehabilitant. Based on medical and psychological reports, a VR counsellor at the FEA - specialised in the placement of ill or disabled people - verifies whether a person is eligible to be granted VR status due to having a health issue and corresponding limitations that prevent him or her from practising their occupation and performing the tasks involved – whether they are currently employed or not [5, 9]. To be eligible for VR, it is not necessary to have disability status, though a person with disability status has a better chance of receiving VR status [9]. Participation in VR is voluntary for the person, but if they are eligible and willing to participate in VR, the VR counsellor is legally obliged to assert the VR status [9].

When the VR status has been confirmed by the VR counsellor, the most promising programme for labour market integration should be chosen, e.g. technical assistance, such as ramps or braille keyboards, programmes in sheltered workshops, supported employment, vocational orientation programmes and/or vocational training. Depending on the level of support required and the regional labour market situation, the VR counsellor makes decisions regarding additional disability-specific support within qualifying programmes, like vocational training programmes, and the type of service provider to be involved [7, 10].

1.4Non-participation in VR

The FEA supports unemployed with and without health issues. Unemployed without disabilities are counselled and placed by general departments of the FEA, persons with disabilities and chronic illnesses need to be directed to VR departments within the FEA. Therefore, in many cases, no application for VR may have been submitted because general case managers at the FEA were unaware that the client might be a case for the VR department [9, 11]. Furthermore, general case managers might have been aware of health limitations but did not assess the chances of the individual being accepted into VR as particularly high; for example health limitations might not be serious enough in that they comprehensively hinder employment participation or the client’s health might not be stable enough to participate in VR [9]. Besides, there is qualitative evidence for Germany suggesting that despite a disability or health impairment, some individuals refrain from applying for VR either to avoid stigma or income loss, or because suitable programmes are unavailable or inaccessible [11].

1.5Vocational training programmes

At the FEA vocational training programmes are widely used instruments to place unemployed persons with and without disabilities into the labour market. The strategic goal for unemployed with and without disabilities is somehow different. Unemployed without disabilities mainly seek improvement in their cultural capital – because their qualifications might be out-dated or too low and they might lack certain certificates that are required by the labour market [12].

While rehabilitants moreover require a new occupational perspective and a task that is compatible with their illness. Within VR, first, within assessment programmes rehabilitants’ strengths and interests are identified; second, vocational training programmes are implemented that are based on these assessments. Certificates prove the acquisition of new occupational skills and in-firm internships provide promoted with contacts to companies where labour market integration can be achieved [7].

In general, vocational training programmes for both groups can be differentiated into those that are available to all clients of the FEA (unemployed and rehabilitants) (“general programmes”) and those that are “disability-specific”, i.e. vocational training programmes intended solely for rehabilitants. Disability-specific training programmes often have the same curriculum as general training programmes but provide additional disability-specific support (e.g. psychological, socio-educational support; are organised at service providers that are specialised for persons with (specific) disabilities) [7].

Furthermore, there are different types of vocational training programmes with different learning objectives and differences in certification. Further training (certified and uncertified) refers to programmes to refresh existing basic and occupational skills or to adapt them to new technical developments, while occupational retraining encompasses all types of skill development in a completely new occupation. Retraining is often certified and can be compared to the three-year vocational training programme in the context of initial vocational education in Germany; however, it is usually reduced to about two years since the participants already have vocational skills to build on. Further training is considerably shorter and varies in duration, content and objective, but does not usually last longer than one year [7]. Both types of vocational training may be conducted by service providers specialised in training people with and without disabilities. The participants generally learn theoretical and practical skills at these institutions and undergo on-the-job training; on-the-job training is furthermore organised in the form of internships at companies in order to get in contact with potential employers. However, there are many different constellations for training arrangements, for example when the service provider is responsible for psychological and socio-educational support and the actual training is undertaken at a company in cooperation with a vocational school [10]. In the following, the different types of training are summarized as “training programmes”.

1.6Relevant structural developments in VR funded by the FEA

For the FEA context, there have been some changes in the population structure of VR in recent years that are worth mentioning. The numbers particularly refer to the comparison groups we have constructed in the context of our study design, enhance the practical relevance of the study and give the reader an idea of the population supported within VR.

2Current state of research

Concerning our first study aim to identify the employment effect of participation in training programmes, previous studies have shown that training programmes improve the employment prospects of displaced workers and the unemployed [12, 13]. However, there is no general understanding of how effective training programmes are for people with disabilities and why they are effective. Some studies have observed the employment transitions of people with disabilities after VR, but many of them focused on programmes that did not involve training, e.g. supported employment [14, 15], or did not identify a causal effect [7]. Furthermore, some studies from Sweden and Norway identify causal effects for VR and rank the effectiveness of specific programmes. Fröhlich et al. [16] for Sweden and Markussen [17] for Norway find more positive employment effects of workplace training than for vocational training that is not organised at a company or workplace. In this context, there are two studies for Germany that focus specifically on training programmes and a causal framework. One of these studies compared different training arrangements, in terms of duration and certification [18] and found that one-year training was superior to two-year training arrangements. No general effect of training was identified. In order to determine the effect of training participation compared to discontinued training, another recent study [4] identified a positive employment and income effect for those who completed their training. Despite the existence of these studies, they are difficult to compare to our study. First, the studies set a different focus with their comparison groups, as they compare participants in different types of training programmes and individuals dropping out of training to those completing the training programmes. Second, both studies focus on people whose VR was funded by the SPI. It is important to note that there are considerable differences between people in programmes funded by the FEA and those funded by the SPI: rehabilitants supported by the FEA are younger, better educated and are more likely to start VR following a period of unemployment. They are also more often diagnosed with psychological health issues and less frequently with musculoskeletal illnesses. Rehabilitants supported by the FEA are mainly allocated to vocational training programmes, whereas the SPI generally provides rehabilitants with technical assistance [19].

In contrast to those studies mentioned above, we focus on rehabilitants supported by the FEA and use rehabilitants who did not participate in any programme as a comparison group.

Furthermore, we add to the discussion of the VR effect itself. Concerning this second study aim, to our knowledge no study has yet investigated the effect of being granted VR status on employment outcomes (or other outcomes), as none compares participants in the same programme with different privileges as in our case. There are studies that evaluate VR and compare the employment effects of different programmes to non-participation in VR. Some of these studies find no differences [16] and some find no effect for men but cautious positive effects for women [20]; others find positive employment effects for granted VR status compared to those rejected/withdrawn from the VR application [5]. Due to differences in methodological approaches, data and sample restrictions, time frames and the rehabilitation systems surveyed, the results of these studies are not easy to compare with those obtained in our study.

3Material and methods

3.2Data

To achieve the two study aims, we used representative VR-specific administrative data (German Reha-Process Data Panel (RehaPro)) which allowed to evaluate various VR programmes. RehaPro was established within a large research project financed by the German Federal Ministry of Labour and Social Affairs [22, 23]. RehaPro combined different sources from the different FEA-registers (employment register, unemployment register, VR register) and contained information on time spent in (un-)employment, in long-term illness, and participating in labour market programmes and included specific information on the VR process. The data set comprised all individuals who applied to the FEA for VR between 2006 and 2015 [24].

3.3Comparison groups

The choice of comparison groups is one crucial part of every evaluation, influencing the focus of the analysis and the interpretation of the results. We used two comparison groups as shown in Fig. 1.

Fig.1

Overview of observed comparison groups.

In the first comparison group we compared rehabilitants participating in general or disability-specific training programmes to rehabilitants not participating in any programme. The treatment and comparison groups, each comprising people who were granted VR status between 2009 and 2010, differed with regard to their programme participation. We were therefore able to identify the effect of programme participation.

With the second comparison group, participants in general training programmes (completed between 2012 and 2013) with and without VR status were compared. To form the second comparison group, we added persons who were registered with the FEA as unemployed and were participating in the same training programmes as people with VR status by linking the RehaPro data set with the “general” administrative records of the FEA, the Integrated Employment Biographies (IEB) [25]. Thus, individuals who had applied for VR and been granted VR status were compared with unemployed people who had not applied for VR (non-rehabilitants) but participated in the same programme. In order to include some health characteristics for the control group in comparison II, those without VR status were restricted to those that were marked by the general case manager as person with “health conditions relevant for placement”. As explained in section 1, there is reason to believe that some people with such health conditions may be entitled to VR but lack information about entitlement or are unwilling to participate in VR or that general case managers do not assess their chances of acceptance into VR to be particularly high. In fact, these people with registered health conditions have never applied for VR. This second comparison identifies the effect of the VR status on employment participation.

3.4Sample restrictions

With respect to both comparison groups, we only considered individuals aged between 25 and 40 for two reasons: first, to avoid individuals for whom disability pensions are an alternative option after job loss, and second, because the average age of the second comparison group was significantly higher. To harmonise the two samples and ensure homogeneity to a certain extent, we also excluded individuals who were employed in a job covered by social security when participating in VR or a training programme. This restriction is necessary in order to ensure that we do not observe rehabilitants who were not taking part in programmes because they had returned to their previous employer. This would bias the results. Furthermore, we restricted the analysis and both comparison groups to those that had finished the training programme successfully and did not drop-out earlier.

3.5Outcomes

We used different outcomes to measure labour force participation. In the first comparison group (effect of programme participation), we distinguished between two types of employment: unsubsidised (regular employment) and marginal employment (part-time employment with a salary not exceeding 450 euro per month). Second, we compared the number of days worked (per year) and the mean yearly income (both from paid work and social benefits) for participants and non-participants. The outcomes were measured one, two, three and four years after the individual received VR status.

The same outcome variables were used in the second comparison (effect of VR status). Furthermore, subsidised employment (where wage subsidies - a type of labour market programme - are paid by the FEA) was used as an additional outcome. However, we only started comparing the individuals after they had completed the training programme. First, we could not identify a date of acceptance among non-rehabilitants, and second, we were interested in the effects of VR status on training programme participants. Hence, we followed participants for one year after programme completion.

3.6Confounders

Based on results from previous studies on VR acceptance and rehabilitants’ selection into training programmes and the description of their employment outcomes [7, 9], we considered the following confounders as relevant explanatory variables for the selection to VR and the selection to training programmes: sex, age, vocational education, disability status, employment status before VR or training programme, place of residence, days in longer illness before VR or training programme (> 6 weeks: in Germany, sick pay is paid by the employer for the first six weeks. For sick leave exceeding six weeks, health insurance covers approximately 70 percent of the usual income [26]) and the local unemployment rate.

3.7Statistical analysis

We applied PSM methods to conduct the comparison. PSM identifies the most similar comparison partners within the treatment and comparison groups with respect to factors influencing the probability of the treatment. Our analysis consisted of three steps. First, the probability of rehabilitants being assigned to training programmes (comparison I) and the probability of participants being granted VR status (comparison II) were calculated by means of probit regression models. The confounders described above served as predictors in the probit regressions. In accordance with Rosenbaum and Rubin [27], we applied an iterative approach to specify our propensity score model and modified it to avoid important residual systematic differences between the treatment and control groups. For this purpose, we conducted several probit models, cf., adding different interaction terms until systematic differences in the confounders of the treatment and control groups were reduced to an acceptable level [28–30]. We used bivariate statistics and compared the bias of the matched and unmatched samples to check the balancing of the comparison populations. Tables 1 and 2 show the standardised differences and the respective variance ratio for the raw and matched samples for comparisons I and II. The differences for each covariate for the matched samples are practically zero, and the variance is practically 1. This suggests a well-balanced sample for both comparison groups [31].

In a second step, the obtained propensity scores were used to identify the most similar partners in the comparison group (nearest neighbour 1 to 1 match; with replacement). We matched each member of the treatment group with the most similar member of the respective control group in terms of their estimated probability of participation/status recognition (propensity score). Matching with replacement reduces bias to a greater extent (compared to the use of matching without replacement) and entails that each case or observation from the comparison group can be matched more than once [32]. Matching with replacement should not pose a problem in comparison II, where the control group is much bigger than the treatment group, but may lead to an increase in measurement error in comparison I, because the control group is much smaller than the treatment group. In order to test the sensitivity of the results, we also estimated an inverse probability weighting model, where the observations are weighted according to their propensity score. The results suggest that our results are reasonably robust and we achieved a good balance of covariates.

Furthermore, we experimented with different caliper sizes - the maximum distance for which two observations from the treatment and the control group are potential neighbours [32]. At the end, we applied a very small caliper of 0.005, to avoid matching with nearest neighbours whose propensity scores still differed considerably (by more than the caliper) from those of the treated individuals. Correspondingly, we excluded a small number of outliers for whom no comparison partners could be found (in comparison I, less than 1 percent was excluded; in comparison II, the threshold was approximately 4 percent).

In the final step, the average treatment effect on the treated participants was identified, i.e., “the mean effect for those who actually participated in the programme” (33: p. 605).

The matching analyses as well as the post-esti-mation and balancing were conducted using the teffects psmatch command in Stata 15 (College Station, TX, USA). We used teffects psmatch instead of psmatch2 as the teffects psmatch command accounts for the circumstance that propensity scores are not known but are estimated when standard errors are calculated [31, 34]. Before matching, the members of the treatment and control groups differed significantly from each other. These differences disappeared after applying PSM. The mean bias after matching was identified using the psmatch2 command; for the first comparison group it was 1.3 and for the second comparison group it was 1.6 (Tables 1 and 2) [21].

3.8Characteristics of the two populations

The first comparison group consisted of 7,905 individuals who had been granted VR status in 2009 or 2010. The treatment group comprised 6,582 rehabilitants who started training programmes afterwards. The programmes lasted approximately two years on average. The control group comprised 1,323 rehabilitants who did not participate in any programme. After matching, 1,185 persons from the control group had to serve as matches several times (see 3.7) in order to match 6,582 individuals from the treatment group to 6,582 individuals from the control group (Tables 1 and 3).

Before matching, the treatment group contained more men (62 vs. 55 percent), more individuals from households in receipt of basic income support (31 vs. 25 percent) and more people living in eastern Germany (28 vs. 20 percent), but fewer people entering VR from marginal employment (11 vs. 19 percent). The treatment group exhibited fewer days of sickness (93 vs. 107 days) (Table 1).

For the second comparison (a total of 21,020 persons), we took another sample and compared rehabilitants completing general training in 2012 and 2013 (n = 6,765, treatment group) with non-rehabilitants who had registered health conditions and were also participating in general training outside of VR in the same time frame (n = 14,255, control group). After matching, the 6,765 persons of the treatment group were matched to 6,765 persons from the control group (Table 4).

Before matching, the treatment group contained fewer men (59 vs. 63 percent), the participants in that group were also slightly younger (31 vs. 33 years), had experienced fewer days of illness (92 vs. 107 days), were less likely to come from a household receiving basic income support and were less likely to lack a vocational qualification (3 vs. 38 percent). Furthermore, there were fewer people with a severe disability (or equal status) in the treatment group (8 vs. 14 percent) (Table 2).

4Results

4.2Comparing employment effects of participants in training programmes with and without VR status (comparison II)

For the second comparison, we took another sample and compared rehabilitants completing general training in 2012 and 2013 with non-rehabilitants who had registered health conditions and were also participating in general training outside of VR in the same time frame.

Table 4 summarises the PSM results. One year after participating in the same training programme, rehabilitants were more likely than non-rehabilitants to be employed. Both unsubsidised and subsidised employment was more common among rehabilitants (0.04, p < 0.01; 0.02, p < 0.001), and rehabilitants were less likely to be only marginally employed one year after training (–0.02, p < 0.01). One year after programme completion, 51 percent was employed in unsubsidised employment compared to 47 percent in the control group, four percent was employed in subsided employment compared to two percent in the control group and six percent of the treated were employed in marginal employment compared to eight percent in the control group. Moreover, rehabilitants exhibited longer employment durations during the first year after training (19 days, p < 0.001) –184 days in the treatment and 165 days in the control group – and their average incomes were higher than those of non-rehabilitants (2,413 euro/year, p < 0.001) – the treated earned a mean yearly income of 13,482 euro vs. 11,069 euro in the control group.

5Discussion

The present study identified the employment effects of training programmes (comparison I) and the VR status (comparison II) for people with disabilities in VR in Germany.

Even after rehabilitants have been granted VR status, approximately a quarter of rehabilitants do not participate in any programme (comparison I). When a training programme has been chosen, approximately half of the rehabilitants participate in general training. Furthermore, similar general programmes are offered to unemployed people with no VR status. We focused on those who had health limitations that affect their placement in employment but did not apply for VR (comparison II). With respect to general training, rehabilitants and non-rehabilitants completed the same type of training. The only difference their legal status. Rehabilitants may be entitled to additional support and benefits (e.g. longer-term subsidised employment after completion of training) and are supervised by specially trained personnel.

To determine the effect of programme participation, we applied PSM to two different comparison groups. First, we focused on the effect of training, comparing rehabilitants who participated in training to those who did not (comparison I). Second, we determined the effect of the VR status by comparing rehabilitants and non-rehabilitants in the same general training programmes (comparison II).

To sum up, comparison I reveals the lock-in effects of training participation. That means that individuals participating in training are “locked” within the training and, thus, are less likely to find employment during the training programme. This lock-in effect has also been observed in other studies on the return to work (e.g. 35, 36). In terms of training, this lock-in effect is desired and makes it necessary for our analysis to have a long observation period. Therefore, four years after the start of VR, when most participants should have finished VR – even if other programmes followed training –, the employment prospects of training participants exceeded those of non-participants. Similar positive employment outcomes after training participation were also found for the unemployed [12] and for rehabilitants supported by the SPI [4].

However, the effects on income are less positive than those on employment. In the first three years, non-participants earn more than those in training. In the fourth year, the income differences disappear. Though this is a positive development compared to the effects for the initial years after training commenced, this means that even after four years the income of those participating in training did not differ from that of the non-participants. Furthermore, non-participants show statistically significant longer employment durations, although the difference was small. Thus, to a certain extent participation in training involves opportunity costs (in terms of income and employment duration) that are not offset shortly after training participation but evolve over time. Therefore, the changes in the effect year on year indicate that this is very likely to balance out in the longer term. Even longer analyses are therefore necessary to detect the long-term effects of training participation [36–40]. This is particularly reasonable since the population supported by the FEA is relatively young [19].

Furthermore, the type of occupation and its suitability regarding the health limitation or disability might serve as an important additional outcome for this first comparison group [41]. It is possible that non-participants may be more likely than training participants to take up employment not suited to their disability. If this is the case, especially (psychological) health is more likely to be affected and might result in sickness periods and shorter employment durations in the long run [42]. This would call into question the definition of successful VR in the context of most previously conducted studies and should therefore also be addressed in future research.

Comparison II analysed the effects of the VR status. The results indicate positive effects of persons holding VR status compared to non-rehabilitants with regard to taking up employment, income and employment duration. Since comparison II only differs according to the VR status, these positive effects are primarily driven by disability-specific support by specific VR counsellors and access to disability-specific services during the programmes. For instance, subsidised employment – which is also found in other studies [17] – seems to integrate rehabilitants into the labour market after training but is only available (in the long term) to individuals who have been granted VR status. A study by Wang et al. [35] supports this explanation, though they refer to the disabled worker status and its associated advantages for recipients. Nivorozhkin [5] is another example of a study supporting this result, although this study uses a different comparison group (applicants for VR whose application was not approved). Subsidised employment accounts for the fact that employers are more willing to employ people with disabilities if they are compensated for additional costs that are associated with their return to work [43].

5.1Strengths and study limitations

There is still little evidence of the employment effects of vocational training for people with disabilities. Furthermore, there are no studies identifying the effect of the VR status itself. This is due to the fact that comparison groups are hard to find. Our study addressed this issue and carefully chose two comparison groups. Based on representative, large-scale administrative data, our study is the first to examine the employment effects of training programmes and of the VR status for people with disabilities in Germany supported by the FEA. The availability of longitudinal data enabled us to follow participants with regard to their medium-term employment outcomes. Further research is necessary to investigate long-term effects. The available data used in this study would also be suitable for such research in the future as the data is to be updated at the end of 2021 to cover an even larger time frame until December 2020.

However, the administrative data also come with constraints. We were unable to take the severity of health conditions into consideration (as most studies on VR do), so we could not rule out the possibility that people with more severe health conditions may more often be assigned to special programmes. However, the study by Nivorozhkin [5] using the same data set and similar outcomes applied a simulation that showed that the labour market outcomes are not substantially affected by unobserved information, such as the type of disability. Furthermore, we already mentioned above that in comparison I matching with replacement may lead to an increase in measurement error, because the control group is much smaller than the treatment group. Also in comparison I, the confounders “local unemployment rate”, “disability status” and “days in longer sickness” were measured at VR acceptance and might change over time/after programme implementation. Finally, although we can measure income on a reliable and daily basis, we cannot control for the hours worked per day.

6Conclusion

On the one hand, participating in training is associated with a programme lock-in effect lasting approximately three years for rehabilitants; thus, keeping rehabilitants from job search for the sake of programme participation. However, in the fourth year when VR is completed for most of the participants employment prospects improve considerably showing positive effects. However, training participation does not lead directly to higher incomes and longer employment durations. We interpret this finding that participation in training therefore constitutes an investment for social security institutions as well as for participants, but are confident from the observed effect development over the prior years that income effects may well become positive after a longer observation period. On the other hand, the VR status is associated with positive employment and income opportunities. This is due to the intensive support and access to long-term subsidised employment, which boosts participants’ employment prospects and increases their ability to return to an independent life. At this point, further research must examine why individuals with health issues do not apply for VR in order to formulate actions to overcome barriers to entering VR.

Ethics approval and consent to participate

All procedures performed in this study were in accordance with the code of ethics of the German Sociological Association and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Consent for publication

Since administrative data refer to social security data, formal individual consent is not required.

Availability of data and materials

The data that support the findings of this study are particularly protected social data. According to §75 Book X of the German Social Code, the data are available on request from The Federal Ministry of Labour and Social Affairs via the Institute for Employment Research (department DIM). Data may only be analysed at the Institute for Employment Research or one of its locations outside Germany.

Conflict of interest

The authors declare that they have no competing interests.

Funding

This work was supported by the German Federal Ministry of Labour and Social Affairs. The funding party had no involvement in the study whatsoever.

Author contributions

Both authors analysed and interpreted the data and wrote the manuscript together. All authors read and approved the final manuscript.