A national benchmark investigation of return-to-work outcome rates between African American, Native American or Alaskan Native, Latino, Asian American or Pacific Islander, and Non-Latino White veterans served by state vocational rehabilitation agencies: Application of bootstrap data expansion

1.1Disability public policy context

Section 21 of the Rehabilitation Act Amendments of 1998 documented patterns of inequitable treatment of minorities (i.e., African Americans, Native Americans, Latinos, and Asian Americans) across all stages of the VR process. More specific, congress found that: (a) persons from these traditionally underrepresented populations possessed higher rates of disability, (b) they were underrepresented in the public VR system, and (c) they were less likely to achieve positive employment outcomes when compared to Whites (Lewis et al., 2007). Findings could lead to future scaled-up inquiries and subsequent new policy and promising service initiatives and strategies aimed at eradicating differential experiences.

3.1Sample

The overall sample for this study consisted of 11,603 VR consumers who were served by the 56 state and territorial VR agencies across the nation during Fiscal Year (FY) 2013 (October 1, 2012, through September 30, 2013), and were (a) reported as being a veteran (veteran status = 1), (b) identified as Latino (ethnicity status = 1), non-Latino and White only (ethnicity status = 0 and race code = 100000), African American only (ethnicity status = 0 and race code = 010000), Native American or Alaskan Native only (ethnicity status = 0 and race code = 00100), or Asian American or Pacific Islander only (ethnicity status = 0 and race code = 00010 or 00001), and (c) reported as having a developed and signed Individualized Plan for Employment (IPE); i.e., closed status 26 (successfully rehabilitated) or 28 (not successfully rehabilitated). Of these 11,603 veterans, 3,072 (26.48%) were African American, 124 (1.07%) were Native American or Alaskan Native, 723 (6.23%) were Latino, 86 (0.74%) were Asian American or Pacific Islander, and 7,598 (65.48%) were non-Latino White. Overall, males accounted for 10,126 (87.27%) of participants while there were 1,477 (12.73%) females in the study sample.

A race by gender breakout indicated that the sample included 630 Latino male veterans (5.43%) and 93 Latina veterans (0.8%); 105 (0.90%) Native American or Alaskan Native male veterans and 19 (0.16%) Native American or Alaskan Native female veterans; 74 (0.64%) Asian American or Pacific Islander male veterans, and 12 (0.10%) Asian American or Pacific Islander female veterans; 2,602 (22.42%) African American male veterans and 470 (4%) African American female veterans; 6,715 (57.87%) non-Latino White male veterans and 883 (7.6%) non-Latino White female veterans. A plurality of these veterans (N = 4,781 or 41.20%) possessed a high school diploma/equivalency or less while those with some post-secondary education (N = 2,895 or 24.95%), an associate’s degree/vocational technical certificate (N = 2,297 or 19.80%), and a bachelor’s degree or higher comprised the residual of the sample. The total sample was utilized to generate the profile and analysis of VR service access and successful return-to-work outcomes.

The employment of a single regression analysis absent resample procedures is oftentimes problematic for accurately predicting the analysis to the population (Bleeker et al., 2003; Harrell, Lee, & Mark, 1996; Kromrey & Hines, 1996; Pedhazur, 1982; Steyerberg et al., 2001). Moreover, resample procedures by themselves can be inadequate for addressing unbalanced populations with small sample sizes for respective study target groups (Dupret & Koda, 2001; He & Garcia, 2009). To address the issue of resample or the lack thereof, several approaches (i.e. cross validation and split sampling techniques) have been recommended that help to estimate a model’s optimism (Bleeker et al., 2003; Efron & Tibshirani, 1997; Efron & Tibshirani, 1994). For example, Moore et al. (2015) employed a randomized split-half cross-model validation research approach to develop and test of a series of logistic regression models across two samples to identify the best fitting final predictive model. Although applicable, this model may not be optimally suited for producing stable results in the case of unbalanced or small comparison group sample sizes (see Table 1).

The stratified bootstrap method addresses both concerns; resample issues as well as small comparison group sample sizes. The concept of resampling data, more commonly referred to as bootstrapping, has been in use for over three decades although it is increasingly being used in medical fields (Henderson, 2005). This technique involves taking a large number of samples with replacement from the original sample by strata and is useful for analyzing small datasets where prior information is sparse (Zhu, 1997). In contrast to cross-validation or split-sample approaches, bootstrap methods are very efficient, as the entire dataset is used for model development, and no new data have to be collected for validation (Steyerberg et al., 2001). The technique provides nearly unbiased estimates of predictive accuracy that are of relatively low variance (Bleeker et al., 2003; Harrell et al., 1996). Moreover, applied bootstrap data expansion has been shown to have good performance in building and testing models for validation across unbalanced sample populations (Gelman et al., 2014). The benefits of this computer-intensive technique is freedom from two major limiting factors that have dominated classical statistical theory from the beginning: (a) the assumption that the data conform to a bell-shape curve, and (b) the need to focus on statistical measures whose theoretical properties can be analyzed mathematically (Zhu, 1997).

The national FY 2013 RSA-911 database (N = 589,402) was used in this analysis. It is important to note that this database does not differentiate between veterans by “wartime” era. For example, the “veteran” variable in the database does not distinguish between a Wounded Warrior, a Vietnam War Era, a Persian Gulf War veteran, or non-war time veteran. The “veteran” variable only indicates whether the consumer was a veteran (code = 1) or not a veteran (code = 0). The two data categories for the criterion included statuses 26 (i.e., exited with an employment outcome) and 28. The RSA data in the type of closure category labeled 4 and 5 were combined to reflect status 28, which indicates that a veteran was not successful in returning to work. The category labeled 4 (“exited without an employment outcome, after receiving services”) included statuses 14, 16, 18 and 20. The category labeled 5 (“Exited without an employment outcome, after a signed IPE, but before receiving services”) included status 12 only.

3.2Data analysis

Descriptive and multivariate statistics were utilized to analyze data. Access frequencies and return-to-work percentage rates were generated, compared, and reported for the five comparison groups. Next, bootstrap resample techniques were applied to increase the efficiency of validation procedures for predictive logistic regression analyses, and the final predictive model was used to evaluate the return-to-work rates across racial/ethnic target groups, gender, and level of educational attainment at closure. The Statistical Analysis System (SAS), desktop version 9.4, was used in these calculations (SAS Institute, 2014).

3.3Key observations-profile

Several observed differences across RSA Regions and the nation emerged as key findings and could have future implications for veterans of color with disabilities, and the state-federal VR program that serves them. Many of these differences can be observed in the tables and figures presented and do not require additional response. As such, we will discuss only a select number of key observations. First, as shown in Table 2, nationally African American, Native American or Alaska Native, Latino, and Asian American or Pacific Islander veterans with signed IPEs were less likely to return-to-work successfully than non-Latino White veterans with signed IPEs. More specific, we found that 42.12%, 43.55%, 47.16%, and 48.84% of African American, Native American or Alaskan Native, Latino, Asian American or Pacific Islander veterans, respectively, with a signed IPE successfully returned to work nationally compared to 52.25% of non-Latino White veterans with a signed IPE. This finding represents a remarkable 10.13% and 8.70% national disparity for African Americans and Native Americans or Alaskan Natives, and a 5.09% and 3.41% difference between such outcomes for Latinos and Asian Americans or Pacific Islanders and non-Latino Whites. Second, as shown in Fig. 1, stark differences in return-to-work outcome benchmark comparisons between target groups across RSA Regions can be gleaned from the results. The national FY 2013 RSA-911 database benchmark for successful return-to-work outcomes was calculated to be 50.89%. Remarkably, African American, Native American or Alaskan Native, Latino, and Asian veterans’ return-to-work outcome rates were found to be below this benchmark across 7, 6, 7, and 5, respectively, of the 10 regions while such outcomes for Whites were below this criterion in only 2 of the 10 regions.

3.4Consumer characteristics and return-to-work outcomes

The association between select characteristics (i.e., race/ethnicity, gender, education level at closure) was tested using multinomial logistic regression and applied bootstrap resample techniques to increase the efficiency of interval validation procedures (Gude et al., 2009; Steyerberg et al., 2001). The procedures are reflected broadly in Fig. 2. The algorithm and detailed description of the procedures employed to bootstrap replicate, resample, build, and test models follow.

Procedure 1. We stratified and selected case records (n = 50, 100, 500 each stratum), based on the overall inclusion criteria, representing the total sample (N = 11,603) with replacement as a sample. The PROC SURVERYSELECT procedure with ethnicity as the strata in SAS version 9.4 was applied in this procedure (SAS Institute, 2012).

Procedure 2. Sample sizes 5*n with B replicate were created, and then a logistic regression with variable selection procedure was applied to each replicate. Logistic regression is a form of statistical modeling and is appropriate for analyzing categorical outcome variables (Agresti, 2013; Chatterjee & Hadi, 2013; Hosmer Jr & Lemeshow, 2004). The PROC LOGISTIC procedure in SAS version 9.4 was applied in these procedures (Stokes, Davis, & Koch, 2012). The data description of the three explanatory variables and the dependent variable is provided in Table 3.

A logistic regression model (1) was conducted on the selected sample with selected predicted variables to find the best fitting model and to estimate thevector βˆ.

where β_i (i = 1, 2, ...  , p) are the coefficients estimated using maximum likelihood estimation., X_i (i = 1, 2, ...  , p) are explanatory variables, andɛ is error. A predicted logit was obtained from the solved logistic regression equation by substituting the explanatory variables’ value into the sample estimate of the logistic regression equation

The predicted probability is given by

This value which represents veterans’ successful return-to-work serves as the binomial distribution of Yat values ofX.

Next, we used a three step model selection process to determine the relationship of the independent variables to the dependent variables of intention. The following steps, as reflected in the scenario presented in Table 4, (sample size = 250 (50*5), and B = 50) with replicate = 10 (table 4) were used to develop and test the models.

Step 1. To develop Model 1, the following null hypothesis H₀ was tested: There is no significant difference in successful return-to-work outcome rates between racial and ethnic target groups. If the p-value of the Wald Chi-square test was less than 0.10 then the null hypothesis would be rejected, otherwise we would remove the race/ethnicity variable from Model 1. The logistic regression results, χ² = 9.82, df = 4 p < 0.10, indicated that race/ethnicity was a significant predictor in Model 1 for this bootstrap sample.

Step 2. To develop Model 2, the gender variable was entered in Model 1. The following null hypothesis H₀ was tested: Model 1 (reduce model) is an adequate model. The alternative hypothesis H₁ was: Model 2 (full model) is an adequate model. If the p-value in the Wald Chi-square test was less than 0.10, yielded significance would provide the research team with the confidence to reject null hypothesis, and Model 2 would be a more adequate fit to the data. The logistic regression results, χ² = 3.16, df = 1, p < 0.10, indicated that gender was a significant predictor in Model 2.

Step 3. To develop Model 3, the level of education attainment at closure (Edu) variable was entered in Model 2. The following null hypothesisH₀was assessed: Model 2 (reduce model) is an adequate model. The alternative hypothesisH₁was: Model 3 (full model) is an adequate model. Similar to step 2, if the p-value of the Chi-square test was less than 0.10 then we would reject the null hypothesis and Model 3 would be a more adequate to fit the data. Chi-square results, χ² = 14.92, df = 3, p < 0.10, yielded significant confidence to reject the null hypothesis indicating that Model 3 made the best fit to the data when compared with Model 2. Overall, Model 3 made the best fit to the data in comparison with Models 1 and 2 in this bootstrap sample.

Through this process we identified the best fitting model in this bootstrap sample, and then recorded the coefficient of the best model.

Procedure 3. We repeated the above procedure B (30, 50 and 100) times to obtain the bootstrap estimates of parameters. We then computed the mean of each of the parameter estimates, which was denoted as β^¯i for the predictive model. We used the bootstrap resample technique not only to refine the selections of predictive variables but also to increase the efficiency of internal validation procedures for the final predictive logistic regression model (Duwe & Freske, 2012; Taylor, Ankerst, & Andridge, 2008).

Procedure 4. We estimated model validation as the next procedure. The c-statistics were located in the same area under the receiver operating characteristic curve and was used to measure how well the model discriminated between observed data at different levels of the outcome. The following steps were used to model validate each scenario (Efron & Tibshirani, 1994; Harrell et al., 1996; Ye & Zhao, 2010).

Step 1: We fitted the logistic regression model into the original data, and estimated the c-statistic, denote as C_app.

Step 2: For b = 1, 2,  ...  B:

Step 3: We calculated the estimate of optimism O=B−1∑b=1BCb,boot−Cb,orig .

Step 4: We calculated the optimism adjusted measure of predictive ability as C_app - O.

Procedure 5. We selected the biggest C statistic as best scenario (see table 5), then calculated the β^¯i as the final predictive model. The best fitting model shown in Table 6 was used to analyze the data to address research question #3.

3.5Analysis results

Logistic regression results indicated that race, African American versus White, (OR = 0.686; 95% confidence interval [CI] = [0.630– 0.747], p < 0.05) and gender, male versus female, (OR = 1.186; CI = [1.061– 1.325], p < 0.05) were significant predictors of successful return-to-work. The odds ratios (OR) or effect sizes for the final predictive model are shown in Table 6 and provide the estimated coefficients that predict successful return-to-work outcomes. The coefficients (B) were the log odds of the event occurring (i.e., change in the log odds associated with one-unit change in the independent variable). All things being equal, the OR coefficient indicated that a White veteran was 1.5 (1/0.686) times more likely to successfully return-to-work compared to an African American veteran. Similarly, a male veteran was 1.19 times more likely to achieve successful rehabilitation compared to female veteran. In addition, results yielded significance for level of educational attainment and return-to-work success; high school diploma/equivalency or less versus bachelor’s degree or higher (OR = 0.702; CI = [0.626– 0.788], p < 0.05), some post-secondary education versus bachelor’s degree or higher (OR = 0.549; CI = [0.485– 0.621], p < 0.05), associate’s degree/vocational technical certificate versus bachelor’s degree or higher (OR = 0.814; CI = [0.715– 0.925], p < 0.05). In short, a veteran who had bachelor’s degree or higher was 1.42 (1/0.702) times more likely to successful return-to-work compared to a veteran who had high school diploma/equivalency or less. Similarity, a veteran with bachelor’s degree or higher was 1.82 (1/0.549) times more likely to successfully return-to-work compared to a veteran who had some post-secondary education (no degree or certificate). Finally, a veteran with bachelor’s degree or higher was 1.23 (1/0.814) times more likely to successful return-to-work compared to a veteran with associate’s degree/vocational technical certificate.

The predicted probabilities for the race/ethnicity, gender, and level of educational attainment at closure variables are presented in Fig. 3. Overall, the analysis yielded the following successful return-to-work probabilities:

3.6Discussion

This study examined the relationship between race/ethnicity, gender, and level of educational attainment and return-to-work outcomes among African American, Native American or Alaskan Native, Latino, and Asian American or Pacific Islander veterans with a signed IPE. The findings indicated that veterans who are African American, female, and those with an educational attainment below a bachelor’s degree were statistically significantly less likely to return-to-work successfully compared to non-Latino Whites, males, and those who have achieved a bachelors’ degree or higher, respectively. Moreover, generated predictive probability results for minority veterans represent new field information and point out that (a) African American veterans had the ‘poorest’ return-to-work probabilities followed by Native American or Alaska Native, Latino, Asian American or Pacific Islander, and non-Latino White veterans, (b) female veterans had lower return-to-work probabilities than their male counterparts, and (c) regardless of the race/ethnicity and gender, those with some post-secondary education had the lowestprobability followed by those with a high school diploma or less, associate’s degree/vocational technical certificate, and bachelor’s degree or higher.

Several plausible explanations exist for these results. First differential physical and mental health condition incidence among minority veterans with disabilities may contribute to the findings on race/ethnicity. Health disparities are defined as systematic, socially produced, and important differences in health between groups that are not only unnecessary and avoidable but, in addition, unjust and unfair (Goode et al., 2014). In general, adult Latinos, Native Americans or Alaska Natives, and African Americans with disabilities more often report fair or poor health (55.2%, 50.5%, and 46.6%, respectively) compared to non-Latino Whites with disabilities (36.9%) (Wolf, Armour, & Campbell, 2008). Sheehan and colleagues (2012) reported that being a Latino veteran is associated with a 1.32 increase in the odds of being in poor health relative to White veterans. They also found that being a Black veteran is associated with a statistically significant 2.91 increase in odds of being in poor or very poor health. In addition to physical health concerns, mental health issues such as Post-Traumatic Stress Disorder (PTSD) represents a unique challenge for many minority veterans, and the lack of employment itself can act as a trigger for occurrence (Atkins, 2011). Health disparities, whether physical or mental, can have a profound impact on minority veterans’ with disabilities potential to obtain or maintain integrated competitive employment (Feist-Price & Khanna, 2011).

Given the devastating effects of poor health realities and negative return-to-work implications for these veterans, proper interventions and treatments are of extreme importance. However, adequate treatment is not always easily obtained. Although the U.S. Veterans Administration (VA) has significantly increased the size and scope of nationwide health services; many African American veterans remain underserved (Moore, Johnson, & Uchegbu, 2011). Furthermore, many who experience lower socioeconomic status cannot afford health insurance, which is especially problematic because they are sometimes employed at small independent businesses or temporary employment agencies that fail to provide coverage (Feist-Price & Khanna, 2011). As such, SVRA and VA counselors should consider developing IPEs for these consumers that include effective health improvement (i.e., physical and mental) objectives. Health improvement among these consumers could result in greater physical and mental functioning, which could ultimately lead to an increase in their successful return-to-work rates.

Transferrable skills and military occupational specialty (MOS) mix-match might also account for the finding on racial and gender differences in return-work outcomes. For example, many minority veterans often obtain their MOS in areas such as infantrymen, weapons specialties, munitions handlers, and door gunners on helicopters (Feist-Price & Khanna, 2011) although the skills gained from these occupations are not easily transferable to jobs or careers in the U.S. civilian sector (National Coalition of Homeless Veterans, 2010). The occupation of infantryman is problematic because although law enforcement is a great option, job availability is insufficient compared to the large number of job-seeking combat veterans of color who have returned from Iraq and Afghanistan. Also, many law enforcement professional positions require college credits to be eligible for consideration (Feist-Price & Khanna, 2011). Additionally, female veterans often compete with male and female civilians who have more site-based training or more personal contacts with potential employers and their existing workers (National Coalition of Homeless Veterans, 2010). Among veterans who are hired, especially females, they are often at the low end of the wage scale and health benefits can be limited or non-existent. From a policy perspective, these results may warrant a renewed focus on developing new funding priorities that promote the modification of existing job-training programs, especially in urban areas, that can respond more rapidly to the needs of the constantly changing industry market demands and opportunities. The match between minority veterans’ competitive skill sets and market demands should remain a policy focal point at the state (SVRAs) and federal (VA Vocational Rehabilitation & Employment Programs) level.

The finding that veterans with an bachelor’s degree or higher were more likely to return-to-worksuccessfully compared to those with a high school diploma/equivalency, associate’s degree/vocational technical certificate, or some post-secondary education is not surprising. In short, those who are gaining skills and knowledge that prepare them for the workforce are more likely to be successful at obtaining employment. One explanation for this finding is that veterans of color may not be as aware of resources available to support their educational aspirations (Moore et al., 2015). One such research initiative grows out of the Post-911 Veterans Educational Assistance Act of 2008 or what is commonly referred to the as The New Government Issue (G.I.) Bill. This initiative is an effort to pay for veterans’ college expenses similar to the extent of the original G.I. Bill after World War II (Madaus, Miller, & Vance, 2009). Veterans are eligible to receive the full amount of tuition and fees charged by a college or university, not to exceed the most expensive in-state public institution (Grossman, 2009). Perhaps SVRA and VA Vocational Rehabilitation & Employment (VR&E) Programs need to develop new initiatives and strategies aimed at reaching into communities of color to educate veterans and their family members about programs that can assist them in pursuing a bachelor’s degree or higher or some sort of re-training that matches with the current job market skill demands.

Overall, there is also a need for SVRA counselors to provide culturally appropriate services to veterans of color to address disparate return-to-work outcome rates. Culture, as defined by the 38th Institute on Rehabilitation Issues (IRI) Prime Study Group, is “a system of language, values, and supports that extend and distinguish a group’s sense of necessary identify” (IRI, 2015). Service delivery should be grounded in the cultural humility model as described by Tervalon and Murray-Garcia (1998) for the health care field. Applicatin of this model to VR requires counselors to be respectful of consumers’ customs and traditions and committed to self-critique to develop mutually beneficial and non-paternalistic service relationships. Technical Assistance and Continuing Education (TACE) Centers and relevant Rehabilitation Research and Training Centers (RRTCs) can provide training resources to SVRAs and their counselors in this regard.