A mixed methods study of employers’ and employees’ evaluations of job seekers with a mental illness, disability, or of a cultural minority

1.1Previous studies

Previous studies have found that employers prefer to hire persons with a physical disability over someone with mental health issues [3–5], but always prefer employees without a health issue [6, 7]. Interview studies have revealed that perceived risk factors in employers’ assessment of applicants with a disability include concerns with productivity level, skills set, fit with work environment, absenteeism and uncertainty of accommodation needs [8–10]. One study found that employers’ specific concerns varied between employees with physical disabilities as opposed to mental health issues [11]. Mental health conditions were generally regarded as more diffuse and challenging to handle than physical disabilities. For job applicants representing a cultural minority, experimental studies have found that those who appear integrated in the majority culture are favoured among employers [12], and that job seekers with an ethnic name get fewer call-backs than job seekers representing the majority culture [13–15].

Studies investigating the influence of having previous experience with working together with people with disabilities have shown mixed results: While some studies show an association between experience and positive attitudes towards people with disabilities [16], other studies have failed to establish the same association [17]. Unger [17] found that studies conducted before the Americans with Disabilities Act, found that experience was associated with more positive attitudes; however, studies conducted after the act was passed indicated that employers expressed positive attitudes irrespective of experience. This may indicate that the effect of experience is hard to establish due to social desirability.

In the current study, Stone and Colella’s model of “Factors affecting the treatment of disabled individuals in organizations” is used as a framework for the hypotheses and for interpretation of the results [18]. The model focuses on disability, while the focus in the current study is also on other groups that are underrepresented in the labour market. In the following, the model will be presented, emphasising aspects that are particularly relevant for the current study.

1.2Stone and Colella’s model of factors affecting the treatment of disabled individuals in organizations

Stone and Colella’s model theorize how contextual factors (mainly legislation), organizational factors (i.e. technology, norms, policies) and individual factors (i.e. nature of disability, former contact with people with disabilities, stereotypes) interact to shape psychological assessments and expectations towards people with disabilities. These assessments and expectations are termed “psychological consequences”.

The behaviour that results from these psychological processes elicits certain responses from people with disabilities on the receiving end. Their response to this treatment may then go on to affect contextual, organizational and individual factors. The interactions in the model are recursive, and the model thus manages to capture the malleable nature of the factors included, which may change over the course of time within an organization. This makes the model a well-suited framework for understanding the demand-side factors of employment not only for people with a disability, but for other groups that are under-represented in the labour market. The current study specifically investigates individual factors and psychological consequences of these, as expressed through assessments of job seekers with different characteristics.

While most studies focus on either disability or cultural minorities, the current study investigates willingness to include job seekers representing a range of groups in society that are underrepresented in the workforce, through a vignette study. It also compares the supervisor perspective with the employee perspective, in order to detect organizational level differences when it comes to willingness to include the different job seekers into their work group. Further, the role of experience is explored, and lastly, different types of barriers salient for certain groups are explored. These aims are answered through a mixed methods design, and formulated in three hypotheses and one explorative research question. A normative job seeker described as a single mother is included as one of the vignettes, to function as a reference case.

Hypothesis 1

Hypothesis 2: Supervisors, because of their concern for productivity and absenteeism, will assess vignette characters with a health issue significantly less favourably than employees will.

Hypothesis 3: Respondents who have previous experience with employees similar to the vignette character in question, will assess them significantly more favourably than those who do not have such experience.

Research question: What barriers to including different vignette characters can be identified, and how do these barriers vary between characters?

2.1Participants

Respondents (n = 1230) were supervisors and middle level managers (n = 305), and employees without supervisor responsibilities (n = 925). Fifty-three percent were female (n = 1207), and mean age was 44 years (n = 1180; SD 12.43). In the supervisor subgroup 45%were female (n = 300), and mean age was 46 years (n = 297; SD 10.22). In the employee subgroup 55%were female (n = 907), and mean age was 43 years (n = 883; SD 13).

Inclusive Workplace Support Centres (IWSC) at the Norwegian Labour and Welfare Administration in eight Norwegian counties assisted with recruitment, by providing lists of companies to invite for participation. IWSCs are resource centres located in all 18 counties, and their main task is to support workplaces in creating a more inclusive work life. The survey was distributed directly through emails to the supervisors and employees in the companies that agreed to participate. Response rates were 29%among supervisors and 19%among employees.

2.2Instruments

Questionnaires were sent to supervisors and to employees without supervisor responsibilities. The questionnaire included demographic background variables and the Workplace Inclusion Questionnaire (WIQ) [19]. WIQ contains vignettes describing job seekers who differ on certain characteristics, such as cultural background, health, or disability issues. The number and types of vignettes included in a given study may vary depending on the study purpose [19]. The descriptions of health issues are based on formal diagnostic criteria in the ICD-10 and medical encyclopaedias. For vignette characters with a mental illness, the diagnosis was not stated explicitly in order to avoid labelling. Only symptoms were described. The vignettes and questions are available as supplementary materials. In order to reduce the time spent answering the survey, the supervisors were randomly assigned to respond to one of two blocks of vignettes (see Table 1). Employees without supervisor responsibilities were presented with all ten vignettes. The vignette character of the single mother did not describe health issues or a cultural minority, and was included in both blocks. The purpose of this character was to include a more or less normative job seeker with no serious health or disability issues, or other traits known to be related to discrimination in the recruitment process.

The vignettes in the questionnaire attempt to give a credible description of an individual with a mental illness, a physical disability, or with a cultural minority background. All vignette characters were approximately the same age, and it was stated that they had the necessary qualifications for the job. Gender varied, with six of the characters being female. Each vignette was followed by the question: “Given the current circumstances, how do you think [name of vignette character] fits into your work group?” The respondent was asked to indicate how well the vignette character in question would fit on a scale from 1 (very poorly) to 5 (very well). If the respondent rated the vignette character negatively or neutrally (1–3 on the scale), a follow-up question was asked: “If [name of vignette character] does not fit quite/very well into your work group: What is the main reason?” Respondents were then asked to indicate their main reason from a set of pre-defined options, such as accommodation, economic consequences, interaction with others, or to fill in an open-ended response. Lastly, respondents were asked to indicate whether or not they had previous experience with an employee or colleague like the vignette character in question (yes/no).

2.3Procedure

The online survey platform Qualtrics was used to collect the data, through distributing emails with a link to the survey. A reminder was sent 1–2 weeks after the initial email. The survey was anonymous and did not store IP addresses. The survey took 10–15 minutes to complete. The survey was part of a larger study, which was submitted to the Norwegian Centre for Research Data (NSD) for consideration. Since no directly or indirectly identifiable data was collected in the survey, NSD deemed that this part of the study was anonymous and therefore did not require active consent. On these grounds, participation in the study was regarded as consent.

2.4Data analyses

The analyses were conducted with SPSS 25 and Excel. Assessments of vignette characters were recoded into dichotomous variables, so that explicitly positive assessments (the character in question fit “Quite well” or “Very well”) were distinguished from neutral and negative assessments (the character in question fit “Neither poorly nor well”, “Quite poorly”, or “Very poorly”).

Frequency analyses showed the distribution of responses in each category (positive vs neutral/negative). In a 2×2 table, type of vignette character was treated as exposure (rows), while type of rating was treated as a condition (columns). Risk ratios for being assessed positively was calculated (with 95%CI), using the vignette character describing a single mother as a reference. Chi square tests of independence (with Yates Continuity Correction) were conducted to examine the relationship between experience and type of assessment, and between supervisor status and type of assessment. Qualitative thematic analysis was used to categorize the open-ended responses in the question about barriers [20]. Two of the authors independently categorized the responses, and where they had categorized a response under two different themes, this was discussed until consensus was reached. Inter-rater reliability before consensus discussions ranged from 59%–84%for the vignette characters.

4.1Assessments of the vignette characters

The anticipated pattern of assessments of the different characters was only partially confirmed. The cultural minority characters were assessed positively, and especially the second-generation immigrant. This can be due to social desirability, but it can also be an expression of preferences for job seekers without health issues. Another interesting finding was that the assessments of vignette characters with a physical disability varied widely. This may indicate that respondents made more individualized assessments of these characters than the vignette characters representing mental illnesses. A recent study among employers found that mental health conditions were generally regarded as more diffuse and challenging to handle than physical disabilities, due to the invisibility and expected instability of mental health conditions [11]. This may explain the generally negative assessments of vignette characters with a mental illness in the current study. The share of respondents who had previous experience with colleagues similar to the one in question varied widely, which is likely to explain the lack of positive assessment of particularly the characters with visual impairment and schizophrenic symptoms.

Significant associations were found between supervisor status and assessments of some of the characters. The vignettes describing somatization disorder, depression, using a wheelchair, and having an audio impairment received significantly less positive assessments from leaders compared to employers. As effect sizes were small, the practical significance of these differences is uncertain. Generally, supervisors and employees agreed in their assessments of the characters. This coherence between organizational levels may facilitate inclusion efforts, as the same types of concerns need to be addressed across organisational levels.

Our findings indicate that previous experience with similar colleagues leads to a more positive assessment of the included vignette characters. This finding aligns well with social psychological theories explaining how humans tend to categorize others into ingroups and outgroups, and how prejudices can be reduced through positive interaction [21–23].

Six vignette characters had RR values below 1 of being positively assessed. This should be interpreted in light of the role of experience, as discussed above, as well as respondents’ own explanations for their assessments, which will be discussed in the following.

4.2Reasoning behind assessments

The reasons provided for assessing a vignette character negatively or neutrally give valuable insight into the demand-side barriers for employment for these specific groups, as shown in Fig. 1. For the vignette characters expressing schizophrenic symptoms and depression symptoms, social interaction seemed to be the main concern, while for both somatization disorder and ADHD, increased workload for colleagues was the most frequently cited concern. For both visual impairment and using a wheelchair, accommodation was by far the most frequently cited barrier. The open-ended response category “Other” enabled respondents to provide a free-text response if the predefined categories were not sufficient. This response category provided useful insight into the knowledge and assumptions of the respondents regarding the conditions described. For the vignette character using a wheelchair, “Other” constituted as much as 29%of the responses to the follow-up question. For the vignette characters with visual impairment and with schizophrenic symptoms, this category constituted 26%and 18%of the responses, respectively.

4.3Analysis of open-ended responses

The pre-defined categories were identified through discussions with supervisors during the development of the questionnaire. Even so, the free-text responses provide even more ecologically valid responses than the pre-defined categories, as the respondents were able to provide unique insights into how organizational characteristics, nature of the job, or attributes of the vignette character in question shaped their assessment. Many of the free-text responses regarding the job seeker with a visual impairment or using a wheelchair were coded into themes already available from or similar to the pre-defined categories, perhaps indicating a need to rationalize one’s negative assessment of these particular characters. Some of the quotes presented in the results, that were categorized under the theme “Accommodation” or “Assumptions about accommodation”, indicate lack of knowledge about the existence and quality of different aids that are available to people with different types of physical disabilities.

The findings in the current study are in line with the theorized relationships made in Stone and Collella’s model of factors influencing the treatment of people with disabilities in an organization [18]. Results support that characteristics of the observer, of the job seeker, and of the workplace contribute to shape how people with disabilities or health issues are assessed. In the model, “nature of the job” is assumed to interact with individual factors on the part of both the observer and the person with a disability, which in turn affect how observers treat someone with a disability [18]. Although these specific relationships were not tested statistically in the current study, these aspects are prevalent in respondents’ own reasoning when explaining the rationale behind the assessment of the vignette characters. A relevant point in this regard made by Stone and Collella, is that supervisors are likely to picture an idealized or customary way of performing a certain job, while at the same time consider the work ability of a disabled person based on more or less faulty assumptions [18]. This creates an overestimated gap between essential job requirements on the one hand, and a disabled person’s actual ability to perform that job on the other hand. The findings in the current study can to some degree be interpreted to support this notion, as lack of knowledge on both technical aids and the work ability of the different vignette characters were evident in the respondents’ reasoning.

4.4Implications of the findings

The findings are somewhat similar to what has been found in other studies about willingness to include and accommodate job seekers with disabilities or health issues [24–26]. All in all, the barriers indicated for the different vignette characters seem to represent a mix of experiences, workplace-specific circumstances, lack of knowledge, and poor attitudes. However, an important contribution of the current study is that it shows what type of barriers are pertinent to which type of health issue or characteristic. The analysis of the open-ended responses gives insight into respondents’ assumptions about work ability, especially for the vignette characters describing visual impairment and using a wheelchair, where open-ended responses were most prevalent. The findings are particularly useful for vocational rehabilitation service providers, which can play an important role as an intermediary between the job seeker and the workplace [27, 28]. Many companies have diversity statements and policies, however, this is not always associated with actually recruiting people with diverse backgrounds [29], implicating that insecurity and/or stigma held by supervisors –or employees –still negatively affect hiring processes. Studies have found that although employers express willingness to hire diverse applicants, many are unsure where to start, and want closer co-operation with vocational rehabilitation agencies [26, 30]. The need for an intermediary link, such as an employment specialist, between employers and job seekers is made clear both in the current study, as well as in previous studies [26, 31–33]. Something as simple as increasing knowledge about accommodation possibilities may create more positive attitudes [16]. The responses provided in the open-ended response category indicates that this kind of knowledge is indeed still lacking among supervisors and employees. Vocational rehabilitation agencies can help bridge this knowledge gap regarding accommodation possibilities, technical aids, reimbursement of accommodation costs, and perhaps most importantly: To provide a job match between employers’ actual needs and job seekers’ competence and motivation [28, 34, 35]. Focusing on job match and follow-along supports for both the job seeker and the employer may help mitigate the barriers that have been identified in the current study.

4.5Implications for research

Future studies could add measures of contextual factors, and address for example perceived organizational culture and its relationship with assessment of job seekers like the ones used in this study. Moreover, longitudinal designs are rare among studies investigating employers’ perspective on diverse recruitment. Future studies can measure how perceptions of different job seekers develop over time within the same industries, and see this in relation to actual hiring practices during that same period of time.

4.6Strengths and limitations

By using a vignette design instead of simply listing diagnoses, condition or cultural background, we attempted to facilitate identification while avoiding labelling based on stereotypes and lack of knowledge about specific diagnoses. We assumed that providing information about behaviour and symptoms common to a condition might increase the ecological validity. A few limitations should be pointed out, however. First, neither the vignettes themselves nor the responses they elicit, take specific contextual factors into consideration, such as cross-pressures or external demands affecting the respondent. There is a risk of eliciting responses to a hypothetical situation that differs from real-life decisions [36]. Furthermore, vignettes such as the ones used in this study do not adequately cover the diversity of how a certain diagnosis or disability manifests. All the persons described in the vignettes are in their 30’s and 40’s, which make them more comparable to each other, but not representative for the broader workforce population. The vignette characters describing mental illness did, however, conform to symptoms fulfilling each of the relevant diagnostic criteria, allowing complex health conditions to be described clearly without stating the diagnosis explicitly, while keeping information about age and qualifications constant. Meanwhile, the gender of the vignette characters was random, and we suggest that future studies manipulate the gender of the vignette characters to control for potential gender effects. This would have required a longer questionnaire, and since gender was not the topic under study, it was not included as an experimental variable. Response rate was low, but some factors may compensate for this: The sample size is relatively large, a broad range of industries is represented, both the supervisor and the employee perspectives are included, and there is gender balance in the sample. Finally, when measuring normative phenomena, such as liking or attitudes towards certain people, socially desirable answers become particularly likely. Hence, although the patterns found in the current study may indeed reflect common attitudes, they might in reality be even more pronounced.