You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

A snapshot of potential: Using situational assessments to capture the moment

Abstract

BACKGROUND:

A contributing factor to the higher unemployment rates of adults with disabilities is the limited opportunities to gain insight into employment options. Assessments are often used to learn a job seekers skills, preferences, and support needs. Situational assessments in real work settings create hands-on experiences with a variety of work tasks and work environments which better inform the employment choices of job seekers.

METHODS:

This article explores situational assessments and introduces a tool designed for the employment support professional conducting the assessment to capture information necessary to make more informed employment choices.

RESULTS:

This article defines situational assessments while introducing a person-centered tool to support employment support professionals to effectively document and analyze the observations of each situational assessment.

1Background

The employment rate of people with disabilities is persistently lower than that of people without disabilities (Sevak et al., 2015). In 2019, the unemployment rate for persons with a disability was 7.3 percent, while the unemployment rate for persons without a disability was 3.5 percent (Bureau of Labor Statistics, 2020). “Despite the ability and interest in work, people with disabilities are employed at significantly lower rates than nondisabled people. Employment is one of the largest gaps between people with and without disabilities. Moreover, those people with disabilities that are employed are often underemployed” (Friedman & Rizzolo, 2016, p. 108).

Supported employment is a service available to adults with intellectual and/or developmental disabilities (IDD) that is designed to address the disparities in employment between people with and without disabilities. The purpose of supported employment is to support individuals with disabilities to gain and maintain integrated, competitive employment (Wehman et al., 2018). An effective supported employment model is one that focuses on the capacities, interests, and skills of the job seeker (Wehman et al., 2007) and seeks to place them with an employer in need of someone with that particular skill set (Wehman et al., 2018).

Opportunities to be exposed to a variety of work environments leads to more informed choices about career preferences (Wehman et al., 2018). However, due to limitations of transition programs, lack of inclusive opportunities to be part of their communities, or lack of support to access opportunities in the community, many adults with disabilities have not had these opportunities to gain insight into employment options (Timmons et al., 2011). As a result, supported employment professionals often utilize career assessments to assist in the formation of employment goals. Historically, vocational assessments have focused on identifying skill deficits and problem areas in need of rehabilitation, which resulted in providers determining that individuals are “not ready” for employment (Wehman et al., 2007). Using more person-centered approaches to assessments can shift the focus from the job seeker’s deficits to their capacities, and determine an individual’s interests and support needs (Wehman et al., 2007).

1.1Situational assessment

Situational assessment, also known as an “on-the-job assessment” or job sampling, is a vocational evaluation whereby an individual is observed completing work tasks in multiple real-life work contexts (Peer & Tenhula, 2010). Employment specialists can utilize situational assessments to systematically “gather, analyze, synthesize and communicate” (Meserve & Bergeron, 2015, p. 1) information about an individual’s skill level by observing them in a variety of work tasks and environments. A situational assessment experience can assist a job seeker to learn more about their skills, preferences, and support needs. This, in turn, informs the career direction of the job seeker.

The Boggs Center on Developmental Disabilities is New Jersey’s federally mandated university center for excellence in disabilities. In my capacity providing technical assistance to supported employment providers in New Jersey, questions regarding assessment choice, use, and functionality consistently emerged. While several assessments exist, each had specific sections that the Boggs Center team would recommend, there was not one form that incorporated all of the useful aspects. Additionally, I observed that employment specialists would often document use of a situational assessments during their billing process, but had no meaningful analysis of how the results informed the job search and placement. This prompted the development of an original, person-centered situational assessment tool that incorporated the best practices from many of the existing models. It was designed to streamline efforts of the evaluator by providing insight into which areas should be observed, as well as how to ensure useful analysis of what was observed.

2Methods

A literature review was conducted around the topic of assessment and, more specifically, situational assessment as well as review of the tools available to record assessment findings. Research has indicated that there is no single “gold standard” assessment tool to recommend; the selection depends on the context and purpose of the assessment as well as individual factors (Peer & Tenhula, 2010). However, many situational assessment options were limited to checklists of skills, complex charts, and accompanying scoring methods. In 2018 and 2019, The Boggs Center provided 46 hours of technical service to agencies who were attempting to utilize these assessments to inform their job placements. After identifying the strengths and weaknesses of many existing tools and utilizing polls and surveys of partner agencies, a new assessment tool was created in order to more specifically meet the needs of supported employment agencies and the job seekers they serve: The Situational Tool for Exploring Possibilities (STEP).

3Situational Tool for Exploring Possibilities (STEP)

The purpose of the situational assessment is to learn more about the job seeker’s skills, preferences and support needs. The STEP guides the evaluator to capture all of the information that is necessary to make thoughtful and thorough observations during an assessment, by providing detailed instructions and guiding questions to support an evaluator during observation. This tool guides an evaluator through the process but remains open-ended enough for the subjectivity of an individualized assessment.

The STEP guides the evaluator to determine the job seeker’s work preferences by identifying how the job seeker indicates that s/he “likes” and “dislikes” a job or task, otherwise known as “like and dislike behavior” (Parsons et al., 2001). Identifying like and dislike behavior requires information to be gathered prior to the observation through discussion with the job seeker, family members, friends, and support personnel who know the person well. Whether the communication is verbal (“I like this” or “I don’t like this”), behavioral (Leaving the location, throwing the work materials, jumping up and down) symbolic (a smile, grimace, engaged body language towards the task), etc., it is unique to that person, and the evaluator must recognize how the job seeker expresses their preferences in order to make insightful observations.

The validation of work preferences can be reflected in the demonstration of a higher frequency of task engagement accompanied by indicators of like (Parsons et al., 2001). The manipulation of the materials necessary to complete the job task or task engagement is likely to occur more frequently with tasks that are preferred by the job seeker. If the job seeker does not engage in a task and indicators of dislike are present during one particular situational assessment further analysis may be necessary to question if the dislike was the particular task or the work environment.

Another important aspect of planning thoroughly prior to an assessment is exploratory observation of the assessment site and a set plan for the assessment so that the assessment site and the evaluator have an understanding of the purpose as well as the schedule of the assessment (Krueger, 2017). The length of the situational assessment is dependent on the needs of the person being assessed and the number of hours the individual wants to work once employed (Wehman et al., 2007). Best practice suggests “the right amount of time” is that which allows for the gathering of sufficient information for the team to make a decision about whether or not to move forward with job development in this particular type of work (Institute for Community Inclusion, 2010).

Within the STEP, the focus is on documenting the comments around the observation of skills rather than on a scoring method to skills observed. Guiding questions provided with each skill area (general skills, time management, interpersonal skills, stress management, initiative and motivation, problem-solving skills, organization skills and productivity) support the evaluator in understanding what information is necessary to observe and provide in subsequent documentation (Table 1). Those skills not observed during a particular situational assessment are documented as such and will inform the need to observe particular skills in future assessments.

Table 1

Guide to skills area assessment of STEP

Skills AreaObserved SkillsDocumentation: Guiding questions/observed examples of success/challenges/support needs
General SkillsTransportationHow did the person get to this particular assessment location?
Work EnduranceIs the person’s work pace/performance consistent throughout the assessment time?
Orienting to EnvironmentHow did the person navigate the particular assessment site?
StrengthDid the person have the strength necessary for the work done during the assessment?
Culture CompatibilityHow did the person manage the general “character and personality” of the specific workplace?
Maintains a Neat AppearanceDid the person have appropriate uniform/dress and hygiene in the beginning of the assessment and throughout the entire time at the site?
Time ManagementTime ManagementWas the person able to prioritize the work given?
PunctualityDid the person arrive early or on time to the scheduled assessment?
Break TimeDid the person use their break time appropriately and take the allotted time?
Interpersonal SkillsSocial InteractionsHow did the person communicate with co-workers and customers?
Supervisor InteractionsDid the person access the supervisor appropriately for assistance, feedback, etc.?
TeamworkHow well did the person cooperate with others on work tasks?
Stress ManagementFrustration ToleranceHow did the person handle overwhelming or frustrating situations?
Handling CriticismHow did the person handle comments, including positive or negative criticism?
Adapts to ChangeIf applicable, did the person tolerate change to routines?
Performs Non-preferred TasksWas the person willing to complete tasks that were not preferred without complaint?
Initiative &MotivationIndependent Work InitiativeDid the person begin repeated routine tasks without being prompted?
Maintains Attention to TaskDid the person stay focused on the task until completion without prompting?
Ability to Ask for AssistanceDid the person independently initiate contact with coworkers or supervisor for assistance?
Problem Solving SkillsInitiates Troubleshooting IndependentlyDid the person attempt to fix a problem on their own when a problem arose?
Organizational Skills &ProductivityMaterials ManagementDid the person retrieve and properly use the items necessary for the job?
Meets Productivity GoalsDid the person meet the expected productivity goal or the average productivity with quality?

Analysis questions at the end of each assessment support the reflection of the evaluator based on that particular assessment. The questions are to support the identification of particular skills that stood out and possible employment options based on those skills. As well as functional limitations, challenges or barriers that were observed and the possible services/supports that would be necessary for the job seeker in that particular type of work. Each situational assessment should provide further information to the conversations with the job seeker and their team around the direction of their job search.

3.1Limitations of STEP

While the STEP can be a useful tool to bring observational data to the job seeker and team to support analysis of the direction of job development, there are also limitations to this tool. The situational assessment observations documented by the evaluator are subjective in nature. The analysis of the information is limited to the amount of information collected during the observations.

4Conclusion

Situational assessments and the data collected can inform the job seekers’ job search and enhance the job supports thereby improving employment outcomes (Peer & Tenhula, 2010). They are an effective way of supporting individuals with disabilities to gain the experience and exposure to work environments to make informed choices about work preferences and interests. The observational data collected can be used to facilitate workplace supports and accommodations for future employment. The STEP is a tool to collect data and add information to the conversation of employment direction.

Conflict of interest

None to report.

References

1 

The Boggs Center on Developmental Disabilities. (2019, February). Situational tool for exploring possibilities. NJ Department of Human Services, Division of Developmental Disabilities. https://rwjms.rutgers.edu/boggscenter/publications/STEP.html

2 

Bureau of Labor Statistics U.S. Department of Labor. (2020, February). Persons with a disability: Labor force characteristics-2019. https://www.bls.gov/news.release/pdf/disabl.pdf

3 

Friedman, C. , & Rizzolo, M. C. (2017, January). Get us real jobs:” Supported employment services for people with intellectual and developmental disabilities in Medicaid Home and Community Based Services Waivers. Journal of Vocational Rehabilitation, 46, 107–116. https://doi.org/10.3233/JVR-160847

4 

Institute for Community Inclusion University of Massachusetts Boston. (2010). Situational assessment fact sheet. https://thinkcollege.net/resource/employment-supports/situational-assessment-fact-sheet

5 

Krueger, R. (2017). Observation in evaluation. University of Minnesota https://www.betterevaluation.org/sites/default/files/Observation%20R.Krueger%2010.17.pdf

6 

Meserve, A. , & Bergeron, K. , (2015, March). Focus on: Six strategic steps for situational assessment. Public Health Ontario Partners for Health. https://www.publichealthontario.ca/-/media/documents/F/2015/focus-on-situational-assessment.pdf?la=en

7 

Parsons, M. B. , Reid, D. H. , & Green, C. W. (2001). Situational assessment of task preferences among adults with multiple severe disabilities in supported work. Journal of The Association of The Severely Handicapped, 26(1), 50–55. https://doi.org/10.2511/rpsd.26.1.50

8 

Peer, J. E. , & Tenhula, W. (2010). Assessment of vocational functioning in serious mental illness: A review of situational assessment and performance based measures. Journal of Vocational Rehabilitation, 32, 175–189. https://doi.org/10.3233/JVR-2010-0508

9 

Sevak, P. , Houtenville, A. J. , Brucker, D. L. , & O’Neill, J. (2015). Individual characteristics and the disability employment ga. Journal of Disability Policy Studies, 26(2), 80–88. https://doi.org/10.1177%2F1044207315585823

10 

Timmons, J. C. , Hall, A. C. , Bose, J. , Wolfe, A. , & Windsor, J. (2011). Choosing employment: Factors that impact employment decisions for individuals with intellectual disability. Intellectual and Developmental Disabilities, 49(4), 285–299. https://doi.org/10.1352/1934-9556-49.4.285

11 

Wehman, P. , Inge, K. J. , Revell, Jr W. G. , & Brooke, V. A. (2007). Real work for real pay: Inclusive employment for people with disabilities. Paul H. Brookes Publishing Co.

12 

Wehman, P. , Taylor, J. , Brooke, V. , Avellone, L. , Whittenburg, H. , Ham, W. , Brooke, A. M. , & Carr, S. (2018). Toward competitive employment for persons with intellectual and developmental disabilities: What progress have we made and where do we need to go. Research and Practice for Persons with Severe Disabilities, 43(3), 131–144. https://doi.org/10.1177%2F1540796918777730