The use of extended reality (XR) for people with moderate to severe intellectual disabilities (ID): A scoping review

2.1Research question

Based on previous research experience and the expertise of the research team, we formulated the following research question: What is known from the existing literature about the use of extended reality (XR) in supporting people with moderate to severe intellectual disabilities (ID)? Specifically, we were interested in (1) the objectives of the XR-based interventions aimed, (2) features of the XR content and technology used, (3) factors that were reported to affect the implementation of the intervention.

2.2Identifying relevant studies

We conducted the initial search for literature in January 2021 using the following search formula: intellectual disability AND virtual reality OR augmented reality OR mixed reality. We used both free-text search terms and MESH headings to search the following electronic databases: Cinahl, PubMed, LibSearch, and Google Scholar (see Fig. 1 Search String). We limited our search to English papers published between 2010 and 2021. Furthermore we manually screened the reference lists of the papers that we found to identify other potential key publications that we had missed. The retrieval process was iterative, and we redefined the search strategy more than once to ensure full coverage of the literature [26].

2.3Selecting studies

We imported the publications resulting from our search into Endnote and removed duplicated records. Subsequently, we selected relevant publications in three stages. Firstly, we screened the titles of the publications, after which we reviewed the abstracts and full texts. We included any papers without an available abstract directly in the full-text review stage. Two reviewers (PLM and RD) discussed the results until consensus was reached.

To be eligible for review, articles had to meet the following criteria: (a) inclusion of individuals with moderate to severe ID; b) use of XR; (c) reporting of empirical data. We did not apply limitations for age or gender. Due to the high comorbidity of intellectual disability with other disorders [22], we also included papers reporting studies that included participants with other disorders such as Down Syndrome or Autism Spectrum Disorder in addition to a moderate or severe ID. Additionally, since the majority of studies included participants with different severity levels of ID, we only selected papers about studies that included at least one participant with moderate or severe ID. We excluded any articles from our review when they (a) did not include participants with moderate or severe ID, (b) included only participants having acquired brain injury; (c) did not use XR; (c) did not report empirical data.

2.4Charting the data

We developed a data-charting form to map the key findings as well as the following descriptive variables reported in the papers: author names, year of publication, study design, research aim(s), target population, XR system used, the setting and the country where the intervention took place, and outcome measures. Two reviewers (PLM and AV) pre-extracted seven randomly selected articles to determine the degree of congruency between the results. Finally, PLM charted the data, discussed the results with the research team and iteratively updated the data-charting.

2.5Collating, summarizing and reporting results

One of the researchers (PLM) initially performed a content analysis while focussing on the objectives of the reported interventions, the features of the XR content and technology that were used, any factors reported by the authors that may have affected implementation, and lessons learned. Results were then discussed and reported with two other researchers (RD and KS) until consensus was reached.

3.1Objectives of interventions

In this section we provide an overview of the main objectives of the XR-interventions reported in the 16 papers included in our scoping review. Most of these interventions focused on specific skills domains. Five domains stood out from our scoping review: activities of daily living, navigation, academic development. In almost all of the interventions discussed below, XR-based training was the main activity. Only in 4 studies, XR-based activities were implemented in combination

with other traditional activities. Lin et al. [27], for example, combined physical exercises on a treadmill with a VR-based exercise program.

Activities of daily living was the main domain of focus in three studies [18, 29, 30]. More specifically, Bridges et al. [18] examined the effects of AR apps addressing specific household tasks such as ironing, making your bed, and setting an alarm clock. Panerai et al. [29] tested the feasibility of VR apps for remote home-based rehabilitation aimed at teaching how to provide personal information taking medication at appropriate times packing a suitcase and grocery shopping. Finally, Aruanno [30] investigated the acceptability, usability, and satisfaction of a wearable MR device aimed at teaching people with neurodevelopmental disorders to perform simple domestic tasks such as garbage collection.

Four studies tackled abilities in the navigation domain [17, 31, 32, 33]. N’Kaoua et al. [20] explored the spatial deficits associated with ID and evaluated the effectiveness of various types of navigational aids in a virtual environment (i.e. signifying landmarks, such as a flashing yellow bench using arrows, or an elevated view). De la Torre et al. [33] tested the efficacy of a VR-based intervention aimed at enhancing visuospatial orientation skills of people with severe disabilities, that is, the ability to find their way and locate specific goals (e.g., find a specific room). Smith et al. [17] examined the effects of using an AR app to improve navigation skills among three students with ID, and finally, Cherix et al. [9] explored the usability of a VR immersive headset as a learning tool aimed at increasing the ability to cross roads using pedestrian crossings among young people with ID.

Three studies focused on academic development and skills in the special education context [34, 35, 36]. More specifically, Cakir et al. [34] explored the potential contribution of AR teaching materials for improving attention spans of children with special education needs. Pareto et al. [35] investigated the utility of a game to improve mathematical literacy among children with ID and the potential of an AR version of the game in enhancing game experience and learning. And finally, Byrne et al. [36] explored the impact of a VR learning programme designed for students with severe ID, for increasing their self-awareness, social and communication abilities, and levels of class participation.

The remaining studies addressed job skills [37], physical abilities or fitness [27, 28] time perception [7], and providing health care information [38].

3.2Features of XR content and technology used

Regarding the technology employed in the studies we reviewed, both custom-made and commercially available XR systems were used. The most common devices used for the XR experience included projectors [28, 32, 33, 35, 36], computer or television screens [37, 38, 39] handheld devices [37, 38, 39], head-mounted devices [7, 30, 31], and a gaming console [27]. Finally, one study used a combination of tablets and projection devices [34].

Regarding the content used in the XR interventions, four studies presented a replication of a realistic scenario in a VR environment [31, 32, 33, 38]. Cherix et al. [31] used a scene showing a sidewalk and a two-way street surrounded by buildings. Participants had to cross the road by using the available pedestrian crossings. In the scenario, five different parameters could be modified: the weather, the probability that a vehicle stops, the time a vehicle waits for the user to start crossing once stopped, the travel speed of the player, and the presence of visual and audio feedback in case of a collision between the participants and a vehicle. Similarly, N’Kaoua et al. [32] used a scene showing the area surrounding a hospital in Bordeaux (France). De la Torre et al. [33] showed a realistic rendering of the ground floor of an existing care facility. Finally, the virtual environment used in the study conducted by Hall et al. [38] consisted of a stylized hospital building giving access to different rooms. The hospital scenario incorporated a nurse who offered participants a tour of the hospital.

Another common content-related practice we found in three of the studies we reviewed [7] was the use of a diversity of realistic scenarios in which users had to complete different tasks that resembled real-life tasks. For instance, in the study conducted by Panerai et al. [29], participants had to place clothes in a suitcase, whereas in the study conducted by Eden [7], participants had to arrange different episodes belonging to the same scenario (e.g. baking a cake) in the correct order.

Also related to XR content, in two studies, games were used to stimulate participants to practice certain skills [27, 28]. Both studies involved training of physical skills. In the study conducted by Lin et al. [27], participants could choose between 15 different virtual games, such as boxing, or playing golf, table tennis, and frisbee.

Specifically regarding content in AR-based interventions in two studies researchers used AR tags (i.e. visual cues triggering the display of virtual information) [18, 39]. In the study conducted by Bridges et al. [18], users used laminated cars as AR tags to activate videos containing step-by-step instructions to complete daily tasks. Chang et al. [39], participants had to move AR tags representing food pictures on an LCD screen in the correct order to prepare a meal. In addition, in one study [35] users moved physical cubes containing sensors which are connected to a virtual world to select a corresponding card on the screen. Cakir et al. [34] explored the use of AR in creating teaching material aimed at teaching students about basic real life concepts, such as traffic lights and seasons; whereas Smith et al. [17] examined the use of an AR application in helping users to travel to unknown places. More specifically, they used a mobile application which combines Google Maps with AR features to enable real-time navigation prompts to users.

Finally, several papers reported on features of MR applications. In the study conducted by Aruanno et al. [30], the authors explored the use of HoloLearn, a MR application using HoloLens technology. HoloLens is a device for wearable MR which enables the user to interact with holograms that are placed in the real environment [30]. Aruanno et al. [30] tested an activity aiming to teach participants how to collect garbage properly by means of different holographic bins and some pieces of trash. Walker et al. [37] conducted their research in a laboratory that combines a physical space with simulated people. In this environment, participants were presented with an avatar seated at a desk in the virtual office to practice a job interview.

3.3Factors that impact implementation

This section provides an overview of the practical factors and issues related to the implementation of XR for people with ID that were reported by the authors of the papers we reviewed. We divided these factors in three categories: (1) factors related to users with moderate to severe ID, (2) factors related to caregivers/facilitators, and (3) factors related to the XR environment and system.

User-related factors were mentioned in 11 studies, with motivation being the factor most frequently cited by authors. In six papers the authors specifically reported that they selected participants based on their interest and willingness to participate [17, 18, 27, 29, 33, 39], while two papers reported difficulties with implementing the XR-based intervention because users refused to complete the session [28, 31]. In addition to motivation, four studies observed that cognition played an important role in understanding the goal of the XR-application as well as the instructions [7, 30, 31, 38], which suggest that the severity of the cognitive impairment should be considered in both XR-based intervention development and implementation. As a result of the cognitive impairment, these studies also stressed the importance of mediation (i.e. help needed in completing a certain task) and caregiver support in capturing users’ attention during a session [7, 30, 31, 38]. Finally, and perhaps contrary to what was to be expected, seven studies included a familiarization or training phase in which participants practiced using the XR system and explicitly reported that no major simulator sickness was experienced [18, 27, 29, 31, 32, 33, 39]. Simulator sickness is a syndrome similar to motion sickness, often experienced during simulator or another virtual reality (VR) exposure [40].

In 11 of the articles, authors stressed the importance of involving caregivers or facilitators to support, guide, and encourage participants while using the XR-systems. Four papers mentioned the inclusion of health behaviour experts (i.e. a child psychiatrist, a psychologist, a neuropsychologist) in the design process [18, 29, 30, 31]; whereas three studies involved a technical assistant to assure proper functioning of the XR system [28, 31, 37], and instruct participants on how to use the tablets [29]. Diving deeper into factors related to this supporting role of caregivers/facilitators, Byrne et al. [36] mentioned a positive therapeutic alliance between a therapist and users as an essential element for the learning experience. Hall et al. [40] speculated on how different facilitation styles might have affected the participants’ experience. Specifically, these authors suggested that the facilitator should not intervene too much since they observed that some participants lost interest or confidence when control was taken away. Instead, they argue that assertive facilitation is the ideal facilitation approach since it gives users access to all opportunities in the virtual environments, while negotiated methods are used to determine what they will do in the environment, and how much assistance they need and want. Finally, in two studies, caregivers were not provided with any training about the VR system in order to determine whether the program could be implemented by direct care staff supervised by a therapist, thus reducing the costs associated with such intervention [27, 28].

Factors related to the XR-system that were reported to affect implementation mostly addressed features that influenced the feasibility and acceptability of the XR-system. Such factors included the heaviness of the headset, motion sickness triggered by the virtual environment, and features related to the scenario that irritated some participants [31]. Aruanno et al. [30] encountered difficulties related to the interaction mechanisms of the HoloLearn system (i.e. gaze and air-tap), which proved to be too complex for people with severe ID. Hall et al. [38] noted that only four participants looked like their avatar, making it difficult to assess whether this affected identification with it. However, they did observe that full identification with the avatar did not seem to be highly important as only a few participants referred to it as “I”.

3.4Outcomes and lessons learned

In this section, we describe the main effects of the XR-interventions that were included in this scoping review. Although many authors of the studies emphasise the potential of XR-interventions for people with moderate to severe ID definitive conclusions about their interventions’ effects cannot be drawn due to several methodological reasons. Sample sizes were small; control groups were often missing, and long term effects were not measured. However, several authors shared important and relevant lessons learned in their papers, which we want to reflect on here as well.

Four of the selected studies described what worked well and what did not work, and provided lessons learned concerning the XR systems’ interfaces and design [30, 31, 33, 38]. Giving the heterogeneity of this population, one of the main recommendations based on these studies is to create scenarios with multiple configuration options allowing for adaptation to the specific needs of each individual user [30, 31]. In addition, due to the wide heterogeneity of the target population in terms of skills and needs it is considered important to incorporate people with ID in the design process of XR systems and content [31]. Moreover, some authors emphasized the importance of simplifying the stimulation settings and adapting the interface to educators and professionals in order to facilitate the application use [30, 31].

In two studies researchers noticed that visual similarities between the virtual world and the real world seemed to facilitate transfer of learning between contexts [33] and enhance enjoyment [38]. Likewise, Cherix et al. [31] emphasized that game elements (e.g. scores, badges, etc.) that do not exist in the real world may also represent an obstacle in this sense, indicating that realistic environments may be more effective when applied to VR interventions. In addition, the presence of a human-like “other” might contribute to the user experience. Additionally, even though an animated virtual assistant seemed effective at attracting the user’s attention and engaging them, it did not seem to have any functional benefit when the disability severity is high [30].

Finally, Aruanno et al. [30] provided two different solutions to mitigate the difficulties faced by individuals with severe ID when interacting with complex interaction mechanisms (i.e. gaze and air-tap). The first solution consists of training users to use the interaction mechanisms over a long period of time, whereas the second solution involves changes in the device such as replacing the default white cursor with a purple one which is more visible and intuitive and using a physical button instead of the air-tap interaction mechanism. Finally, Cherix et al. [9] recommended reducing the speed of the movements and avoiding re-creating head movements in VR environments to prevent motion sickness.