Delivering script therapy for people with aphasia in EVA park: Two single case treatment studies

2.1Participants

Keats and Austen (Pseudonyms) were recruited. They met the following selection criteria: aphasia following a stroke that occurred at least 4 months prior to the study; fluent pre-stroke users of English (both were monolingual); not receiving speech and language therapy elsewhere during the study; no severe impairments of hearing or vision and no additional diagnosis affecting cognition. As the facilitation techniques for Script Therapy involve repetition, they were required to score above 50% on the single word repetition subtest of the Comprehensive Aphasia Test (CAT 12; Swinburn, Porter & Howard, 2004). Repetition of complex words (CAT 13) and sentences (CAT 16) was also screened. They needed to demonstrate poor connected speech (screened via the CAT picture description task and clinical judgement) and be motivated to improve this aspect of their aphasia.

Participant details are provided in Table 1 and connected speech samples are in Table 2. Both participants met the selection criteria with respect to single word repetition. However, Keats showed poor repetition of complex words and sentences (CAT subtests 13 & 16).

2.2Design

The study employed a single case repeated measures design, in which data from each participant were analysed separately. Assessments were administered at four time points (T1 –T4) each separated by a period of five weeks. Therapy was administered between T2 and T3. Thus, each participant was assessed twice before therapy, providing a double baseline, once immediately after therapy and once five weeks later (follow up). Assessments were administered face to face (i.e. not in EVA Park) by a non-treating therapist in the case of Keats and by a non-treating student of speech and language therapy in the case of Austen. Assessments were filmed and scoring of the assessments was conducted blind to time point. This was accomplished by presenting the films in random and blinded order to the scorer (who was not the assessor).

2.3Measures

RQ1 Feasibility of delivering Script Therapy in EVA Park was assessed through compliance and fidelity data. The percentage of treatment sessions attended and completion of generalisation practice was recorded (via self-report). Eleven treatment sessions (27.5% of the total) were filmed, 7 with Keats and 4 with Austen. These films were subject to fidelity checking, conducted by two students (authors PT and KC), who were not otherwise involved in the project. A checklist of core treatment components was devised, based on the therapy protocol (see Supplementary Materials 1). Each student rated the 11 recorded sessions of therapy against the list, indicating whether a component was fully present, partially present, absent or not applicable. The latter rating was given if a component was not relevant to an observed session. For example, rehearsal of previously practised scripts was not relevant in early sessions. The ratings provided a fidelity score for treatment delivery (the percentage of rated components that were fully present). Inter-rater reliability of the checklist was assessed, using Cohen’s Kappa statistic.

RQ2 To explore acceptability, each participant was interviewed by their non-treating assessor at T3. Questions covered views about EVA Park, navigational aspects, being represented by an avatar, receiving therapy in EVA Park, relationship with the treating therapist, and any perceived benefits from therapy. Eleven questions also elicited rating responses. Interviews were recorded and transcribed. Data were analysed thematically (Braun & Clark, 2006).

RQ3 & 4 Five personalised scripts were developed with each participant prior to T1 (see below). An elicitation question for each was also agreed, for example: ‘Can you tell me what you would say to introduce yourself at the stroke group?’ At each assessment point, participants were asked to produce the five scripts, in random order, in response to the elicitation questions. No further cues were given, beyond invitations to continue.

Two scores were extracted from the scripts at each assessment point. One was the percentage of script related words. This was the number of words produced from each script expressed as a percentage of the total word count. All words were counted, i.e., content and function words. Only verbatim realisations were scored (e.g., not synonyms). Phonetic/phonological errors were accepted, providing these were recognisable attempts at the target. The other score was the speech rate. The time taken to produce each script was recorded, extracting any time taken by the tester. The total number of words produced in that period was recorded and expressed as a word per minute (wpm) score. In line with previous research (e.g., Youmans et al., 2005) any real word contributed to the wpm score, not just script words. Scripts were trained in a random order until a criterion was reached (see intervention section). Keats was accordingly trained on two scripts and Austen on four. Scores were therefore available on both trained scripts (RQ3) and untrained scripts (RQ4).

RQ5 A sample of connected speech was taken at each assessment point in the form of a personal narrative. Each participant was asked to recount a story that was personally meaningful, using consistent, scripted prompts. No time limit was imposed. If production was limited the participants were encouraged to continue or add more information. Samples were recorded and transcribed and 3 measures were extracted, using criteria from the Quantitative Production Analysis (Rochon, Saffran, Berndt & Schwartz, 2000). These were: Number of words; Number of utterances; Number of well-formed sentences.

Communication Activities of Daily Living (CADL-2, Holland, Frattali & Fromm, 1999) was administered at each assessment point to investigate any changes in functional communication.

2.4Script development

Following recommendations in the literature (Holland, Halper & Cherney, 2010) personally relevant scripts were developed with each participant. Participants were invited to think about different communication scenarios in their lives and evaluate them for importance and difficulty. From this discussion five scenarios were prioritised for each participant, which formed the basis for their scripts. For example, one of Austen’s scripts focussed on her FaceTime conversations with her grandsons; while Keats developed a script for talking with stroke survivors which was important for his hospital volunteering role. The wording of each script was drafted by the treating therapist, with the participants giving direction and feedback on each phrase. Austen’s scripts were also reviewed by her husband, who confirmed that the wording reflected what she might have said pre-stroke. Script development took place face to face, not in EVA Park, over one session.

2.5Intervention

Prior to the intervention, EVA Park was set up on each participants’ home computer. They also developed their avatar, making choices about all aspects of appearance. Each participant was given about 30 minutes training in how to use EVA Park. Set up, avatar creation and training was supported face to face by the treating therapist or the student/therapist assessor.

Participants were offered 20, one hour intervention sessions over 5 weeks (4 sessions per week). Sessions were delivered 1:1 by authors JB and ND. All sessions were delivered in EVA Park, with the participant and therapist represented as an avatar. Participants worked at home and the treating therapist worked either from their home (JB) or from the University (ND).

Scripts were randomly assigned a number between 1 and 5, which determined the order in which they were trained. The training protocol was developed from published accounts of Script Therapy (Youmans et al., 2005; Goldberg et al., 2012). Each script was worked on one phrase at a time. First the therapist modelled each phrase for repetition. This was followed by choral reading in which the therapist and participant read the phrase aloud together. Then the participant produced the phrase without the therapist. Problem words, that were not achieved, were cued via repetition, first phoneme cues or syllabic segmentation. Once achieved they were repeated 5 times before the whole phrase was practised again. EVA Park contains a note card facility, which allows written material to be available on the screen. This was used to provide the written text of scripts while they were being practised. The therapist also used an Instant Messaging facility in EVA Park, to type problem words as they were being rehearsed.

Each phrase in a script had to be achieved independently at least 10 times, before progressing to the next phrase. Mastery of a script was defined as 10/20 independent productions of the whole script. Once this was achieved, the next script was introduced. Previously mastered script(s) were revisited at the end of each session, to promote maintenance of learning. Keats received training on two scripts, reaching criterion on one. It took 12 sessions for him to reach criterion. Austen received training on 4 scripts, reaching criterion on three. The number of sessions taken by Austen to reach criterion on each script ranged between 4 and 6.

2.6Generalisation practice

Scripts were practiced in varying locations in EVA Park. Both participants were required to practice their scripts with different conversation partners. Keats received one session a week in EVA Park with a student of speech and language therapy (this was included in the 20 hours of intervention) with whom he rehearsed his scripts. Austen was encouraged to practise her scripts outside EVA Park with her husband. She received a text message of her current script and was asked to practise this for 15 minutes per day.

Participants also had the option to rehearse their script with ‘Ruby Robot’. Ruby Robot was an automated avatar in EVA Park. When encountered she offered an opportunity to practice, with a pre-recorded invitation: ‘Hello, I’m Ruby Robot. Do you want to practice your script with me?’ No further facilitation or feedback was provided, e.g., Ruby Robot did not employ speech recognition and did not contribute to any dialogue. Participants were made aware of Ruby Robot, and her limitations. She appeared with a label, which identified her as a robot.

Script training was supplemented with at least 10 minutes of open conversation per session. This focused on topics that were both related and unrelated to the scripts. Appropriate settings in EVA Park were employed for the practice. For example, a conversation about cooking took place in the EVA Park house kitchen.

See Supplementary Materials 2 for a description of the intervention based on the Template for Intervention Description and Replication (TIDieR; Hoffmann et al., 2014).

3.1RQ1: Feasibility

Both participants completed all stages of data collection. Keats attended all scheduled therapy sessions. Austen missed three, one due to ill health and two because of internet connectivity problems. Keats attended all practice sessions with the student of speech and language therapy and also carried out independent practice, in which he listened to recordings of his script (produced by the therapist) while driving his car. Austen in contrast, indicated that she rarely practised outside the therapy sessions, despite being invited to do so with her husband. Both reported that they made no use of Ruby Robot.

132 treatment components were assessed for fidelity (12 components over 11 sessions). Of these, 14 were rated as not applicable; 81.4% of the remaining components were rated as fully present, and a further 14.4% were rated as partially present. Fidelity scores for individual sessions ranged from 72.7% to 91.6% (percentage of components that were fully present). All sessions were independently rated by two assessors. Percentage agreement between the two raters was 81.8% . The Kappa value was.63 (p <  0.001). According to the benchmarks set by Landis and Koch (1977) this equates to ‘substantial agreement’.

3.2RQ2: Acceptability

Four main themes emerged from the interview data and associated ratings (see Table 3). The first theme related to the overall response to EVA Park. Here the ratings and views of both participants were extremely positive:

Both participants indicated that they would recommend EVA Park to other stroke survivors with aphasia. Austen enjoyed using EVA Park even when she was on her own. This was not true for Keats who gave a low rating for lone use. His main criticism was the lack of company:

Keats: Nothing. I can be . . . You can go . . . computer, nothing. Nobody there.

The second theme related to the experience of receiving therapy in EVA Park. Again this was highly rated, with associated positive comments. Keats singled out his therapy sessions as the most enjoyable aspect of the experience:

Austen flagged her therapist for praise, describing her as ‘brilliant’

Keats indicated that he would value more features in EVA Park to support independent practice of his scripts, such as audio recordings. He described having to record himself using his phone. He also found the therapy sessions tiring:

‘It’s . . . one hour. She’s talking. She’s.. bloody . . . knackered [points to self]. My . . . I was . . . I was sleeping. After’ (Keats)

The third theme related to usage and technical features. Navigational aspects, such as moving the avatar, and using the mouse caused no difficulties. There were no negative responses to being represented by an avatar. Both participants identified some technical problems, Keats with sound during early sessions and Austen with internet connectivity.

The final theme related to perceived impacts of the intervention. Both participants detected changes in their communication, e.g.:

‘Speech flows better’ (Austen)

‘It’s . . . it’s talking. It’s . . . it’s . . . it’s really happy’ (Keats)

Keats also gave an example of being able to use one of his scripts at his stroke group.

3.3RQ3 & 4: Improvement in trained and untrained scripts

Table 4 reports the percentage of script related words achieved on each script at each time point. Taking Keats first, the table suggests that the production of words in treated scripts increased at T3 (immediately post therapy), but this gain was not maintained at T4. This was tested statistically by using McNemar chi square to compare the number of correct words achieved across the treated scripts at paired time points. The analysis revealed no significant difference over the baseline period (T1 vs T2 McNemar chi square p = 0.15), but a significant increase following therapy (T2 vs T3, McNemar chi square, p <  0.001). This increase was not maintained at T4 (T2 vs T 4, McNemar chi square, p = 0.45). A similar analysis was carried out on the pooled data from the untreated scripts. Here none of the findings was significant (T1 vs T2, McNemar, p = 0.45; T2 vs T3 McNemar chi square p = 0.34; T2 vs T4, McNemar chi square p = 1). Thus, Keats demonstrated a significant improvement in the production of script related words. However, this improvement was confined to treated scripts and was not maintained at the follow up assessment.

Turning to Austen, across the pooled treated script data there was no change in the number of correct words over the baseline period (T1 vs T2 McNemar Chi Square p = 0.344). Scores rose significantly at T3 (T2 vs T3 McNemar Chi Square, p <  0.001) and this change was maintained at T4 (T2 vs T4 McNemar Chi Square p <  0.001). The realisation of words in the one untreated script did not change significantly over time (T2 vs T3, McNemar chi square p >  0.5; T2 vs T4 McNemar chi square p >  0.5). Thus, Austen demonstrated a significant and well-maintained improvement in the production of script related words, but this was confined to treated scripts.

Speech rate data are also reported in Table 4. These were explored descriptively. Keats’ rate increased on one treated script at T3 (Script 1) but not on the other (Script 2). His speaking rate on the untreated scripts was similarly variable. Two were produced more fluently immediately after therapy (Scripts 4 and 5), but one was not (Script 3). His speaking rates at T4 showed little change from the baseline values. Austen’s speech rate increased at T3 and T4 on two treated scripts (Scripts 2 and 4). However, this was not the case for the other two treated scripts (Scripts 1 and 3), or for the untreated script (Script 5). Thus, Script Therapy did not consistently impact on the speaking rate of either participant.

3.4RQ5: Improvements in narrative speech and functional communication

Table 5 reports the three analysed values from the personal narratives. These data offer little evidence of therapy induced change, for either participant.

Table 6 reports results from CADL-2 (Holland et al, 1999). A therapy effect cannot be argued for Keats. Austen’s raw score increased by 7 points at T3, following a stable baseline, and that gain was maintained at T4.