An exploratory case study on letter-based, head-movement-driven communication

Article type: Research Article

Authors: Miksztai-Réthey, Brigitta^{a; *} | Faragó, Kinga Bettina^b

Affiliations: [a] Bárczi Gusztáv Faculty of Special Education, Eötvös Loránd University, Budapest, Hungary | [b] Faculty of Informatics, Eötvös Loránd University, Budapest, Hungary

Correspondence: [*] Corresponding author: Brigitta Miksztai-Réthey, Bárczi Gusztáv Faculty of Special Education, Eötvös Loránd University, H-1097, Ecseri út 3, Budapest, Hungary. Tel.: +36 13483137; E-mail: [email protected].

Abstract: BACKGROUND: With alternative and augmentative communication (AAC) people with complex communication needs (CCN) become more independent and express themselves to the fullest extent possible. In finding the best AAC solution, mobile technology and ICT (information and communications technology) provide new opportunities every day. Although a wide range of assistive technologies (AT) are available, matching person and technology (MPT) and setting the optimal parameters individually are essential. For an AAC solution to be optimal for letter-based communication it has to be easy-to-use, comfortable, and fast. OBJECTIVES: For people with severe speech and physical impairments (SSPI), one method to interact with a computer is using head-movement-driven mouse. There are different on-screen devices available for typing via head movements, and much work has been done to compare them in terms of the time required for typing. Dasher is one of the fastest software tools with a setting option for zooming speed. An optimistic initial model (OIM) based on Markov decision process (MDP) has already been shown to optimize this zooming speed for increasing the typing efficiency of persons without SSPI. Since this reinforcement learning component has so far been tested on neurotypical users only (e.g., research assistants), in the present case study we involved a user with SSPI. Our question was whether the algorithm can optimize its own parameters in these circumstances. METHODS: To document all relevant aspects of the human-computer interaction log files, screen and webcam videos were collected. These input data were later analyzed with mathematical methods based on the OIM reward systems feedbacks. In addition, manual interpretation using semi-supervised machine video annotation was carried out for analyzing screen events and user behaviors. RESULTS: The human annotations of the recorded video data indicated that the participant had at least two different typing strategies. In contrast with the data from a previous study, in our study the artificial intelligence (AI) component was unable to find optimal settings similar to those attained when only one typing strategy was used by subjects without SSPI. CONCLUSIONS: To maximize communication efficiency, a more complex assistive tool may be more appropriate. Closer cooperation between different areas of expertise is suggested in order to achieve solutions employing various methods.

Keywords: Augmentative and alternative communication (AAC), assistive technology (AT), head movement driven typing, self-improvement, matching person and technology (MPT)

DOI: 10.3233/TAD-160163

Journal: Technology and Disability, vol. 29, no. 4, pp. 153-161, 2018