Interactive activity recognition and prompting to assist people with cognitive disabilities

Abstract

This paper presents a model of interactive activity recognition and prompting for use in an assistive system for persons with cognitive disabilities. The system can determine the user's state by interpreting sensor data and/or by explicitly querying the user, and can prompt the user to begin, resume, or end tasks. The objective of the system is to help the user maintain a daily schedule of activities while minimizing interruptions from questions or prompts. The model is built upon an option-based hierarchical POMDP. Options can be programmed and customized to specify complex routines for prompting or questioning.

The paper proposes a heuristic approach to solving the POMDP based on a dual control algorithm using selective-inquiry that can appeal for help from the user explicitly when the sensor data is ambiguous. The dual control algorithm is working effectively in the unified control model which features the adaptive option and robust state estimation. Simulation results show that the unified dual control model achieves the best performance and efficiency comparing with various alternatives. To further demonstrate the system's performance, lab experiments have been carried out with volunteer actors performing a series of carefully designed scenarios with different kinds of interruption cases. The results show that the system is able to successfully guide the agent through the sample schedule by delivering correct prompts while efficiently dealing with ambiguous situations.