Towards activity recognition from eye-movements using contextual temporal learning

Article type: Research Article

Authors: Martinez, Francis^a | Pissaloux, Edwige^{a; *} | Carbone, Andrea^{a; b}

Affiliations: [a] Institut de Systèmes Intelligents et de Robotique, Sorbonne Université, Paris, France | [b] CHArt, Université Paris 8, Paris, France

Correspondence: [*] Corresponding author: Edwige Pissaloux, Institut de Systèmes Intelligents et de Robotique, Sorbonne Université, 4 place Jussieu, 75005 Paris, France. E-mail:[email protected]

Abstract: Patterns of gaze play an important role in video analysis and understanding human behavior. This paper proposes a novel approach to solve the problem of activity recognition from first-person gaze movements. Novel n-gram statistical features are extracted from these movements via a multi scale and temporal representation scheme. A joint classification and segmentation of activities and a spatio-temporal contextual learning approach built upon confidence values from long-range neighborhoods reinforces the novelty and efficiency of the proposed approach. Experimental results show that the proposed method improves by 18% on the current baseline.

Keywords: Activity recognition, first-person (gaze) vision, multi scale and temporal features, contextual learning, SVM

DOI: 10.3233/ICA-160520

Journal: Integrated Computer-Aided Engineering, vol. 24, no. 1, pp. 1-16, 2017