Drift robust non-rigid optical flow enhancement for long sequences

Issue title: Multimedia in technology enhanced learning

Guest editors: Zhihan Lv

Article type: Research Article

Authors: Li, Wenbin^{a; b; *} | Cosker, Darren^b | Brown, Matthew^{b; c}

Affiliations: [a] Department of Computer Science, University College London, UK | [b] Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA), University of Bath, UK | [c] Google, Mountain View, CA, US

Correspondence: [*] Corresponding author. Wenbin Li. Tel.: +44 (0)20 3108 7134; Fax: +44 (0)20 7387 1397; E-mail: [email protected].

Abstract: It is hard to densely track a nonrigid object in long term, which is a fundamental research issue in the computer vision community. This task often relies on estimating pairwise correspondences between images over time where the error is accumulated and leads to a drift. In this paper, we introduce a novel optimisation framework with an Anchor Patch constraint. It is supposed to significantly reduce overall errors given long sequences containing nonrigidly deformable objects. Our framework can be applied to any dense tracking algorithm, e.g. optical flow. We demonstrate the success of our approach by showing significant error reduction on 6 popular optical flow algorithms applied to a range of realworld nonrigid benchmarks. We also provide quantitative analysis of our approach given synthetic occlusions and image noise.

Keywords: Computer vision, dense tracking, anchor patch, optical flow, drift, long sequence

DOI: 10.3233/JIFS-169099

Journal: Journal of Intelligent & Fuzzy Systems, vol. 31, no. 5, pp. 2583-2595, 2016