A novel processing pipeline for optical multi-touch surfaces

Abstract

In this thesis a new approach for touch detection on optical multi-touch devices is proposed that exploits the fact that the camera images reveal not only the actual touch points but also objects above the screen such as the hand or arm of a user. The touch processing relies on the Maximally Stable Extremal Regions algorithm for finding the users' fingertips in the camera image. The hierarchical structure of the generated extremal regions serves as a starting point for agglomerative clustering of the fingertips into hands. Furthermore, a heuristic is suggested that supports the identification of individual fingers as well as the distinction between left hands and right hands if all five fingers of a hand are in contact with the touch surface. The evaluation confirmed that the system is robust against detection errors resulting from non-uniform illumination and reliably assigns touch points to individual hands based on the implicitly tracked context information. The efficient multi-threaded implementation handles two-handed input from multiple users in real-time.