Software support for multitouch interaction: the end-user programming perspective

Empowering users with tools for developing multitouch interaction is a promising step toward the materialization of ubiquitous computing. This survey frames the state of the art of existing multitouch software development tools from an end-user programming perspective.This research has been partially funded by the EUFP7 project meSch (grant agreement 600851 and CREAx grant (Spanish Ministry of Economy and Competitivity TIN2014-56534-R

Learning to Recognize Touch Gestures: Recurrent vs. Convolutional Features and Dynamic Sampling

International audienceWe propose a fully automatic method for learning gestures on big touch devices in a potentially multiuser context. The goal is to learn general models capable of adapting to different gestures, user styles and hardware variations (e.g. device sizes, sampling frequencies and regularities). Based on deep neural networks, our method features a novel dynamic sampling and temporal normalization component, transforming variable length gestures into fixed length representations while preserving finger/surface contact transitions, that is, the topology of the signal. This sequential representation is then processed with a convolutional model capable, unlike recurrent networks, of learning hierarchical representations with different levels of abstraction. To demonstrate the interest of the proposed method, we introduce a new touch gestures dataset with 6591 gestures performed by 27 people, which is, up to our knowledge, the first of its kind: a publicly available multi-touch gesture dataset for interaction. We also tested our method on a standard dataset of symbolic touch gesture recognition, the MMG dataset, outperforming the state of the art and reporting close to perfect performance

Learning to recognize touch gestures: recurrent vs. convolutional features and dynamic sampling

We propose a fully automatic method for learning gestures on big touch devices in a potentially multi-user context. The goal is to learn general models capable of adapting to different gestures, user styles and hardware variations (e.g. device sizes, sampling frequencies and regularities). Based on deep neural networks, our method features a novel dynamic sampling and temporal normalization component, transforming variable length gestures into fixed length representations while preserving finger/surface contact transitions, that is, the topology of the signal. This sequential representation is then processed with a convolutional model capable, unlike recurrent networks, of learning hierarchical representations with different levels of abstraction. To demonstrate the interest of the proposed method, we introduce a new touch gestures dataset with 6591 gestures performed by 27 people, which is, up to our knowledge, the first of its kind: a publicly available multi-touch gesture dataset for interaction. We also tested our method on a standard dataset of symbolic touch gesture recognition, the MMG dataset, outperforming the state of the art and reporting close to perfect performance.Comment: 9 pages, 4 figures, accepted at the 13th IEEE Conference on Automatic Face and Gesture Recognition (FG2018). Dataset available at http://itekube7.itekube.co

Supporting Collaborative Multi-User Interactions in a Video Surveillance Application Using Microsoft Tabletop Surface

This report examines the progressing exploration done on my chosen subject, which is A Multi-touch Interface in a Video Surveillance System. It discusses method of early prototype interacting with security surveillance footage using natural user interfaces instead of traditional mouse and keyboard interaction. Current project is an evidence of idea on exhibiting that multi-touch interfaces are helpful in a video surveillance system to specifically control the surveillance videos, both of the live or of recorded. In case of any occurrence, this proposed system of interaction may require the user to spend an extra time amounts time obtaining circumstantial and location awareness, which is counter-beneficial. The framework proposed in this paper show how a multi-touch screen and natural interaction can empower the surveillance observing station users to rapidly recognize the area of a security camera and proficiently react to an occurrence. One of the main objective of this project is to engage more than 1 user to perform moving, scaling, rotating ,highlighting and recording on a surveillance video in the meantime, particularly during emergency periods. Furthermore, the scope of study for this project is to improve user collaborative interactions on Microsoft tabletop surface .A methodology was developed based upon a combination of the available literature and the experiences of the authors, who are actively involved with the development of multi-user interactions. This will cover many parts such as surveys, data gathering from respective Subject-Matter Experts, focal points, and analyzing information. It is intended to have a Surveillance application with user friendly collaborative touch on surface and eye-catching interface to reflect the quick paced nature of today's correspondences and better advertise its new activities and accessible assets. The future improvements and plans have been recommended and discussed in the recommendations section. Up to now, this research report has been run for twelve (12) weeks and going to be continued running for sixteen (16) weeks with a specific end goal to attain project primary objectives

MT4j - A Cross-platform Multi-touch Development Framework

This article describes requirements and challenges of crossplatform multi-touch software engineering, and presents the open source framework Multi-Touch for Java (MT4j) as a solution. MT4j is designed for rapid development of graphically rich applications on a variety of contemporary hardware, from common PCs and notebooks to large-scale ambient displays, as well as different operating systems. The framework has a special focus on making multi-touch software development easier and more efficient. Architecture and abstractions used by MT4j are described, and implementations of several common use cases are presented.Comment: ACM EICS 2010, Workshop: Engineering patterns for multi-touch interfaces (2010), p. 52-5