Sign Language Recognition

This chapter covers the key aspects of sign-language recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data set

Towards faster activity search using embedding-based subsequence matching

Event search is the problem of identifying events or activity of interest in a large database storing long sequences of activity. In this paper, our topic is the problem of identifying activities of interest in databases where such activities are represented as time series. In the typical setup, the user presents a query that represents an activity of interest, and the system needs to retrieve the most similar activities stored in the database. We focus on the case where the best database matches are not segmented a priori: the database contains representations of long, continuous activity, that occurs throughout relatively extensive periods of time, and, given a query, there are no constraints as to when exactly a database match starts and ends within the longer activity pattern where it is contained. Using the popular DTW measure, the best database matches can be found using dynamic programming. However, retrieval time is linear to the size of the database and can become too long as the database size becomes larger. To achieve more efficient retrieval time, we apply to this problem a recently proposed technique called Embedding-based Subsequence Matching (EBSM), and we demonstrate that using EBSM we can obtain significant speedups in retrieval time

Modification of Colors in Images for Enhancing the Visual Perception of Protanopes

Part 2: MHDW WorkshopInternational audienceThe purpose of this paper is to design and to implement proper color modification methods for enhancing visual perception of digital images for people with protanopia (one of the types of Color Vision Deficiency). This paper proposes one simulation method and two daltonization methods for the modification of digital colored images, implementing the additional step of quantization for a faster modification of colors which are not correctly perceived. In order to avoid color confusion a color checking module is implemented and optimal daltonization parameters are selected. The ultimate objective of this paper is to minimize the total processing time to such an extent that it is possible to apply the proposed methods to real-time sequences (online video). The entire application is tested in a set of artificial and real images, while the performance of the proposed algorithms is evaluated in an appropriately selected set of colored images

A conditional random field-based model for joint sequence segmentation and classification

In this paper, we consider the problem of joint segmentation and classification of sequences in the framework of conditional random field (CRF) models. To effect this goal, we introduce a novel dual-functionality CRF model: on the first level, the proposed model conducts sequence segmentation, whereas, on the second level, the whole observed sequences are classified into one of the available learned classes. These two procedures are conducted in a joint, synergetic fashion, thus optimally exploiting the information contained in the used model training sequences. Model training is conducted by means of an efficient likelihood maximization algorithm, and inference is based on the familiar Viterbi algorithm. We evaluate the efficacy of our approach considering a real-world application, and we compare its performance to popular alternative

Hand pose estimation from a single RGB-D image

Abstract. Hand pose estimation is an important task in areas such as human computer interaction (HCI), sign language recognition and robotics. Due to the high variability in hand appearance and many degrees of freedom (DoFs) of the hand, hand pose estimation and tracking is very challenging, and different sources of data and methods are used to solve this problem. In the paper, we propose a method for model-based full DoF hand pose estimation from a single RGB-D image. The main advantage of the proposed method is that no prior manual initialization is required and only very general assumptions about the hand pose are made. Therefore, this method can be used for hand pose estimation from a single RGB-D image, as an initialization step for subsequent tracking, or for tracking recovery.