Sign language recognition with transformer networks

Sign languages are complex languages. Research into them is ongoing, supported by large video corpora of which only small parts are annotated. Sign language recognition can be used to speed up the annotation process of these corpora, in order to aid research into sign languages and sign language recognition. Previous research has approached sign language recognition in various ways, using feature extraction techniques or end-to-end deep learning. In this work, we apply a combination of feature extraction using OpenPose for human keypoint estimation and end-to-end feature learning with Convolutional Neural Networks. The proven multi-head attention mechanism used in transformers is applied to recognize isolated signs in the Flemish Sign Language corpus. Our proposed method significantly outperforms the previous state of the art of sign language recognition on the Flemish Sign Language corpus: we obtain an accuracy of 74.7% on a vocabulary of 100 classes. Our results will be implemented as a suggestion system for sign language corpus annotation

Theory of spiral wave dynamics in weakly excitable media: asymptotic reduction to a kinematic model and applications

In a weakly excitable medium, characterized by a large threshold stimulus, the free end of an isolated broken plane wave (wave tip) can either rotate (steadily or unsteadily) around a large excitable core, thereby producing a spiral pattern, or retract causing the wave to vanish at boundaries. An asymptotic analysis of spiral motion and retraction is carried out in this weakly excitable large core regime starting from the free-boundary limit of the reaction-diffusion models, valid when the excited region is delimited by a thin interface. The wave description is shown to naturally split between the tip region and a far region that are smoothly matched on an intermediate scale. This separation allows us to rigorously derive an equation of motion for the wave tip, with the large scale motion of the spiral wavefront slaved to the tip. This kinematic description provides both a physical picture and exact predictions for a wide range of wave behavior, including: (i) steady rotation (frequency and core radius), (ii) exact treatment of the meandering instability in the free-boundary limit with the prediction that the frequency of unstable motion is half the primary steady frequency (iii) drift under external actions (external field with application to axisymmetric scroll ring motion in three-dimensions, and spatial or/and time-dependent variation of excitability), and (iv) the dynamics of multi-armed spiral waves with the new prediction that steadily rotating waves with two or more arms are linearly unstable. Numerical simulations of FitzHug-Nagumo kinetics are used to test several aspects of our results. In addition, we discuss the semi-quantitative extension of this theory to finite cores and pinpoint mathematical subtleties related to the thin interface limit of singly diffusive reaction-diffusion models

A spatially distributed model for foreground segmentation

Foreground segmentation is a fundamental first processing stage for vision systems which monitor real-world activity. In this paper we consider the problem of achieving robust segmentation in scenes where the appearance of the background varies unpredictably over time. Variations may be caused by processes such as moving water, or foliage moved by wind, and typically degrade the performance of standard per-pixel background models. Our proposed approach addresses this problem by modeling homogeneous regions of scene pixels as an adaptive mixture of Gaussians in color and space. Model components are used to represent both the scene background and moving foreground objects. Newly observed pixel values are probabilistically classified, such that the spatial variance of the model components supports correct classification even when the background appearance is significantly distorted. We evaluate our method over several challenging video sequences, and compare our results with both per-pixel and Markov Random Field based models. Our results show the effectiveness of our approach in reducing incorrect classifications