Sign language recognition with transformer networks

Sign languages are complex languages. Research into them is ongoing, supported by large video corpora of which only small parts are annotated. Sign language recognition can be used to speed up the annotation process of these corpora, in order to aid research into sign languages and sign language recognition. Previous research has approached sign language recognition in various ways, using feature extraction techniques or end-to-end deep learning. In this work, we apply a combination of feature extraction using OpenPose for human keypoint estimation and end-to-end feature learning with Convolutional Neural Networks. The proven multi-head attention mechanism used in transformers is applied to recognize isolated signs in the Flemish Sign Language corpus. Our proposed method significantly outperforms the previous state of the art of sign language recognition on the Flemish Sign Language corpus: we obtain an accuracy of 74.7% on a vocabulary of 100 classes. Our results will be implemented as a suggestion system for sign language corpus annotation

What the eye does not see: visualizations strategies for the data collection of personal networks

The graphic representation of relational data is one of the central elements of social network analysis. In this paper, the author describe the use of visualization in interview-based data collection procedures designed to obtain personal networks information, exploring four main contributions. First, the author shows a procedure by which the visualization is integrated with traditional name generators to facilitate obtaining information and reducing the burden of the interview process. Second, the author describes the reactions and qualitative interpretation of the interviewees when they are presented with an analytical visualization of their personal network. The most frequent strategies consist in identifying the key individuals, dividing the personal network in groups and classifying alters in concentric circles of relative importance. Next, the author explores how the visualization of groups in personal networks facilitates the enumeration of the communities in which individuals participate. This allows the author to reflect on the role of social circles in determining the structure of personal networks. Finally, the author compares the graphic representation obtained through spontaneous, hand-drawn sociograms with the analytical visualizations elicited through software tools. This allows the author to demonstrate that analytical procedures reveal aspects of the structure of personal networks that respondents are not aware of, as well as the advantages and disadvantages of using both modes of data collection. For this, the author presents findings from a study of highly skilled migrants living in Spain (n = 95) through which the author illustrates the challenges, in terms of data reliability, validity and burden on both the researcher and the participants

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterances in a dialogue. In an effort to model this kind of generative process, we propose a neural network-based generative architecture, with latent stochastic variables that span a variable number of time steps. We apply the proposed model to the task of dialogue response generation and compare it with recent neural network architectures. We evaluate the model performance through automatic evaluation metrics and by carrying out a human evaluation. The experiments demonstrate that our model improves upon recently proposed models and that the latent variables facilitate the generation of long outputs and maintain the context.Comment: 15 pages, 5 tables, 4 figure