Control de un sistema de información para bomberos mediante gestos de la mano

[ES] El trabajo propuesto va dirigido a la implementación de un guante con seis acelerómetros, uno para cada dedo y uno en la palma de la mano, controlados por un ESP32. La información será comunicada de manera inalámbrica a la CPU para poder identificar diferentes gestos mediante una red neuronal para controlar un sistema de información incorporado en el equipamiento de un bombero. El propósito es proveer al agente de un interfaz compatible con el uniforme y el guante que llevara cuando este en operaciones de emergencia. En el sistema final, la CPU estará incorporada en un dispositivo en el traje, pero para este trabajo se empleara una CPU genérica con el objetivo de desarrollar, entrenar y evaluar la red.[EN] The proposed work aims to implement a glove with six accelerometers, one for each finger and one in the palm of the hand, managed by an esp32. The information is communicated wirelessly to a CPU in which various gestures will be identified through a neural network to control an information system incorporated into the equipment of a firefighter. The purpose is to provide the agent with an interface compatible with the suit and gloves that he must wear when they are in an emergency operation. In the final system, the cpu will be embedded in a wearable system incorporated into the suit, but in this work, a generic cpu will be used to develop the network, train it and evaluate it.Paracuellos De Los Santos, D. (2020). Control de un sistema de información para bomberos mediante gestos de la mano. Universitat Politècnica de València. http://hdl.handle.net/10251/156883TFG

Motion-capture-based hand gesture recognition for computing and control

This dissertation focuses on the study and development of algorithms that enable the analysis and recognition of hand gestures in a motion capture environment. Central to this work is the study of unlabeled point sets in a more abstract sense. Evaluations of proposed methods focus on examining their generalization to users not encountered during system training. In an initial exploratory study, we compare various classification algorithms based upon multiple interpretations and feature transformations of point sets, including those based upon aggregate features (e.g. mean) and a pseudo-rasterization of the capture space. We find aggregate feature classifiers to be balanced across multiple users but relatively limited in maximum achievable accuracy. Certain classifiers based upon the pseudo-rasterization performed best among tested classification algorithms. We follow this study with targeted examinations of certain subproblems. For the first subproblem, we introduce the a fortiori expectation-maximization (AFEM) algorithm for computing the parameters of a distribution from which unlabeled, correlated point sets are presumed to be generated. Each unlabeled point is assumed to correspond to a target with independent probability of appearance but correlated positions. We propose replacing the expectation phase of the algorithm with a Kalman filter modified within a Bayesian framework to account for the unknown point labels which manifest as uncertain measurement matrices. We also propose a mechanism to reorder the measurements in order to improve parameter estimates. In addition, we use a state-of-the-art Markov chain Monte Carlo sampler to efficiently sample measurement matrices. In the process, we indirectly propose a constrained k-means clustering algorithm. Simulations verify the utility of AFEM against a traditional expectation-maximization algorithm in a variety of scenarios. In the second subproblem, we consider the application of positive definite kernels and the earth mover\u27s distance (END) to our work. Positive definite kernels are an important tool in machine learning that enable efficient solutions to otherwise difficult or intractable problems by implicitly linearizing the problem geometry. We develop a set-theoretic interpretation of ENID and propose earth mover\u27s intersection (EMI). a positive definite analog to ENID. We offer proof of EMD\u27s negative definiteness and provide necessary and sufficient conditions for ENID to be conditionally negative definite, including approximations that guarantee negative definiteness. In particular, we show that ENID is related to various min-like kernels. We also present a positive definite preserving transformation that can be applied to any kernel and can be used to derive positive definite EMD-based kernels, and we show that the Jaccard index is simply the result of this transformation applied to set intersection. Finally, we evaluate kernels based on EMI and the proposed transformation versus ENID in various computer vision tasks and show that END is generally inferior even with indefinite kernel techniques. Finally, we apply deep learning to our problem. We propose neural network architectures for hand posture and gesture recognition from unlabeled marker sets in a coordinate system local to the hand. As a means of ensuring data integrity, we also propose an extended Kalman filter for tracking the rigid pattern of markers on which the local coordinate system is based. We consider fixed- and variable-size architectures including convolutional and recurrent neural networks that accept unlabeled marker input. We also consider a data-driven approach to labeling markers with a neural network and a collection of Kalman filters. Experimental evaluations with posture and gesture datasets show promising results for the proposed architectures with unlabeled markers, which outperform the alternative data-driven labeling method