Unbiased Learning of Deep Generative Models with Structured Discrete Representations

By composing graphical models with deep learning architectures, we learn generative models with the strengths of both frameworks. The structured variational autoencoder (SVAE) inherits structure and interpretability from graphical models, and flexible likelihoods for high-dimensional data from deep learning, but poses substantial optimization challenges. We propose novel algorithms for learning SVAEs, and are the first to demonstrate the SVAE's ability to handle multimodal uncertainty when data is missing by incorporating discrete latent variables. Our memory-efficient implicit differentiation scheme makes the SVAE tractable to learn via gradient descent, while demonstrating robustness to incomplete optimization. To more rapidly learn accurate graphical model parameters, we derive a method for computing natural gradients without manual derivations, which avoids biases found in prior work. These optimization innovations enable the first comparisons of the SVAE to state-of-the-art time series models, where the SVAE performs competitively while learning interpretable and structured discrete data representations.Comment: 38 pages, 7 figure

Biomechanical Markerless Motion Classification Based On Stick Model Development For Shop Floor Operator

Motion classification system marks a new era of industrial technology to monitor task performance and validate the quality of manual processes using automation. However, the current study trend pointed towards the marker-based motion capture system that demanded the expensive and extensive equipment setup. The markerless motion classification model is still underdeveloped in the manufacturing industry. Therefore, this research is purposed to develop a markerless motion classification model of shopfloor operators using stick model augmentation on the motion video and identify the best data mining strategy for the industrial motion classification. Eight participants within 23 to 24 years old participated in an experiment to perform four distinct motion sequences: moving box, moving pail, sweeping and mopping the floor, recorded in separate videos. All videos were augmented with a stick model made up of keypoints and lines using the programming model. The programming model incorporated the COCO dataset and OpenCV module to estimate the coordinates and body joints for a stick model overlay. The data extracted from the stick model featured the initial velocity, cumulative velocity and acceleration for each body joint. Motion data mining process included the data normalization, random subsampling method and data classification to discover the best information for separating motion classes. The motion vector data extracted were normalized with three different techniques: the decimal scaling normalization, min-max normalization, and Z-score normalization, to create three datasets for further data mining. All the datasets were experimented with eight classifiers to determine the best machine learning classifier and normalization technique to classify the model data. The eight tested classifiers were ZeroR, OneR, J48, random forest, random tree, Naïve Bayes, K-nearest neighbours (K = 5) and multilayer perceptron. The result showed that the random forest classifier scored the best performance with the highest recorded data classification accuracy in its min-max normalized dataset, 81.75% for the dataset before random subsampling and 92.37% for the resampled dataset. The min-max normalization gives only a slight advantage over the other normalization techniques using the same dataset. However, the random subsampling method dramatically improves the classification accuracy by eliminating the noise data and replacing them with replicated instances to balance the class. The best normalization method and data mining classifier were inserted into the motion classification model to complete the development process

Dynamic Calibration of EMG Signals for Control of a Wearable Elbow Brace

Musculoskeletal injuries can severely inhibit performance of activities of daily living. In order to regain function, rehabilitation is often required. Assistive devices for use in rehabilitation are an avenue explored to increase arm mobility by guiding therapeutic exercises or assisting with motion. Electromyography (EMG), which are the muscle activity signals, may be able to provide an intuitive interface between the patient and the device if appropriate classification models allow smart systems to relate these signals to the desired device motion. Unfortunately, there is a gap in the accuracy of pattern recognition models classifying motion in constrained laboratory environments, and large reductions in accuracy when used for detecting dynamic unconstrained movements. An understanding of combinations of motion factors (limb positions, forces, velocities) in dynamic movements affecting EMG, and ways to use information about these motion factors in control systems is lacking. The objectives of this thesis were to quantify how various motion factors affect arm muscle activations during dynamic motion, and to use these motion factors and EMG signals for detecting interaction forces between the person and the environment during motion. To address these objectives, software was developed and implemented to collect a unique dataset of EMG signals while healthy individuals performed unconstrained arm motions with combinations of arm positions, interaction forces with the environment, velocities, and types of motion. An analysis of the EMG signals and their use in training classification models to predict characteristics (arm positions, force levels, and velocities) of intended motion was completed. The results quantify how EMG features change significantly with variations in arm positions, interaction forces, and motion velocities. The results also show that pattern recognition models, usually used to detect movements, were able to detect intended characteristics of motion based solely on EMG signals, even during complex activities of daily living. Arm position during elbow flexion--extension was predicted with 83.02 % accuracy by a support vector machine model using EMG signal inputs. Prediction of force, the motion characteristic that cannot be measured without impeding motion, was improved from 76.85 % correct to 79.17 % accurate during elbow flexion--extension by providing measurable arm position and velocity information as additional inputs to a linear discriminant analysis model. The accuracy of force prediction was improved by 5.2 % (increased from 59.38 % to 64.58 %) during an activity of daily living when motion speeds were included as an input to a linear discriminant analysis model in addition to EMG signals. Future work should expand on using motion characteristics and EMG signals to identify interactions between a person and the environment, in order to guide high level tuning of control models working towards controlling wearable elbow braces during dynamic movements

Clasificador de modelos de movilidad mediante Deep Learning

El objetivo de este proyecto es el desarrollo de una red neuronal capaz de clasificar hasta cuatro patrones de movimientos con la mejor precisión posible. Se utilizan modelos sintetizados de movimientos, dado que no se disponen de datos reales de patrones de movimiento, generados con la herramienta Bonn Motion. Esta herramienta nos permite simular diferentes modelos de movimiento. Para este proyecto se utilizan los modelos Random waypoint, Truncated Levy Walk, Gauss Markov y Manhattan. Se entrenan redes neuronales de tres tipos: Fully connected, Recurrentes y Convolucionales. Las redes neuronales convolucionales y las recurrentes tienen en cuenta la secuencia temporal del movimiento, por lo que se espera que aporten mejores resultados. La sintonización de las redes neuronales se comienza con un modelo básico de las mismas y se hacen experimentos que nos permiten ajustarlas hasta dar con su mejor resultado, observando la pérdida y la precisión. Por otro lado, se analiza también la influencia del set de datos en los resultados. Se entrenan las redes neuronales variando, por un lado, el tiempo de las simulaciones de los patrones de movimiento y por otro, el número de patrones movimiento de entrada. Para el desarrollo de dichas redes neuronales, se utiliza Python 3.7 y las librerías Tensor Flow y Keras. Salvo para las redes neuronales recurrentes más complejas, se trabaja con la CPU del ordenador. Para el resto, se trabaja con la GPU. La GPU permite realizar cálculos matriciales a gran velocidad, lo que reduce el tiempo de entrenamiento. Con los resultados obtenidos, las conclusiones principales son las siguientes: Entrenando redes recurrentes y convolucionales se obtienen mejores resultados que con las redes fully connected, aunque estas últimas requieren menor coste computacional (el tiempo de entrenamiento es considerablemente menor). Las redes neuronales recurrentes presentan mejores resultados que las convolucionales para patrones de movimiento de corta duración.The aim of this project is to develop a neural network capable of classifying up to four movement patterns with the best possible accuracy. In this case, since no real movement pattern data is available, synthesized movement models, generated with the Bonn Motion tool, are used. This tool allows us to simulate different movement models. For this project, Random waypoint, Truncated Levy Walk, Gauss Markov and Manhattan models are used. Three types of neural networks are trained: Fully connected, Recurrent and Convolutional. Convolutional and recurrent neural networks take into account the time sequence of movement, so they are expected to provide better results. The tuning of the neuronal networks starts with a basic model of them and experiments are made that allow us to adjust them until we find the best result, observing the loss and precision. On the other hand, the influence of the data set on the results is also analysed. The neural networks are trained by varying, on the one hand, the time of the movement pattern simulations and, on the other hand, the number of input movement patterns. Python 3.7 and the Tensor Flow and Keras libraries are used for the development of these neural networks. Except for the more complex recurrent neural networks, we work with the computer CPU. For the rest, we work with the GPU. The GPU enables high-speed matrix computing, which reduces training time. With the results obtained, the main conclusions are the following: - Training recurrent and convolutional networks provides better results than fully connected networks, although the latter require less computational cost (training time is considerably less). - Recurrent neural networks show better results than convolutional networks for short duration movement patterns.Universidad de Sevilla. Máster en Ingeniería Industria

The Efficacy of the Eigenvector Approach to South African Sign Language Identification

Masters of ScienceThe communication barriers between deaf and hearing society mean that interaction between these communities is kept to a minimum. The South African Sign Language research group, Integration of Signed and Verbal Communication: South African Sign Language Recognition and Animation (SASL), at the University of the Western Cape aims to create technologies to bridge the communication gap. In this thesis we address the subject of whole hand gesture recognition. We demonstrate a method to identify South African Sign Language classifiers using an eigenvector approach. The classifiers researched within this thesis are based on those outlined by the Thibologa Sign Language Institute for SASL. Gesture recognition is achieved in real time. Utilising a pre-processing method for image registration we are able to increase the recognition rates for the eigenvector approach

The efficacy of the Eigenvector approach to South African sign language identification

Masters of ScienceThe communication barriers between deaf and hearing society mean that interaction between these communities is kept to a minimum. The South African Sign Language research group, Integration of Signed and Verbal Communication: South African Sign Language Recognition and Animation (SASL), at the University of the Western Cape aims to create technologies to bridge the communication gap. In this thesis we address the subject of whole hand gesture recognition. We demonstrate a method to identify South African Sign Language classifiers using an eigenvector ap- proach. The classifiers researched within this thesis are based on those outlined by the Thibologa Sign Language Institute for SASL. Gesture recognition is achieved in real- time. Utilising a pre-processing method for image registration we are able to increase the recognition rates for the eigenvector approach.South Afric

Motion Intention Estimation using sEMG-ACC Sensor Fusion

Musculoskeletal injuries can severely impact the ability to produce and control body motion. In order to regain function, rehabilitation is often required. Wearable smart devices are currently under development to provide therapy and assistance for people with impaired arm function. Electromyography (EMG) signals are used as an input to pattern recognition systems to determine intended movements. However, there is a gap between the accuracy of pattern recognition systems in constrained laboratory settings, and usability when used for detecting dynamic unconstrained movements. Motion factors such as limb position, interaction force, and velocity, are known to have a negative impact on the pattern recognition. A possible solution lies in the use of data from other sensors along with the EMG signals, such as signals from accelerometers (ACC), in the training and use of classifiers in order to improve classification accuracy. The objectives of this study were to quantify the impact of motion factors on ACC signals, and to use these ACC signals along with EMG signals for classifying categories of motion factors. To address these objectives, a dataset containing EMG and ACC signals while individuals performed unconstrained arm motions was studied. Analyses of the EMG and accelerometer signals and their use in training classification models to predict characteristics of intended motion were completed. The results quantify how accelerometer features change with variations in arm position, interaction forces, and motion velocities. The results also show that the combination of EMG and ACC data have relatively increased the accuracy of motion intention detection. Velocity could be distinguished between stationary and moving with less than 10% error using a Decision Tree ensemble classifier. Future work should expand on motion factors and EMG-ACC sensor fusion to identify interactions between a person and the environment, in order to guide tuning of control models working towards controlling wearable mechatronic devices during dynamic movements