Machine learning in 3D space gesture recognition

The rapid increase in the development of robotic systems in a controlled and uncontrolled environment leads to the development of a more natural interaction system. One such interaction is gesture recognition. The proposed paper is a simple approach towards gesture recognition technology where the hand movement in a 3-dimensional space is utilized to write the English alphabets and get the corresponding output in the screen or a display device. In order to perform the experiment, an MPU-6050 accelerometer, a microcontroller and a Bluetooth for wireless connection are used as the hardware components of the system. For each of the letters of the alphabets, the data instances are recorded in its raw form. 20 instances for each letter are recorded and it is then standardized using interpolation. The standardized data is fed as inputs to an SVM (Support Vector Machine) classifier to create a model. The created model is used for classification of future data instances at real time. Our method achieves a correct classification accuracy of 98.94% for the English alphabets’ hand gesture recognition. The primary objective of our approach is the development of a low-cost, low power and easily trained supervised gesture recognition system which identifies hand gesture movement efficiently and accurately. The experimental result obtained is based on use of a single subject

A framework for developing motion-based games

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaNowadays, whenever one intents to develop an application that allows interaction through the use of more or less complex gestures, it is necessary to go through a long process. In this process, the gesture recognition system may not obtain high accuracy results, particularly among different users. Since the total number of applications for mobile systems, like Android and iOS, is close to a million and a half and is still increasing, it appears essential the development of a platform that abstracts developers from all the low-level gesture gathering and that streamlines the process of developing applications that make use of this kind of interaction, in a standardize way. In this case such was developed for the iOS system. At the present time, given the existing environment issues, it is ideal to attract the attention, motivate and influence the greatest number of people into having more proenvironmental behaviors. Thus, as a proof of concept for the developed framework, an educational game was created, using persuasive technology, to influence players’s behaviors and attitudes in a pro-environmental way. Therefore, having this idea as a basis, it was also developed a game that is presented in a public ambient display and can be played by any participant close to the displaywho has a device with iOS mobile system

Perancangan Sarung Tangan Untuk Pengenalan Sistem Isyarat Bahasa Indonesia Berbasis Sensor

Penelitian ini bertujuan untuk mengembangkan sarung tangan yang dilengkapi sensor (embedded system) yang digunakan dalam sistem pengenalan Sistem Isyarat Bahasa Indonesia (SIBI). Dengan pendekatan berbasis data sensor, sistem pengenalan SIBI diharapkan dapat memiliki akurasi yang lebih baik, yaitu dengan menggunakan sensor flex (untuk gerakan lekukan jari, dan menggunakan kombinasi sensor accelerometer-gyroscope untuk mengetahui kemiringan/orientasi tangan. Penelitian ini masih dalam tahap perancangan sarung tangan. Dalam tahap perancangan ini telah diselesaikan untuk desain rangkaian, desain PCB, pembuatan PCB, pemasangan sensor flex dan desain program mikrokontroler

Towards streaming gesture recognition

The emergence of low-cost sensors allows more devices to be equipped with various types of sensors. In this way, mobile device such as smartphones or smartwatches now may contain accelerometers, gyroscopes, etc. This offers new possibilities for interacting with the environment and benefits would come to exploit these sensors. As a consequence, the literature on gesture recognition systems that employ such sensors grow considerably. The literature regarding online gesture recognition counts many methods based on Dynamic Time Warping (DTW). However, this method was demonstrated has non-efficient for time series from inertial sensors unit as a lot of noise is present. In this way new methods based on LCSS (Longest Common SubSequence) were introduced. Nevertheless, none of them focus on a class optimization process. In this master thesis, we present and evaluate a new algorithm for online gesture recognition in noisy streams. This technique relies upon the LM-WLCSS (Limited Memory and Warping LCSS) algorithm that has demonstrated its efficiency on gesture recognition. This new method involves a quantization step (via the K-Means clustering algorithm) that transforms new data to a finite set. In this way, each new sample can be compared to several templates (one per class). Gestures are rejected based on a previously trained rejection threshold. Thereafter, an algorithm, called SearchMax, find a local maximum within a sliding window and output whether or not the gesture has been recognized. In order to resolve conflicts that may occur, another classifier (i.e. C4.5) could be completed. As the K-Means clustering algorithm needs to be initialized with the number of clusters to create, we also introduce a straightforward optimization process. Such an operation also optimizes the window size for the SearchMax algorithm. In order to demonstrate the robustness of our algorithm, an experiment has been performed over two different data sets. However, results on tested data sets are only accurate when training data are used as test data. This may be due to the fact that the method is in an overlearning state. L’apparition de nouveaux capteurs à bas prix a permis d’en équiper dans beaucoup plus d’appareils. En effet, dans les appareils mobiles tels que les téléphones et les montres intelligentes nous retrouvons des accéléromètres, gyroscopes, etc. Ces capteurs présents dans notre vie quotidienne offrent de toutes nouvelles possibilités en matière d’interaction avec notre environnement et il serait avantageux de les utiliser. Cela a eu pour conséquence une augmentation considérable du nombre de recherches dans le domaine de reconnaissance de geste basé sur ce type de capteur. La littérature concernant la reconnaissance de gestes en ligne comptabilise beaucoup de méthodes qui se basent sur Dynamic Time Warping (DTW). Cependant, il a été démontré que cette méthode se révèle inefficace en ce qui concerne les séries temporelles provenant d’une centrale à inertie puisqu’elles contiennent beaucoup de bruit. En ce sens de nouvelles méthodes basées sur LCSS (Longest Common SubSequence) sont apparues. Néanmoins, aucune d’entre elles ne s’est focalisée sur un processus d’optimisation par class. Ce mémoire de maîtrise consiste en une présentation et une évaluation d’un nouvel algorithme pour la reconnaissance de geste en ligne avec des données bruitées. Cette technique repose sur l’algorithme LM-WLCSS (Limited Memory and Warping LCSS) qui a d’ores et déjà démontré son efficacité quant à la reconnaissance de geste. Cette nouvelle méthode est donc composée d’une étape dite de quantification (grâce à l’algorithme de regroupement K-Means) qui se charge de convertir les nouvelles données entrantes vers un ensemble de données fini. Chaque nouvelle donnée peut donc être comparée à plusieurs motifs (un par classe) et un geste est reconnu dès lors que son score dépasse un seuil préalablement entrainé. Puis, un autre algorithme appelé SearchMax se charge de trouver un maximum local au sein d’une fenêtre glissant afin de préciser si oui ou non un geste a été reconnu. Cependant des conflits peuvent survenir et en ce sens un autre classifieur (c.-àd. C4.5) est chainé. Étant donné que l’algorithme de regroupement K-Means a besoin d’une valeur pour le nombre de regroupements à faire, nous introduisons également une technique simple d’optimisation à ce sujet. Cette partie d’optimisation se charge également de trouver la meilleure taille de fenêtre possible pour l’algorithme SearchMax. Afin de démontrer l’efficacité et la robustesse de notre algorithme, nous l’avons testé sur deux ensembles de données différents. Cependant, les résultats sur les ensembles de données testées n’étaient bons que lorsque les données d’entrainement étaient utilisées en tant que données de test. Cela peut être dû au fait que la méthode est dans un état de surapprentissage

Diseño y validación de un software de reconocimiento de gestos sirviéndose del acelerómetro de un reloj inteligente

[ES] El trabajo propone un software para el microcontrolador ESP32 de un reloj inteligente, que interpreta los movimientos del brazo y los convierte en órdenes visibles en la interfaz del propio reloj. Previamente se analiza el funcionamiento de los acelerómetros y del microcontrolador, como base del hardware en el que nos apoyaremos. Así como la metodología de análisis de componentes principales y los algoritmos de reconocimiento de gestos Dynamic Time Warping (DTW), DTW with AP and CS, y Multilayer Perceptron (MLP), entre otros posibles métodos. También se analizan otros trabajos de investigación que usaron dichos algoritmos en otros dispositivos emisores diferentes a los relojes inteligentes. Y, a partir de ahí, se diseña el software específico para los relojes inteligentes, como resultado principal del trabajo. El algoritmo que se diseña es sometido, por último, a validación utilizando la metodología de matrices de confusión.[EN] This mather thesis proposes the development of a firmware for the ESP32 microcontroller in a smart watch, which interprets the movements of the arm and converts them into visible orders on the smart watch interface. Previously, the operation of the accelerometers and the microcontroller is analyzed, as the basis of the hardware that we will use. As well as the principal component analysis methodology and the Dynamic Time Warping (DTW), DTW with AP and CS, and Multilayer Perceptron (MLP) gesture recognition algorithms, among other possible methods. Other research works that use these algorithms are also analyzed. And, from there, the specific software for smart watches is designed, as the main result of the work. Finally, the algorithm that is designed is subjected to validation using the confusion matrix methodology.Herrera Crespo, J. (2021). Diseño y validación de un software de reconocimiento de gestos sirviéndose del acelerómetro de un reloj inteligente. Universitat Politècnica de València. http://hdl.handle.net/10251/172272TFG