Robust Signal Processing Techniques for Wearable Inertial Measurement Unit (IMU) Sensors

Activity and gesture recognition using wearable motion sensors, also known as inertial measurement units (IMUs), provides important context for many ubiquitous sensing applications including healthcare monitoring, human computer interface and context-aware smart homes and offices. Such systems are gaining popularity due to their minimal cost and ability to provide sensing functionality at any time and place. However, several factors can affect the system performance such as sensor location and orientation displacement, activity and gesture inconsistency, movement speed variation and lack of tiny motion information. This research is focused on developing signal processing solutions to ensure the system robustness with respect to these factors. Firstly, for existing systems which have already been designed to work with certain sensor orientation/location, this research proposes opportunistic calibration algorithms leveraging camera information from the environment to ensure the system performs correctly despite location or orientation displacement of the sensors. The calibration algorithms do not require extra effort from the users and the calibration is done seamlessly when the users present in front of an environmental camera and perform arbitrary movements. Secondly, an orientation independent and speed independent approach is proposed and studied by exploring a novel orientation independent feature set and by intelligently selecting only the relevant and consistent portions of various activities and gestures. Thirdly, in order to address the challenge that the IMU is not able capture tiny motion which is important to some applications, a sensor fusion framework is proposed to fuse the complementary sensor modality in order to enhance the system performance and robustness. For example, American Sign Language has a large vocabulary of signs and a recognition system solely based on IMU sensors would not perform very well. In order to demonstrate the feasibility of sensor fusion techniques, a robust real-time American Sign Language recognition approach is developed using wrist worn IMU and surface electromyography (EMG) sensors

Robust Signal Processing Techniques for Wearable Inertial Measurement Unit (IMU) Sensors

Activity and gesture recognition using wearable motion sensors, also known as inertial measurement units (IMUs), provides important context for many ubiquitous sensing applications including healthcare monitoring, human computer interface and context-aware smart homes and offices. Such systems are gaining popularity due to their minimal cost and ability to provide sensing functionality at any time and place. However, several factors can affect the system performance such as sensor location and orientation displacement, activity and gesture inconsistency, movement speed variation and lack of tiny motion information. This research is focused on developing signal processing solutions to ensure the system robustness with respect to these factors. Firstly, for existing systems which have already been designed to work with certain sensor orientation/location, this research proposes opportunistic calibration algorithms leveraging camera information from the environment to ensure the system performs correctly despite location or orientation displacement of the sensors. The calibration algorithms do not require extra effort from the users and the calibration is done seamlessly when the users present in front of an environmental camera and perform arbitrary movements. Secondly, an orientation independent and speed independent approach is proposed and studied by exploring a novel orientation independent feature set and by intelligently selecting only the relevant and consistent portions of various activities and gestures. Thirdly, in order to address the challenge that the IMU is not able capture tiny motion which is important to some applications, a sensor fusion framework is proposed to fuse the complementary sensor modality in order to enhance the system performance and robustness. For example, American Sign Language has a large vocabulary of signs and a recognition system solely based on IMU sensors would not perform very well. In order to demonstrate the feasibility of sensor fusion techniques, a robust real-time American Sign Language recognition approach is developed using wrist worn IMU and surface electromyography (EMG) sensors

Reconocimiento de gestos basado en acelerómetros

En los últimos años, ha crecido de forma significativa el interés por la utilización de dispositivos capaces de reconocer gestos humanos. En este trabajo, se pretenden reconocer gestos manuales colocando sensores en la mano de una persona. El reconocimiento de gestos manuales puede ser implementado para diversos usos y bajo diversas plataformas: juegos (Wii), control de brazos robóticos, etc. Como primer paso, se realizará un estudio de las actuales técnicas de reconocimiento de gestos que utilizan acelerómetros como sensor de medida. En un segundo paso, se estudiará como los acelerómetros pueden utilizarse para intentar reconocer los gestos que puedan realizar una persona (mover el brazo hacia un lado, girar la mano, dibujar un cuadrado, etc.) y los problemas que de su utilización puedan derivarse. Se ha utilizado una IMU (Inertial Measurement Unit) como sensor de medida. Está compuesta por tres acelerómetros y tres giróscopos (MTi-300 de Xsens). Con las medidas que proporcionan estos sensores se realiza el cálculo de la posición y orientación de la mano, representando esta última en función de los ángulos de Euler. Un aspecto importante a destacar será el efecto de la gravedad en las medidas de las aceleraciones. A través de diversos cálculos y mediante la ayuda de los giróscopos se podrá corregir dicho efecto. Por último, se desarrollará un sistema que identifique la posición y orientación de la mano como gestos reconocidos utilizando lógica difusa. Tanto para la adquisición de las muestras, como para los cálculos de posicionamiento, se ha desarrollado un código con el programa Matlab. También, con este mismo software, se ha implementado un sistema de lógica difusa con la que se realizará el reconocimiento de los gestos, utilizando la herramienta FIS Editor. Las pruebas realizadas han consistido en la ejecución de nueve gestos por diferentes personas teniendo una tasa de reconocimiento comprendida entre el 90 % y 100 % dependiendo del gesto a identificar. ABSTRACT In recent years, it has grown significantly interest in the use of devices capable of recognizing human gestures. In this work, we aim to recognize hand gestures placing sensors on the hand of a person. The recognition of hand gestures can be implemented for different applications on different platforms: games (Wii), control of robotic arms ... As a first step, a study of current gesture recognition techniques that use accelerometers and sensor measurement is performed. In a second step, we study how accelerometers can be used to try to recognize the gestures that can make a person (moving the arm to the side, rotate the hand, draw a square, etc...) And the problems of its use can be derived. We used an IMU (Inertial Measurement Unit) as a measuring sensor. It comprises three accelerometers and three gyroscopes (Xsens MTI-300). The measures provided by these sensors to calculate the position and orientation of the hand are made, with the latter depending on the Euler angles. An important aspect to note is the effect of gravity on the measurements of the accelerations. Through various calculations and with the help of the gyroscopes can correct this effect. Finally, a system that identifies the position and orientation of the hand as recognized gestures developed using fuzzy logic. Both the acquisition of samples to calculate position, a code was developed with Matlab program. Also, with the same software, has implemented a fuzzy logic system to be held with the recognition of gestures using the FIS Editor. Tests have involved the execution of nine gestures by different people having a recognition rate between 90% and 100% depending on the gesture to identify

Towards Energy Efficient Mobile Eye Tracking for AR Glasses through Optical Sensor Technology

After the introduction of smartphones and smartwatches, Augmented Reality (AR) glasses are considered the next breakthrough in the field of wearables. While the transition from smartphones to smartwatches was based mainly on established display technologies, the display technology of AR glasses presents a technological challenge. Many display technologies, such as retina projectors, are based on continuous adaptive control of the display based on the user’s pupil position. Furthermore, head-mounted systems require an adaptation and extension of established interaction concepts to provide the user with an immersive experience. Eye-tracking is a crucial technology to help AR glasses achieve a breakthrough through optimized display technology and gaze-based interaction concepts. Available eye-tracking technologies, such as Video Oculography (VOG), do not meet the requirements of AR glasses, especially regarding power consumption, robustness, and integrability. To further overcome these limitations and push mobile eye-tracking for AR glasses forward, novel laser-based eye-tracking sensor technologies are researched in this thesis. The thesis contributes to a significant scientific advancement towards energy-efficientmobile eye-tracking for AR glasses. In the first part of the thesis, novel scanned laser eye-tracking sensor technologies for AR glasses with retina projectors as display technology are researched. The goal is to solve the disadvantages of VOG systems and to enable robust eye-tracking and efficient ambient light and slippage through optimized sensing methods and algorithms. The second part of the thesis researches the use of static Laser Feedback Interferometry (LFI) sensors as low power always-on sensor modality for detection of user interaction by gaze gestures and context recognition through Human Activity Recognition (HAR) for AR glasses. The static LFI sensors can measure the distance to the eye and the eye’s surface velocity with an outstanding sampling rate. Furthermore, they offer high integrability regardless of the display technology. In the third part of the thesis, a model-based eye-tracking approach is researched based on the static LFI sensor technology. The approach leads to eye-tracking with an extremely high sampling rate by fusing multiple LFI sensors, which enables methods for display resolution enhancement such as foveated rendering for AR glasses and Virtual Reality (VR) systems. The scientific contributions of this work lead to a significant advance in the field of mobile eye-tracking for AR glasses through the introduction of novel sensor technologies that enable robust eye tracking in uncontrolled environments in particular. Furthermore, the scientific contributions of this work have been published in internationally renowned journals and conferences

Proceedings of the 7th Sound and Music Computing Conference

Proceedings of the SMC2010 - 7th Sound and Music Computing Conference, July 21st - July 24th 2010

Human Machine Interaction

In this book, the reader will find a set of papers divided into two sections. The first section presents different proposals focused on the human-machine interaction development process. The second section is devoted to different aspects of interaction, with a special emphasis on the physical interaction

On-line Time Warping of Human Motion Sequences

Some application areas require motions to be time warped on-line as a motion is captured, aligning a partially captured motion to a complete prerecorded motion. For example movement training applications for dance and medical procedures, require on-line time warping for analysing and visually feeding back the accuracy of human motions as they are being performed. Additionally, real-time production techniques such as virtual production, in camera visual effects and the use of avatars in live stage performances, require on-line time warping to align virtual character performances to a live performer. The work in this thesis first addresses a research gap in the measurement of the alignment of two motions, proposing approaches based on rank correlation and evaluating them against existing distance based approaches to measuring motion similarity. The thesis then goes onto propose and evaluate novel methods for on-line time warping, which plot alignments in a forward direction and utilise forecasting and local continuity constraint techniques. Current studies into measuring the similarity of motions focus on distance based metrics for measuring the similarity of the motions to support motion recognition applications, leaving a research gap regarding the effectiveness of similarity metrics bases on correlation and the optimal metrics for measuring the alignment of two motions. This thesis addresses this research gap by comparing the performance of variety of similarity metrics based on distance and correlation, including novel combinations of joint parameterisation and correlation methods. The ability of each metric to measure both the similarity and alignment of two motions is independently assessed. This work provides a detailed evaluation of a variety of different approaches to using correlation within a similarity metric, testing their performance to determine which approach is optimal and comparing their performance against established distance based metrics. The results show that a correlation based metric, in which joints are parameterised using displacement vectors and correlation is measured using Kendall Tau rank correlation, is the optimal approach for measuring the alignment between two motions. The study also showed that similarity metrics based on correlation are better at measuring the alignment of two motions, which is important in motion blending and style transfer applications as well as evaluating the performance of time warping algorithms. It also showed that metrics based on distance are better at measuring the similarity of two motions, which is more relevant to motion recognition and classification applications. A number of approaches to on-line time warping have been proposed within existing research, that are based on plotting an alignment path backwards from a selected end-point within the complete motion. While these approaches work for discrete applications, such as recognising a motion, their lack of monotonic constraint between alignment of each frame, means these approaches do not support applications that require an alignment to be maintained continuously over a number of frames. For example applications involving continuous real-time visualisation, feedback or interaction. To solve this problem, a number of novel on-line time warping algorithms, based on forward plotting, motion forecasting and local continuity constraints are proposed and evaluated by applying them to human motions. Two benchmarks standards for evaluating the performance of on-line time warping algorithms are established, based on UTW time warping and compering the resulting alignment path with that produced by DTW. This work also proposes a novel approach to adapting existing local continuity constraints to a forward plotting approach. The studies within this thesis demonstrates that these time warping approaches are able to produce alignments of sufficient quality to support applications that require an alignment to be maintained continuously. The on-line time warping algorithms proposed in this study can align a previously recorded motion to a user in real-time, as they are performing the same action or an opposing action recorded at the same time as the motion being align. This solution has a variety of potential application areas including: visualisation applications, such as aligning a motion to a live performer to facilitate in camera visual effects or a live stage performance with a virtual avatar; motion feedback applications such as dance training or medical rehabilitation; and interaction applications such as working with Cobots

Proceedings of the 19th Sound and Music Computing Conference

Proceedings of the 19th Sound and Music Computing Conference - June 5-12, 2022 - Saint-Étienne (France). https://smc22.grame.f

XLIII Jornadas de Automática: libro de actas: 7, 8 y 9 de septiembre de 2022, Logroño (La Rioja)

[Resumen] Las Jornadas de Automática (JA) son el evento más importante del Comité Español de Automática (CEA), entidad científico-técnica con más de cincuenta años de vida y destinada a la difusión e implantación de la Automática en la sociedad. Este año se celebra la cuadragésima tercera edición de las JA, que constituyen el punto de encuentro de la comunidad de Automática de nuestro país. La presente edición permitirá dar visibilidad a los nuevos retos y resultados del ámbito, y su uso en un gran número de aplicaciones, entre otras, las energías renovables, la bioingeniería o la robótica asistencial. Además de la componente científica, que se ve reflejada en este libro de actas, las JA son un punto de encuentro de las diferentes generaciones de profesores, investigadores y profesionales, incluyendo la componente social que es de vital importancia. Esta edición 2022 de las JA se celebra en Logroño, capital de La Rioja, región mundialmente conocida por la calidad de sus vinos de Denominación de Origen y que ha asumido el desafío de poder ganar competitividad a través de la transformación verde y digital. Pero también por ser la cuna del castellano e impulsar el Valle de la Lengua con la ayuda de las nuevas tecnologías, entre ellas la Automática Inteligente. Los organizadores de estas JA, pertenecientes al Área de Ingeniería de Sistemas y Automática del Departamento de Ingeniería Eléctrica de la Universidad de La Rioja (UR), constituyen un pilar fundamental en el apoyo a la región para el estudio, implementación y difusión de estos retos. Esta edición, la primera en formato íntegramente presencial después de la pandemia de la covid-19, cuenta con más de 200 asistentes y se celebra a caballo entre el Edificio Politécnico de la Escuela Técnica Superior de Ingeniería Industrial y el Monasterio de Yuso situado en San Millán de la Cogolla, dos marcos excepcionales para la realización de las JA. Como parte del programa científico, dos sesiones plenarias harán hincapié, respectivamente, sobre soluciones de control para afrontar los nuevos retos energéticos, y sobre la calidad de los datos para una inteligencia artificial (IA) imparcial y confiable. También, dos mesas redondas debatirán aplicaciones de la IA y la implantación de la tecnología digital en la actividad profesional. Adicionalmente, destacaremos dos clases magistrales alineadas con tecnología de última generación que serán impartidas por profesionales de la empresa. Las JA también van a albergar dos competiciones: CEABOT, con robots humanoides, y el Concurso de Ingeniería de Control, enfocado a UAVs. A todas estas actividades hay que añadir las reuniones de los grupos temáticos de CEA, las exhibiciones de pósteres con las comunicaciones presentadas a las JA y los expositores de las empresas. Por último, durante el evento se va a proceder a la entrega del “Premio Nacional de Automática” (edición 2022) y del “Premio CEA al Talento Femenino en Automática”, patrocinado por el Gobierno de La Rioja (en su primera edición), además de diversos galardones enmarcados dentro de las actividades de los grupos temáticos de CEA. Las actas de las XLIII Jornadas de Automática están formadas por un total de 143 comunicaciones, organizadas en torno a los nueve Grupos Temáticos y a las dos Líneas Estratégicas de CEA. Los trabajos seleccionados han sido sometidos a un proceso de revisión por pares