Using Common Spatial Patterns to Select Relevant Pixels for Video Activity Recognition

first_page settings Open AccessArticle Using Common Spatial Patterns to Select Relevant Pixels for Video Activity Recognition by Itsaso Rodríguez-Moreno * [OrcID] , José María Martínez-Otzeta [OrcID] , Basilio Sierra [OrcID] , Itziar Irigoien , Igor Rodriguez-Rodriguez and Izaro Goienetxea [OrcID] Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebastián, Spain * Author to whom correspondence should be addressed. Appl. Sci. 2020, 10(22), 8075; https://doi.org/10.3390/app10228075 Received: 1 October 2020 / Revised: 30 October 2020 / Accepted: 11 November 2020 / Published: 14 November 2020 (This article belongs to the Special Issue Advanced Intelligent Imaging Technology Ⅱ) Download PDF Browse Figures Abstract Video activity recognition, despite being an emerging task, has been the subject of important research due to the importance of its everyday applications. Video camera surveillance could benefit greatly from advances in this field. In the area of robotics, the tasks of autonomous navigation or social interaction could also take advantage of the knowledge extracted from live video recording. In this paper, a new approach for video action recognition is presented. The new technique consists of introducing a method, which is usually used in Brain Computer Interface (BCI) for electroencephalography (EEG) systems, and adapting it to this problem. After describing the technique, achieved results are shown and a comparison with another method is carried out to analyze the performance of our new approach.This work has been partially funded by the Basque Government, Research Teams grant number IT900-16, ELKARTEK 3KIA project KK-2020/00049, and the Spanish Ministry of Science (MCIU), the State Research Agency (AEI), and the European Regional Development Fund (FEDER), grant number RTI2018-093337-B-I100 (MCIU/AEI/FEDER, UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research

Automatic Food Intake Assessment Using Camera Phones

Obesity is becoming an epidemic phenomenon in most developed countries. The fundamental cause of obesity and overweight is an energy imbalance between calories consumed and calories expended. It is essential to monitor everyday food intake for obesity prevention and management. Existing dietary assessment methods usually require manually recording and recall of food types and portions. Accuracy of the results largely relies on many uncertain factors such as user\u27s memory, food knowledge, and portion estimations. As a result, the accuracy is often compromised. Accurate and convenient dietary assessment methods are still blank and needed in both population and research societies. In this thesis, an automatic food intake assessment method using cameras, inertial measurement units (IMUs) on smart phones was developed to help people foster a healthy life style. With this method, users use their smart phones before and after a meal to capture images or videos around the meal. The smart phone will recognize food items and calculate the volume of the food consumed and provide the results to users. The technical objective is to explore the feasibility of image based food recognition and image based volume estimation. This thesis comprises five publications that address four specific goals of this work: (1) to develop a prototype system with existing methods to review the literature methods, find their drawbacks and explore the feasibility to develop novel methods; (2) based on the prototype system, to investigate new food classification methods to improve the recognition accuracy to a field application level; (3) to design indexing methods for large-scale image database to facilitate the development of new food image recognition and retrieval algorithms; (4) to develop novel convenient and accurate food volume estimation methods using only smart phones with cameras and IMUs. A prototype system was implemented to review existing methods. Image feature detector and descriptor were developed and a nearest neighbor classifier were implemented to classify food items. A reedit card marker method was introduced for metric scale 3D reconstruction and volume calculation. To increase recognition accuracy, novel multi-view food recognition algorithms were developed to recognize regular shape food items. To further increase the accuracy and make the algorithm applicable to arbitrary food items, new food features, new classifiers were designed. The efficiency of the algorithm was increased by means of developing novel image indexing method in large-scale image database. Finally, the volume calculation was enhanced through reducing the marker and introducing IMUs. Sensor fusion technique to combine measurements from cameras and IMUs were explored to infer the metric scale of the 3D model as well as reduce noises from these sensors

Predicting Alzheimer's disease by segmenting and classifying 3D-brain MRI images using clustering technique and SVM classifiers.

Alzheimer's disease (AD) is the most common form of dementia affecting seniors age 65 and over. When AD is suspected, the diagnosis is usually confirmed with behavioural assessments and cognitive tests, often followed by a brain scan. Advanced medical imaging and pattern recognition techniques are good tools to create a learning database in the first step and to predict the class label of incoming data in order to assess the development of the disease, i.e., the conversion from prodromal stages (mild cognitive impairment) to Alzheimer's disease. Advanced medical imaging such as the volumetric MRI can detect changes in the size of brain regions due to the loss of the brain tissues. Measuring regions that atrophy during the progress of Alzheimer's disease can help neurologists in detecting and staging the disease. In this thesis, we want to diagnose the Alzheimer’s disease from MRI images. We segment brain MRI images to extract the brain chambers. Then, features are extracted from the segmented area. Finally, a classifier is trained to differentiate between normal and AD brain tissues. We discuss an automatic scheme that reads volumetric MRI, extracts the middle slices of the brain region, performs 2-dimensional (volume slices) and volumetric segmentation methods in order to segment gray matter, white matter and cerebrospinal fluid (CSF), generates a feature vector that characterizes this region, creates a database that contains the generated data, and finally classifies the images based on the extracted features. For our results, we have used the MRI data sets from the Alzheimer’s disease Neuroimaging Initiative (ADNI) database1. We assessed the performance of the classifiers by using results from the clinical tests.Master of Science (M.Sc.) in Computational Science

Machine learning approaches to video activity recognition: from computer vision to signal processing

244 p.La investigación presentada se centra en técnicas de clasificación para dos tareas diferentes, aunque relacionadas, de tal forma que la segunda puede ser considerada parte de la primera: el reconocimiento de acciones humanas en vídeos y el reconocimiento de lengua de signos.En la primera parte, la hipótesis de partida es que la transformación de las señales de un vídeo mediante el algoritmo de Patrones Espaciales Comunes (CSP por sus siglas en inglés, comúnmente utilizado en sistemas de Electroencefalografía) puede dar lugar a nuevas características que serán útiles para la posterior clasificación de los vídeos mediante clasificadores supervisados. Se han realizado diferentes experimentos en varias bases de datos, incluyendo una creada durante esta investigación desde el punto de vista de un robot humanoide, con la intención de implementar el sistema de reconocimiento desarrollado para mejorar la interacción humano-robot.En la segunda parte, las técnicas desarrolladas anteriormente se han aplicado al reconocimiento de lengua de signos, pero además de ello se propone un método basado en la descomposición de los signos para realizar el reconocimiento de los mismos, añadiendo la posibilidad de una mejor explicabilidad. El objetivo final es desarrollar un tutor de lengua de signos capaz de guiar a los usuarios en el proceso de aprendizaje, dándoles a conocer los errores que cometen y el motivo de dichos errores

Medición, procesado y clasificación de señales electroencefalográficas

El objetivo principal de este trabajo es el de implementar un sistema BCI (Brain Computer Interface) mediante un dispositivo de captación EEG (Electroencefalograma) para medir, procesar y clasificar señales cerebrales en tiempo real con el fin de poder discriminar entre dos clases de movimientos imaginados sin necesidad de realizar una actividad motora. Las aplicaciones prácticas que se abren ante el investigador en caso de éxito son indudablemente numerosas y atractivas. En los primeros capítulos se ven los fundamentos teóricos necesarios para transitar hacia la parte práctica. Se estudia la fisiología del cerebro y, particularmente, el comportamiento de las neuronas que, a través de los potenciales de acción, generan las ondas cerebrales que se acaban registrando en un EEG. La monitorización de la actividad cerebral lleva al desarrollo de los sistemas BCI que proporcionan un canal de comuniación con el mundo exterior sin necesidad de recurrir a actividades musculares. Se repasa el estado de la investigación BCI actual y sus características y se plantean algunas consideraciones sobre su desarrollo futuro. El dispositivo Emotiv EPOC empleado en el trabajo es también presentado en una primera aproximación técnica. Se estudia a continuación la teoría de reconocimiento de patrones y de clasificación que, apoyándose en la teoría de decisión bayesiana, sirve de cimiento conceptual y matemático para las tareas de discriminación entre clases que es necesario hacer en este trabajo. Se estudia en profundidad la técnica del LDA (Linear Discriminant Analysis) concebida y optimizada para la clasificación y que se usa en el sistema diseñado. También se analiza el algoritmo CSP (Common Spatial Patterns) empleado para calcular el filtro espacial óptimo que reduce la dimensionalidad de las señales originales. Se introduce además el procedimiento de PCA (Principal Component Analysis) utilizado en reconocimiento de patrones para obtener las componentes principales de un conjunto de observaciones, reduciendo así las dimensiones del conjunto total. En la parte final del trabajo se ve el esquema del sistema BCI diseñado con sus características técnicas y, antes de detallar las simulaciones realizadas y los resultados obtenidos, se comprueba el funcionamiento del sistema con datos EEG registrados profesionalmente. Se muestran además un par de aplicaciones prácticas que se han desarrollado y probado, un juego de ping-pong y una herramienta de deletreo, con el fin de que sirvan de ejemplo de las utilidades que pueden tener cabida en el mundo real a raíz de la investigación sobre las señales electroencefalográficas. Para concluir este trabajo se hace una recopilación de los temas tratados que, junto con las problemáticas afrontadas durante su realización y algunas reflexiones personales, derivan en las conclusiones finales.xv Abstract he main goal of this project is to implement a BCI (Brain Computer Interface) system using an EEG monitoring device to measure, to process and to classify brain signals in real time in order to discriminate between two classes of imagined movements without the need to perform motor activity. In case of success, the practical applications that present themselves in front of the investigator are undoubtfully numerous and attractive. In the first chapters, we review the theoretical fundaments that are going to be applied in the practical part of the project. We study the physiology of the brain and, more precisely, the behaviour of the neurons through its action potentials that generate brain waves that can be registered with an EEG device. The monitorization of brain activities leads to the development of BCI systems that supply a channel to communicate with the outside world without muscle activity. A summary of recent BCI investigations is made, whith explanations about its characteristics and some considerations about future evolutions on the subject. The Emotiv EPOC device is the one that is going to be used in this project and it is presented with a brief technical approach. Afterwards we focus on pattern recognition theory and classification theory, and these two along Bayesian decision theory are used as mathematical and conceptual foundations for the tasks of class discrimination needed in this project. We study thoroughly the LDA (Linear Discriminant Analysis) technique that was conceived and optimized for classification and it is used on the designed system. CSP (Common Spatial Patterns) algorithm is also explained to find the optimal spatial filter that reduces the dimensionality of the original signals. We also introduce the PCA (Principal Component Analysis) algorithm used in pattern recognition to obtain the principal components from a set of observations and therefore to reduce the total dimensions. In the final part of the project, the scheme of the designed BCI system is shown and technically explained. Before introducing the simulations and the final results, the system is verified using professionally recorded EEG data. We find a couple of software applications that were developed and tested, a ping-pong game and a spelling tool, with the aim to serve as an example of the outcomes of research in electroencephalographic signals that can have their place in the real world. To finish this project we review all the subjects, the problems that were faced and some personal reflections that altogether lead to the final conclusions.Universidad de Sevilla. Grado en Ingeniería de las Tecnologías de Telecomunicació