Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information

Adding intelligence to a floor based array personnel detector

According to the World Health Organization-WHO, a fall is defined, as "inadvertently coming to rest on the ground, floor or other lower level, excluding intentional change in position to rest in furniture, wall or other objects". It is known that a senior who falls is at risk for serious injury and after necessary spiraling down eventually to death. Researchers concern is to develop new technology or enhance existing one to detect falls and reduce the consequences of a fall. We enhanced smart carpet, which is a floor based personnel detector system, to detect falls using a faster but low cost processor. Our new hardware front end reads from 128 sensors (the sensors output a voltage due to a person walking or falling on the carpet). The processor is Jetson TK1, which provides more computing power than before. We generated a dataset with volunteers who walked and fell to test our algorithms. Data Obtained allowed examining data frames read from the data acquisition system. We used different algorithms and techniques, and varied the windows size of number of frames (WS>=1) and threshold (TH). We found that at (WS=8), and threshold (TH=8) using connected component labeling algorithm (CCL) produced a fall sensitivity of 87.9%. We then used the dataset obtained from applying a set of fall detection algorithms and the video recorded for the fall patterns experiments to train a set of classifiers using multiple test options using the Weka framework. We found that the widow size (WS=8) at a threshold (TH=8) using connected component algorithm generated attribute contributed to the fall sensitivity. We measured the performance of each testing options. The best feature was again the size of the connected component with WS=8, with classification accuracy of 96.94%. Other algorithms attributes did not contribute significantly to the detection of the fall

Configuración y ejecución de algoritmos de visión artificial en la tarjeta Nvidia Jetson TK1 DevKit

En el presente Trabajo Fin de Grado (TFG) se aborda la evaluación de la tarjeta de desarrollo NVidia Jetson TK1. Se trata de una tarjeta orientada a la ejecución de algoritmos de visión artificial a través del cálculo en paralelo mediante la Unidad de Procesamiento Gráfico (GPU) de la tarjeta, que dispone de un SOC (System on a Chip) Tegra K1 el cual incluye una GPU NVidia Tegra y un microprocesador ARM Cortex A-15 entre otros periféricos. La evaluación de la tarjeta se lleva a cabo desde dos perspectivas diferentes. En primer lugar, se realiza un análisis a nivel de hardware para encontrar las ventajas y limitaciones para su uso en aplicaciones de visión artificial, en concreto, se evalúa el uso de las librerías de OpenCV para visión en estéreo, combinadas con un desarrollo de entorno gráfico en OpenGL. Posteriormente, se comparan los tiempos de ejecución de diferentes algoritmos para evaluar los distintos rendimientos de la tarjeta y de su GPU y CPU (Unidad Central de Proceso).This final degree thesis (TFG) adresses the evaluation of the NVidia Jetson TK1 development board. It is a board oriented to the execution of computer vision algorithms using paralel computing on the Graphics Processing Unit (GPU) integrated on the board. The Jetson TK1 includes Tegra K1 SOC (System on a Chip) that integrates a NVidia Tegra GPU and an ARM Cortex A-15 microprocessor among other peripherals. The evaluation of the development board is carried out from two different perspectives. First, a hardware level analisis is made in order to analyze the advantages and limitations for computer vision applications, specially those that use OpenCV libraries for stereo vision, combined with a OpenGL graphical environment. Then, computation cost are evaluated for different algorithms, so a comparaive of the performance can be made between GPU and CPU (Central Processing Unit).Grado en Ingeniería en Electrónica y Automática Industria

Personalized data analytics for internet-of-things-based health monitoring

The Internet-of-Things (IoT) has great potential to fundamentally alter the delivery of modern healthcare, enabling healthcare solutions outside the limits of conventional clinical settings. It can offer ubiquitous monitoring to at-risk population groups and allow diagnostic care, preventive care, and early intervention in everyday life. These services can have profound impacts on many aspects of health and well-being. However, this field is still at an infancy stage, and the use of IoT-based systems in real-world healthcare applications introduces new challenges. Healthcare applications necessitate satisfactory quality attributes such as reliability and accuracy due to their mission-critical nature, while at the same time, IoT-based systems mostly operate over constrained shared sensing, communication, and computing resources. There is a need to investigate this synergy between the IoT technologies and healthcare applications from a user-centered perspective. Such a study should examine the role and requirements of IoT-based systems in real-world health monitoring applications. Moreover, conventional computing architecture and data analytic approaches introduced for IoT systems are insufficient when used to target health and well-being purposes, as they are unable to overcome the limitations of IoT systems while fulfilling the needs of healthcare applications. This thesis aims to address these issues by proposing an intelligent use of data and computing resources in IoT-based systems, which can lead to a high-level performance and satisfy the stringent requirements. For this purpose, this thesis first delves into the state-of-the-art IoT-enabled healthcare systems proposed for in-home and in-hospital monitoring. The findings are analyzed and categorized into different domains from a user-centered perspective. The selection of home-based applications is focused on the monitoring of the elderly who require more remote care and support compared to other groups of people. In contrast, the hospital-based applications include the role of existing IoT in patient monitoring and hospital management systems. Then, the objectives and requirements of each domain are investigated and discussed. This thesis proposes personalized data analytic approaches to fulfill the requirements and meet the objectives of IoT-based healthcare systems. In this regard, a new computing architecture is introduced, using computing resources in different layers of IoT to provide a high level of availability and accuracy for healthcare services. This architecture allows the hierarchical partitioning of machine learning algorithms in these systems and enables an adaptive system behavior with respect to the user's condition. In addition, personalized data fusion and modeling techniques are presented, exploiting multivariate and longitudinal data in IoT systems to improve the quality attributes of healthcare applications. First, a real-time missing data resilient decision-making technique is proposed for health monitoring systems. The technique tailors various data resources in IoT systems to accurately estimate health decisions despite missing data in the monitoring. Second, a personalized model is presented, enabling variations and event detection in long-term monitoring systems. The model evaluates the sleep quality of users according to their own historical data. Finally, the performance of the computing architecture and the techniques are evaluated in this thesis using two case studies. The first case study consists of real-time arrhythmia detection in electrocardiography signals collected from patients suffering from cardiovascular diseases. The second case study is continuous maternal health monitoring during pregnancy and postpartum. It includes a real human subject trial carried out with twenty pregnant women for seven months

Human-vehicle collaborative driving to improve transportation safety

This dissertation proposes a collaborative driving framework which is based on the assessments of both internal and external risks involved in vehicle driving. The internal risk analysis includes driver drowsiness detection, driver distraction detection, and driver intention recognition which help us better understand the human driver's behavior. Steering wheel data and facial expression are used to detect the drowsiness. Images from a camera observing the driver are used to detect various types of driver distraction by using the deep learning approach. Hidden Markov Models (HMM) is implemented to recognize the driver's intention using the vehicle's laneposition, control and state data. For the external risk analysis, the co-pilot utilizes a Collision Avoidance System (CAS) to estimate the collision probability between the ego vehicle and other vehicles. Based on these two risk analyses, a novel collaborative driving scheme is proposed by fusing the control inputs from the human driver and the co-pilot to obtain the final control input for the vehicle under different circumstances. The proposed collaborative driving framework is validated in an Intelligent Transportation System (ITS) testbed which enables both autonomous and manual driving capabilities

Integrated architecture for vision-based indoor localization and mapping of a quadrotor micro-air vehicle

Atualmente os sistemas de pilotagem autónoma de quadricópteros estão a ser desenvolvidos de forma a efetuarem navegação em espaços exteriores, onde o sinal de GPS pode ser utilizado para definir waypoints de navegação, modos de position e altitude hold, returning home, entre outros. Contudo, o problema de navegação autónoma em espaços fechados sem que se utilize um sistema de posicionamento global dentro de uma sala, subsiste como um problema desafiante e sem solução fechada. Grande parte das soluções são baseadas em sensores dispendiosos, como o LIDAR ou como sistemas de posicionamento externos (p.ex. Vicon, Optitrack). Algumas destas soluções reservam a capacidade de processamento de dados dos sensores e dos algoritmos mais exigentes para sistemas de computação exteriores ao veículo, o que também retira a componente de autonomia total que se pretende num veículo com estas características. O objetivo desta tese pretende, assim, a preparação de um sistema aéreo não-tripulado de pequeno porte, nomeadamente um quadricóptero, que integre diferentes módulos que lhe permitam simultânea localização e mapeamento em espaços interiores onde o sinal GPS ´e negado, utilizando, para tal, uma câmara RGB-D, em conjunto com outros sensores internos e externos do quadricóptero, integrados num sistema que processa o posicionamento baseado em visão e com o qual se pretende que efectue, num futuro próximo, planeamento de movimento para navegação. O resultado deste trabalho foi uma arquitetura integrada para análise de módulos de localização, mapeamento e navegação, baseada em hardware aberto e barato e frameworks state-of-the-art disponíveis em código aberto. Foi também possível testar parcialmente alguns módulos de localização, sob certas condições de ensaio e certos parâmetros dos algoritmos. A capacidade de mapeamento da framework também foi testada e aprovada. A framework obtida encontra-se pronta para navegação, necessitando apenas de alguns ajustes e testes.Nowdays, the existing systems for autonomous quadrotor control are being developed in order to perform navigation in outdoor areas where the GPS signal can be used to define navigational waypoints and define flight modes like position and altitude hold, returning home, among others. However, the problem of autonomous navigation in closed areas, without using a global positioning system inside a room, remains a challenging problem with no closed solution. Most solutions are based on expensive sensors such as LIDAR or external positioning (f.e. Vicon, Optitrack) systems. Some of these solutions allow the capability of processing data from sensors and algorithms for external systems, which removes the intended fully autonomous component in a vehicle with such features. Thus, this thesis aims at preparing a small unmanned aircraft system, more specifically, a quadrotor, that integrates different modules which will allow simultaneous indoor localization and mapping where GPS signal is denied, using for such a RGB-D camera, in conjunction with other internal and external quadrotor sensors, integrated into a system that processes vision-based positioning and it is intended to carry out, in the near future, motion planning for navigation. The result of this thesis was an integrated architecture for testing localization, mapping and navigation modules, based on open-source and inexpensive hardware and available state-of-the-art frameworks. It was also possible to partially test some localization frameworks, under certain test conditions and algorithm parameters. The mapping capability of the framework was also tested and approved. The obtained framework is navigation ready, needing only some adjustments and testing

Молодежь и современные информационные технологии: сборник трудов XV Международной научно-практической конференции студентов, аспирантов и молодых учёных, 04-07 декабря 2017 г., г. Томск

Сборник содержит доклады, представленные на XV Международной научно-практическую конференцию студентов, аспирантов и молодых ученых "Молодежь и современные информационные технологии", прошедшей в Томском политехническом университете на базе Инженерной школы информационных технологий и робототехники. Материалы сборника отражают доклады студентов, аспирантов и молодых ученых, принятые к обсуждению на секциях: "Математическое моделирование и компьютерный анализ данных", "Автоматизация и управление в технических системах", "Информационные и программные системы в производстве и управлении", "Компьютерная графика и дизайн", "Информационные технологии в гуманитарных и медицинских исследованиях". Сборник предназначен для специалистов в области информационных технологий, студентов и аспирантов соответствующих специальностей