Decision-making problems in computationally constrained robot perception

Despite recent advances in robotic perception, tasks such as object detection continue to remain challenging on edge devices, especially when real-time operation is required, due to limited processing power and a high volume of input data to process. This thesis examines these challenges from the points of view of architectural design and training of neural networks, the software involved in deploying perception models and algorithms, and decision-making. It does so in the domain of robot soccer and in the context of the RoboCup SPL competition, where only limited, mobile scale hardware is available, and real-time perception is necessary. The thesis makes the following contributions: 1) the design of YOLO-based network architectures and training of lightweight, deep object detection models based on the YOLO paradigm for robot soccer, and the realization of a realtime, on-robot vision system incorporating these object detectors. 2) the design and implementation of a software framework for robot perception based on the Robot Operating System (ROS) that is conducive to usage in the RoboCup SPL, and as a research platform for robot perception under computational constraints. 3) a formalization of the problem of perceptual decision-making, wherein an agent needs to makes choices that decide how visual information is processed. 4) Investigative analysis through experiments conducted in the robot soccer domain showing that perceptual decision-making can enhance task performance through better utilization of limited time and computational resources for perception.Computer Science

Deep Reinforcement Learning for 3D-based Object Grasping

Nowadays, collaborative robots based on Artificial Intelligence algorithms are very common to see in workstations and laboratories and they are expected to help their human colleagues in their everyday work. However, this type of robots can also assist in a domestic home, in tasks such as separate and organizing cutlery objects, but for that they need an algorithm to tell them which object to grasp and where to it. The main focus of this thesis is to create or improve an existing algorithm based on a Deep Reinforcement Learning for 3D-based Object Grasping, aiming to help collaborative robots on such tasks. Therefore, this work aims to present the state of the art and the study carried out, that enables the implementation of the proposed model that will help such robots to detect, grasp and separate each type of cutlery objects and consecutive experiments and results, as well as the retrospective of all the work done.Hoje em dia, ouve-se falar mais de robôs e do crescimento da robótica do que se ouviria há duas décadas atrás. A indústria da robótica tem vindo a evoluir imenso e a prova disso é a existência de robôs em estações de trabalho e laboratórios, cujo seu propósito é colaborar nas tarefas dos seus colegas trabalhadores humanos. A este tipo de robôs dá-se o nome de Cobot ou robô colaborativo. Estes robôs têm de suporte algoritmos da Inteligência Artificial para os ajudar a tomar as decisões mais corretas nas tarefas que têm de desempenhar. Contudo, este tipo de robôs já começa a ser adotado para tarefas domésticas. O tema desta dissertação envolve três grandes áreas: Inteligência Artificial, Visão Computacional e Robótica e tem como principal objetivo o desenvolvimento de um algoritmo de Aprendizagem por Reforço, que dê suporte a um robô universal, versão 3, na tomada de decisões para apanhar e separar objetos de cozinha por tipo. Assim sendo optou-se pelo uso de um algoritmo já desenvolvido, chamado Visual Pushing-for-Grasping, que permite simular robôs colaborativos a empurrar e apanhar objetos. Todavia, os objetos utilizados por este algoritmo em simulação não eram objetos de cozinha e o algoritmo apenas realiza apanha de objetos sem realizar a separação dos mesmos. Como tal, propomos uma nova abordagem com base no algoritmo anteriormente referido, e que passará a utilizar modelos 3D de objetos de cozinha, fará a deteção do tipo de objeto no cenário com recurso a um modelo de deteção de objetos exterior ao algoritmo base e que procederá à separação dos objetos por tipo. Os resultados experimentais permitem concluir que esta nova abordagem ainda precisa de ser melhorada, contudo e por ser uma abordagem nova tanto no ramo da Robótica como no ramo da Inteligència Artificial, para uso com o robôs universais da versão 3, afirmamos que os resultados estão melhores do que o esperado e expectamos que um dia esta possa ser aplicada a um robô físico em contexto real

Open-Source Face Recognition Frameworks: A Review of the Landscape

publishedVersio

Real-time RGB-Depth preception of humans for robots and camera networks

This thesis deals with robot and camera network perception using RGB-Depth data. The goal is to provide efficient and robust algorithms for interacting with humans. For this reason, a special care has been devoted to design algorithms which can run in real-time on consumer computers and embedded cards. The main contribution of this thesis is the 3D body pose estimation of the human body. We propose two novel algorithms which take advantage of the data stream of a RGB-D camera network outperforming the state-of-the-art performance in both single-view and multi-view tests. While the first algorithm works on point cloud data which is feasible also with no external light, the second one performs better, since it deals with multiple persons with negligible overhead and does not rely on the synchronization between the different cameras in the network. The second contribution regards long-term people re-identification in camera networks. This is particularly challenging since we cannot rely on appearance cues, in order to be able to re-identify people also in different days. We address this problem by proposing a face-recognition framework based on a Convolutional Neural Network and a Bayes inference system to re-assign the correct ID and person name to each new track. The third contribution is about Ambient Assisted Living. We propose a prototype of an assistive robot which periodically patrols a known environment, reporting unusual events as people fallen on the ground. To this end, we developed a fast and robust approach which can work also in dimmer scenes and is validated using a new publicly-available RGB-D dataset recorded on-board of our open-source robot prototype. As a further contribution of this work, in order to boost the research on this topics and to provide the best benefit to the robotics and computer vision community, we released under open-source licenses most of the software implementations of the novel algorithms described in this work

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

Semantic models of scenes and objects for service and industrial robotics

What may seem straightforward for the human perception system is still challenging for robots. Automatically segmenting the elements with highest relevance or salience, i.e. the semantics, is non-trivial given the high level of variability in the world and the limits of vision sensors. This stands up when multiple ambiguous sources of information are available, which is the case when dealing with moving robots. This thesis leverages on the availability of contextual cues and multiple points of view to make the segmentation task easier. Four robotic applications will be presented, two designed for service robotics and two for an industrial context. Semantic models of indoor environments will be built enriching geometric reconstructions with semantic information about objects, structural elements and humans. Our approach leverages on the importance of context, the availability of multiple source of information, as well as multiple view points showing with extensive experiments on several datasets that these are all crucial elements to boost state-of-the-art performances. Furthermore, moving to applications with robots analyzing object surfaces instead of their surroundings, semantic models of Carbon Fiber Reinforced Polymers will be built augmenting geometric models with accurate measurements of superficial fiber orientations, and inner defects invisible to the human-eye. We succeeded in reaching an industrial grade accuracy making these models useful for autonomous quality inspection and process optimization. In all applications, special attention will be paid towards fast methods suitable for real robots like the two prototypes presented in this thesis

Cyber Security of Critical Infrastructures

Critical infrastructures are vital assets for public safety, economic welfare, and the national security of countries. The vulnerabilities of critical infrastructures have increased with the widespread use of information technologies. As Critical National Infrastructures are becoming more vulnerable to cyber-attacks, their protection becomes a significant issue for organizations as well as nations. The risks to continued operations, from failing to upgrade aging infrastructure or not meeting mandated regulatory regimes, are considered highly significant, given the demonstrable impact of such circumstances. Due to the rapid increase of sophisticated cyber threats targeting critical infrastructures with significant destructive effects, the cybersecurity of critical infrastructures has become an agenda item for academics, practitioners, and policy makers. A holistic view which covers technical, policy, human, and behavioural aspects is essential to handle cyber security of critical infrastructures effectively. Moreover, the ability to attribute crimes to criminals is a vital element of avoiding impunity in cyberspace. In this book, both research and practical aspects of cyber security considerations in critical infrastructures are presented. Aligned with the interdisciplinary nature of cyber security, authors from academia, government, and industry have contributed 13 chapters. The issues that are discussed and analysed include cybersecurity training, maturity assessment frameworks, malware analysis techniques, ransomware attacks, security solutions for industrial control systems, and privacy preservation methods