Percepção do ambiente urbano e navegação usando visão robótica : concepção e implementação aplicado à veículo autônomo

Orientadores: Janito Vaqueiro Ferreira, Alessandro Corrêa VictorinoTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia MecânicaResumo: O desenvolvimento de veículos autônomos capazes de se locomover em ruas urbanas pode proporcionar importantes benefícios na redução de acidentes, no aumentando da qualidade de vida e também na redução de custos. Veículos inteligentes, por exemplo, frequentemente baseiam suas decisões em observações obtidas a partir de vários sensores tais como LIDAR, GPS e câmeras. Atualmente, sensores de câmera têm recebido grande atenção pelo motivo de que eles são de baixo custo, fáceis de utilizar e fornecem dados com rica informação. Ambientes urbanos representam um interessante mas também desafiador cenário neste contexto, onde o traçado das ruas podem ser muito complexos, a presença de objetos tais como árvores, bicicletas, veículos podem gerar observações parciais e também estas observações são muitas vezes ruidosas ou ainda perdidas devido a completas oclusões. Portanto, o processo de percepção por natureza precisa ser capaz de lidar com a incerteza no conhecimento do mundo em torno do veículo. Nesta tese, este problema de percepção é analisado para a condução nos ambientes urbanos associado com a capacidade de realizar um deslocamento seguro baseado no processo de tomada de decisão em navegação autônoma. Projeta-se um sistema de percepção que permita veículos robóticos a trafegar autonomamente nas ruas, sem a necessidade de adaptar a infraestrutura, sem o conhecimento prévio do ambiente e considerando a presença de objetos dinâmicos tais como veículos. Propõe-se um novo método baseado em aprendizado de máquina para extrair o contexto semântico usando um par de imagens estéreo, a qual é vinculada a uma grade de ocupação evidencial que modela as incertezas de um ambiente urbano desconhecido, aplicando a teoria de Dempster-Shafer. Para a tomada de decisão no planejamento do caminho, aplica-se a abordagem dos tentáculos virtuais para gerar possíveis caminhos a partir do centro de referencia do veículo e com base nisto, duas novas estratégias são propostas. Em primeiro, uma nova estratégia para escolher o caminho correto para melhor evitar obstáculos e seguir a tarefa local no contexto da navegação hibrida e, em segundo, um novo controle de malha fechada baseado na odometria visual e o tentáculo virtual é modelado para execução do seguimento de caminho. Finalmente, um completo sistema automotivo integrando os modelos de percepção, planejamento e controle são implementados e validados experimentalmente em condições reais usando um veículo autônomo experimental, onde os resultados mostram que a abordagem desenvolvida realiza com sucesso uma segura navegação local com base em sensores de câmeraAbstract: The development of autonomous vehicles capable of getting around on urban roads can provide important benefits in reducing accidents, in increasing life comfort and also in providing cost savings. Intelligent vehicles for example often base their decisions on observations obtained from various sensors such as LIDAR, GPS and Cameras. Actually, camera sensors have been receiving large attention due to they are cheap, easy to employ and provide rich data information. Inner-city environments represent an interesting but also very challenging scenario in this context, where the road layout may be very complex, the presence of objects such as trees, bicycles, cars might generate partial observations and also these observations are often noisy or even missing due to heavy occlusions. Thus, perception process by nature needs to be able to deal with uncertainties in the knowledge of the world around the car. While highway navigation and autonomous driving using a prior knowledge of the environment have been demonstrating successfully, understanding and navigating general inner-city scenarios with little prior knowledge remains an unsolved problem. In this thesis, this perception problem is analyzed for driving in the inner-city environments associated with the capacity to perform a safe displacement based on decision-making process in autonomous navigation. It is designed a perception system that allows robotic-cars to drive autonomously on roads, without the need to adapt the infrastructure, without requiring previous knowledge of the environment and considering the presence of dynamic objects such as cars. It is proposed a novel method based on machine learning to extract the semantic context using a pair of stereo images, which is merged in an evidential grid to model the uncertainties of an unknown urban environment, applying the Dempster-Shafer theory. To make decisions in path-planning, it is applied the virtual tentacle approach to generate possible paths starting from ego-referenced car and based on it, two news strategies are proposed. First one, a new strategy to select the correct path to better avoid obstacles and to follow the local task in the context of hybrid navigation, and second, a new closed loop control based on visual odometry and virtual tentacle is modeled to path-following execution. Finally, a complete automotive system integrating the perception, path-planning and control modules are implemented and experimentally validated in real situations using an experimental autonomous car, where the results show that the developed approach successfully performs a safe local navigation based on camera sensorsDoutoradoMecanica dos Sólidos e Projeto MecanicoDoutor em Engenharia Mecânic

Emerging research directions in computer science : contributions from the young informatics faculty in Karlsruhe

In order to build better human-friendly human-computer interfaces, such interfaces need to be enabled with capabilities to perceive the user, his location, identity, activities and in particular his interaction with others and the machine. Only with these perception capabilities can smart systems ( for example human-friendly robots or smart environments) become posssible. In my research I\u27m thus focusing on the development of novel techniques for the visual perception of humans and their activities, in order to facilitate perceptive multimodal interfaces, humanoid robots and smart environments. My work includes research on person tracking, person identication, recognition of pointing gestures, estimation of head orientation and focus of attention, as well as audio-visual scene and activity analysis. Application areas are humanfriendly humanoid robots, smart environments, content-based image and video analysis, as well as safety- and security-related applications. This article gives a brief overview of my ongoing research activities in these areas

Ein modulares Konzept von Klassifikatoren für Aktivitätserkennung auf Mobiltelefonen

In this thesis a modular activity recognition using accelerometer sensors on mobile phones is presented, which includes solutions to five challenges: 1.Flexibility: The conditions of the mobile phone usage and therefore for the activity recognition can always change. An activity recognition needs to flexibly adapt to this changes. 2.Extensibility: Different users have different demands of activities to be recognized. Only a small set of activities are performed by nearly every user. Therefore, the recognition needs to be extensible to the individual needs. 3.Robustness: The device is typically not firmly attached to any position, which results in noisy sensor data. A robust recognition is needed, which is able to detect the activities with high accuracy. 4.Resources: The resources on mobile phones are limited (processor and battery capacity), therrfore the activity recognition needs not to have a high impact on these. 5.Conditionality: The user and her phone can be situated in various different conditions. Each of these conditionalities implies different sensor patterns, which need representation in the activity recognition algorithm. The modularity of the proposed approach enables the individual adaption of parts of the activity recognition to offer flexibility. A modular recognition is extensible by new modules which detect new activities. The recurrence of the classification process stabilizes the recognition and enables the derivation of a reliability measure. Only one module and not the whole activity recognition is active at each point in time, which decreases the calculation effort and therefore the energy consumption. Each module can be suited for dealing with one conditionality, through which neither the complexity of the recognition is increased nor the accuracy is significantly lowered. All these solutions to the challenges of activity recognition on mobile phones are rounded by a service, which supports the novel system on the common user's phone.In dieser Dissertation wird eine modulare Aktivitätserkennung mit Beschleunigungssensoren auf Mobiltelefonen vorgestellt, die Lösungen für folgende fünf Herausforderungen bereitstellt: 1.Flexibilität: Die Bedingungen der Nutzung eines Mobiltelefons und damit auch für die Aktivitätserkennung können sich jederzeit ändern. Eine Aktivitätserkennung muss flexibel auf diese Veränderungen reagieren können. 2.Erweiterbarkeit: Unterschiedliche Anwender haben unterschiedliche Anforderungen welche Aktivitäten erkannt werden sollen. Daher muss die Erkennung erweiterbar sein, um die individuellen Bedürfnisse befriedigen zu können. 3.Robustheit: Das Gerät ist typischerweise nicht fest an einer Position angebracht, woraus verrauschten Sensordaten resultieren. Desswegen ist eine robuste Erkennung erforderlich, welche in der Lage ist die Aktivitäten trotzdem mit hoher Genauigkeit zu detektieren. 4.Resources: Die Ressourcen (Prozessor und Akku-Kapazität) auf Handys sind beschränkt, weshalb die Aktivitätserkennung diese nicht noch zusätzlich übermäßig einschränken sollte. 5.Konditionalität: Der Benutzer und sein Telefon können in verschiedensten Gegebenheiten situiert sein. Jede dieser Konditionen impliziert andere Muster der Sensoren, welche jeweils durch die Aktivitätserkennung repräsentiert sein müssen. Durch die Modularität, welche in dieser Dissertation zur Bewältigung der Herausforderungen vorgeschlagen wird, wird ermöglicht, dass Flexibilität bereitgestellt werden kann. Eine modulare Erkennung ist erweiterbar durch neue Module, welche neue Aktivitäten erkennen. Die Rekurrenz des Klassifikationsprozesses stabilisiert die Erkennung. Nur ein Modul ist zu einem Zeitpunkt aktiv, was Ressourcen schont. Jedes Modul kann passend sein, um mit einer Konditionalität umzugehen, wobei die Komplexität weder erheblich erhöht noch die Genauigkeit stark erniedrigt wird. Alle diese Lösungen für die Herausforderungen der Aktivitätserkennung werden durch einen speziellen Service abgerundet