End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners

For human drivers, having rear and side-view mirrors is vital for safe driving. They deliver a more complete view of what is happening around the car. Human drivers also heavily exploit their mental map for navigation. Nonetheless, several methods have been published that learn driving models with only a front-facing camera and without a route planner. This lack of information renders the self-driving task quite intractable. We investigate the problem in a more realistic setting, which consists of a surround-view camera system with eight cameras, a route planner, and a CAN bus reader. In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and low-level driving maneuvers (e.g. steering angle and speed) by human drivers. With such a sensor setup we collect a new driving dataset, covering diverse driving scenarios and varying weather/illumination conditions. Finally, we learn a novel driving model by integrating information from the surround-view cameras and the route planner. Two route planners are exploited: 1) by representing the planned routes on OpenStreetMap as a stack of GPS coordinates, and 2) by rendering the planned routes on TomTom Go Mobile and recording the progression into a video. Our experiments show that: 1) 360-degree surround-view cameras help avoid failures made with a single front-view camera, in particular for city driving and intersection scenarios; and 2) route planners help the driving task significantly, especially for steering angle prediction.Comment: to be published at ECCV 201

Towards secure & robust PNT for automated systems

This dissertation makes four contributions in support of secure and robust position, navigation, and timing (PNT) for automated systems. The first two relate to PNT security while the latter two address robust positioning for automated ground vehicles. The first contribution is a fundamental theory for provably-secure clock synchronization between two agents in a distributed automated system. All one-way synchronization protocols, such as those based on the Global Positioning System (GPS) and other Global Navigation Satellite Systems (GNSS), are shown to be vulnerable to man-in-the-middle delay attacks. This contribution is the first to identify the necessary and sufficient conditions for provably secure clock synchronization. The second contribution, also related to PNT security, is a three-year study of the world-wide GPS interference landscape based on data from a dual-frequency GNSS receiver operating continuously on the International Space Station (ISS). This work is the first publicly-reported space-based survey of GNSS interference, and unveils previously-unreported GNSS interference activity. The third contribution is a novel ground vehicle positioning technique that is robust to GNSS signal blockage, poor lighting conditions, and adverse weather events such as heavy rain and dense fog. The technique relies on sensors that are commonly available on automated vehicles and are insensitive to lighting and inclement weather: automotive radar, low-cost inertial measurement units (IMUs), and GNSS. Remarkably, it is shown that, given a prior radar map, the proposed technique operating on data from off-the-shelf all-weather automotive sensors can maintain sub-50-cm horizontal position accuracy during 60 min of GNSS-denied driving in downtown Austin, TX. This dissertation’s final contribution is an analysis and demonstration of the feasibility of crowd-sourced digital mapping for automated vehicles. Localization techniques, such as the one described in the previous contribution, rely on such digital maps for accuracy and robustness. A key enabler for large-scale up-to-date maps is enlisting the help of the very consumer vehicles that need the map to build and update it. A method for fusing multi-session vision data into a unified digital map is developed. The asymptotic limit of such a map’s globally-referenced position accuracy is explored for the case in which the mapping agents rely on low-cost GNSS receivers performing standard code-phase-based navigation. Experimental validation along a semi-urban route shows that low-cost consumer vehicles incrementally tighten the accuracy of the jointly-optimized digital map over time enough to support sub-lane-level positioning in a global frame of reference.Electrical and Computer Engineerin

A Survey of Positioning Systems Using Visible LED Lights

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.As Global Positioning System (GPS) cannot provide satisfying performance in indoor environments, indoor positioning technology, which utilizes indoor wireless signals instead of GPS signals, has grown rapidly in recent years. Meanwhile, visible light communication (VLC) using light devices such as light emitting diodes (LEDs) has been deemed to be a promising candidate in the heterogeneous wireless networks that may collaborate with radio frequencies (RF) wireless networks. In particular, light-fidelity has a great potential for deployment in future indoor environments because of its high throughput and security advantages. This paper provides a comprehensive study of a novel positioning technology based on visible white LED lights, which has attracted much attention from both academia and industry. The essential characteristics and principles of this system are deeply discussed, and relevant positioning algorithms and designs are classified and elaborated. This paper undertakes a thorough investigation into current LED-based indoor positioning systems and compares their performance through many aspects, such as test environment, accuracy, and cost. It presents indoor hybrid positioning systems among VLC and other systems (e.g., inertial sensors and RF systems). We also review and classify outdoor VLC positioning applications for the first time. Finally, this paper surveys major advances as well as open issues, challenges, and future research directions in VLC positioning systems.Peer reviewe

Play JBT – Mobile Application for the Tropical Botanical Garden of Lisbon

Trabalho de projecto de mestrado, Informática, Universidade de Lisboa, Faculdade de Ciências, 2020Com o progresso das tecnologias de informação e comunicação (TIC), as instituições culturais diversificaram as modalidades de interação com as pessoas. TIC permite hoje as várias instituições culturais de assumir papeis diferentes perante a comunidade (por exemplo, educação dos cidadãos e das suas associações; formador de várias competências; e de perito em vários programas governamentais para desenvolvimento de comunidades). Neste documento está apresentado o trabalho de desenvolvimento de uma aplicação móvel para Jardim Botânico Tropical de Lisboa. Técnicas diversas foram utilizadas no desenvolvimento de aplicação móvel (por exemplo, entrevistas, listagem de conteúdos, prototipagem, avaliação heurística, testes de usabilidade). São apresentados detalhes das tecnologias usadas (software e hardware), procedimentos de implementação, como também sobre arquitetura final do sistema desenvolvido. A aplicação móvel permite aos visitantes de Jardim Botânico Tropical interagir de formas diferentes com os componentes de jardim (plantas, aves e edifícios). Vários recursos educativos são incluídos na aplicação de modo a ser adaptados de modo automático ao perfil do utilizador. A aplicação permite também captar e armazenar os dados produzidos por utilizadores da aplicação de modo a serem utilizados para melhoria de experiência dos visitantes do jardim. Vários serviços Web foram incluídos para melhorar apresentação dos conteúdos e para melhorar os serviços do jardim. Foram também realizados testes com peritos no jardim e recolhido feedback dos utilizadores dos quais recebemos boas críticas e sugestões que foram integradas na aplicação. Foram também realizados um conjunto de testes de desempenho do servidor.Through the progress of information and communication technologies (ICT), cultural institutions have diversified the modalities of interacting with people. Today, ICTs allow various cultural institutions to take on different roles in the community (e.g. educating citizens and their associations; shaping various skills; supporting government programs for community development). This document introduces the process of development of a mobile application, which acts mainly as a helping guide for visitors of the Lisbon Tropical Botanical Garden. This mobile application allows these visitors to interact in different ways with garden components (plants, buildings and birds), as well as to have access to the several educational resources included in it, which are to be adapted to the user's profile. The application also allows them to capture and store the data produced, data which is also used for help with improving garden services. Web services have been developed to provide content and to centrally store data on the visitor’s trajectory in the garden and demographics. Furthermore, various techniques were used in the process of development (e.g. interviews, content listing, prototyping, heuristic evaluation, usability testing). Details on the technologies used (software and hardware), implementation procedures, as well as the final architecture of the developed system will be demonstrated. Finally, a set of usability tests is presented, from which we received positive feedback from the users as well as the performance tests executed on the server

Vision-Based Georeferencing of GPR in Urban Areas

Ground Penetrating Radar (GPR) surveying is widely used to gather accurate knowledge about the geometry and position of underground utilities. The sensor arrays need to be coupled to an accurate positioning system, like a geodetic-grade Global Navigation Satellite System (GNSS) device. However, in urban areas this approach is not always feasible because GNSS accuracy can be substantially degraded due to the presence of buildings, trees, tunnels, etc. In this work, a photogrammetric (vision-based) method for GPR georeferencing is presented. The method can be summarized in three main steps: tie point extraction from the images acquired during the survey, computation of approximate camera extrinsic parameters and finally a refinement of the parameter estimation using a rigorous implementation of the collinearity equations. A test under operational conditions is described, where accuracy of a few centimeters has been achieved. The results demonstrate that the solution was robust enough for recovering vehicle trajectories even in critical situations, such as poorly textured framed surfaces, short baselines, and low intersection angles

Mobile client for collecting sport activity statistics

Vývoj mobilních aplikací může být poměrně náročný v případech, kdy je vyžadována dostupnost aplikace na více platformách. Cílem této práce bylo prozkoumat možnosti pro vývoj multiplatformních mobilních aplikací, seznámit se s technologií Apache Cordova a použít ji k vývoji mobilního klienta pro uživatele portálu Jäsenverkko.fi. Aplikace umožňuje uživatelům spravovat jejich data uložená na vzdáleném serveru. Pokud je zařízení připojeno k internetu, aplikace data automaticky synchronizuje za použití aplikačního rozhraní portálu, je však schopna pracovat i bez připojení díky implementaci lokálního úložiště. Aplikace umožňuje ukládání vzdáleně definovaných tréninkových statistik pomocí snadno rozšiřitelných modulů. Výsledkem práce je přenositelná aplikace otestovaná na platformách Android a Windows Phone.ObhájenoMobile applications development can be rather demanding in cases where it is required to users on multiple platforms. The goal of this thesis was to investigate methods of mobile multiplatform development, get acquainted with and utilize Apache Cordova to implement a mobile client application for Jäsenverkko.fi portal users. The application allows its users to manage their data stored on a remote server. It automatically synchronizes them using portal's RESTful API when connected to the internet but is able to work even when offline as it contains a local storage facility. The application makes it possible to collect custom sport statistics specified remotely with a module that is easily extendable. A result of the work is a portable application tested on Android and Windows Phone platforms