Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection

Multispectral pedestrian detection has received extensive attention in recent years as a promising solution to facilitate robust human target detection for around-the-clock applications (e.g. security surveillance and autonomous driving). In this paper, we demonstrate illumination information encoded in multispectral images can be utilized to significantly boost performance of pedestrian detection. A novel illumination-aware weighting mechanism is present to accurately depict illumination condition of a scene. Such illumination information is incorporated into two-stream deep convolutional neural networks to learn multispectral human-related features under different illumination conditions (daytime and nighttime). Moreover, we utilized illumination information together with multispectral data to generate more accurate semantic segmentation which are used to boost pedestrian detection accuracy. Putting all of the pieces together, we present a powerful framework for multispectral pedestrian detection based on multi-task learning of illumination-aware pedestrian detection and semantic segmentation. Our proposed method is trained end-to-end using a well-designed multi-task loss function and outperforms state-of-the-art approaches on KAIST multispectral pedestrian dataset

Nonparametric Spatio-Temporal Joint Probabilistic Data Association Coupled Filter and Interfering Extended Target Tracking

Extended target tracking estimates the centroid and shape of the target in space and time. In various situations where extended target tracking is applicable, the presence of multiple targets can lead to interference, particularly when they maneuver behind one another in a sensor like a camera. Nonetheless, when dealing with multiple extended targets, there's a tendency for them to share similar shapes within a group, which can enhance their detectability. For instance, the coordinated movement of a cluster of aerial vehicles might cause radar misdetections during their convergence or divergence. Similarly, in the context of a self-driving car, lane markings might split or converge, resulting in inaccurate lane tracking detections. A well-known joint probabilistic data association coupled (JPDAC) filter can address this problem in only a single-point target tracking. A variation of JPDACF was developed by introducing a nonparametric Spatio-Temporal Joint Probabilistic Data Association Coupled Filter (ST-JPDACF) to address the problem for extended targets. Using different kernel functions, we manage the dependency of measurements in space (inside a frame) and time (between frames). Kernel functions are able to be learned using a limited number of training data. This extension can be used for tracking the shape and dynamics of nonparametric dependent extended targets in clutter when targets share measurements. The proposed algorithm was compared with other well-known supervised methods in the interfering case and achieved promising results.Comment: 12 pages, 8 figures, Journa

Multimodal fusion architectures for pedestrian detection

Pedestrian detection provides a crucial functionality in many human-centric applications, such as video surveillance, urban scene analysis, and autonomous driving. Recently, multimodal pedestrian detection has received extensive attention since the fusion of complementary information captured by visible and infrared sensors enables robust human target detection under daytime and nighttime scenes. In this chapter, we systematically evaluate the performance of different multimodal fusion architectures in order to identify the optimal solutions for pedestrian detection. We made two important observations: (1) it is useful to combine the most commonly used concatenation fusion scheme with a global scene-aware mechanism to learn both human-related features and correlation between visible and thermal feature maps; (2) the two-stream segmentation supervision without multimodal fusion provides the most effective scheme to infuse segmentation information as supervision for learning human-related features. Based on these studies, we present a unified multimodal fusion framework for joint training of target detection and segmentation supervision which achieves the state-of-the-art multimodal pedestrian detection performance on the public KAIST benchmark dataset.</p

Parametrização de algoritmos para deteção de estrada a bordo do AtlasCar

Mestrado em Engenharia MecânicaO veículo AtlasCar é um protótipo desenvolvido pelo Laboratório de Automação e Robótica do Departamento de Engenharia Mecânica da Universidade de Aveiro, para o estudo de sistemas de segurança ativos e passivos e soluções de condução autónoma. O objetivo deste trabalho _e dotar o AtlasCar de um sistema de visão artificial capaz de reconhecer os limites da estrada, sendo para isso necessária a procura de marcas na estrada e/ou características das mesmas. Este trabalho encontra-se dividido em duas partes principais: o desenvolvimento de módulos de software para a implementação de algoritmos com metodologias de deteção distintas e o estudo dos parâmetros que os compõem. O primeiro algoritmo baseia-se no método de RANSAC para efetuar a aproximação de uma spline aos limites da estrada, por sua vez o segundo algoritmo utiliza a transformada de Hough para efetuar esta aproximação. Os métodos apresentados foram testados em condições reais e foram analisadas as situações de falha, vantagens e desvantagens da aplicação de um método em relativamente ao outro. Com a implementação deste sistema ser a possível a distinguir as zonas navegáveis das não navegáveis em contexto de estrada, permitindo assim a outros processos fazer um planeamento local da navegação.The AtlasCar vehicle is a prototype developed by the Laboratory of Automation and Robotics, Department of Mechanical Engineering of the University of Aveiro, for the study of active and passive safety systems and autonomous driving solutions. The objective of this work is to provide the AtlasCar an artificial vision system able to recognize road borders, identifying road lane markings and their characteristics. This work is divided into two main parts: development of software modules to implement algorithms with different detection methodologies and the study of parameters that compose them. The first algorithm is based on RANSAC method for performing spline fitting to the road limits, the second algoritm uses the Hough transform to make this fitting. The methods performed were tested under real conditions, and failure situations, advantages and drawbacks of their applications are analysed and compared. With the implementation of this system it will be possible to distinguish navigable from non-navigable road surface, thus endowing path planning algorithms to operate in local context

Exploración de tolerancia a imprecisiones en aplicaciones ADAS y su efecto en un sistema de control

Proyecto de Graduación (Licenciatura en Ingeniería Mecatrónica) Instituto Tecnológico de Costa Rica. Área Académica de Ingeniería Mecatrónica, 2018Los vehículos que día a día son conducidos en las calles poseen Sistemas Avanzados de Asistencia el Conductor (Advanced Driver Assistance Systems, ADAS). Estas aplicaciones ADAS le brindan apoyo al conductor para facilitarle la tarea de conducir e inclusive pueda desentenderse de algunas tareas. Este desarrollo tecnológico en la industria automotriz tiene como meta llegar a la conducción autónoma de vehículos en un 100%. El reto que se planteó para el desarrollo de este proyecto fue desarrollar 3 de estas aplicaciones que involucren Visión por Computadora (Computer Vision, CV) y comprobar su tolerancia a imprecisiones mediante la implementación Computación Aproximada (Aproximate Computing, AC), lo cual aunado a un medio de pruebas donde permite observar su efecto en un sistema de control y comprobar así contra el funcionamiento “exacto" de las mismas aplicaciones.The day-to-day vehicles that are driven on the streets have Advanced Driver Assistance Systems (ADAS). These ADAS applications provide support to the driver to facilitate the task of driving and even can get off tasks. This technological development in the automotive industry aims to reach the autonomous driving of vehicles by 100%. The challenge for the development of this project is to develop 3 of these applications involving Computer Vision (CV) and test their tolerance to errors through Approximate Computing (AC) implementation, this coupled with a test medium where its effect will be seen in a system of Control and will be checked against the "exact" operation of the same applications

Design, Data Collection, and Driver Behavior Simulation for the Open- Mode Integrated Transportation System (OMITS)

With the remarkable increase in the population and number of vehicles, traffic has become a severe problem in most metropolitan areas. Traffic congestion has imposed tight constraints on economic growth, national security, and mobility of riders and goods. The open-mode integrated transportation system (OMITS) has been designed to improve the traffic condition of roadways by increasing the ridership of vehicles and optimizing transportation modes through smart services integrating emerging information communication technologies, big data management, social networking, and transportation management. Even a modest reduction in the number of vehicles on roadways will lead to a considerable cost savings in terms of time and money. Additionally the reduction in traffic jams will lead to a significant decrease in both gasoline consumption and greenhouse gas emissions. As a result, novel transportation management is critical to reduce vehicle mileage in the peak time of the road network. The OMITS was proposed to enhance transportation services in respect to the following three aspects: optimization of the transportation modes by multimodal traveling assignment, dynamic routing and ridesharing service with advanced traveler information systems, and interactive user interface for social networking and traveling information. Therefore, the OMITS encompasses a broad range of advanced transportation research topics, say dynamic trip- match, transportation-mode optimization, traffic prediction, dynamic routing, and social network- based carpooling. This dissertation will focus on a kernel part of the OMITS, namely traffic simulation and prediction based on data containing the distribution of vehicles and the road network configuration. A microscopic traffic simulation framework has been developed to take into account various traffic phenomena, such as traffic jams resulting from bottlenecking, incidents, and traffic flow shock waves. Four fundamental contributions of the present study are summarized as follows: Firstly, an accurate and robust vehicle trajectory data collection method based on image data of unmanned aerial vehicle (UAV) has been presented, which can be used to rapidly and accurately acquire the real-time traffic conditions of the region of interest. Historically, a lack in the availability of trajectory data has posed a significant obstacle to the enhancement of microscopic simulation models. To overcome this obstacle, a UAV based vehicle trajectory data collection algorithm has been developed. This method extracts vehicle trajectory data from the UAV’s video at different altitudes with different view scopes. Compared with traditional methods, the present data collection algorithm incorporates many unique features to customize the vehicle and traffic flow, through which vehicle detection and tracking system accuracy can be considerably increased. Secondly, an open mechanics-based acceleration model has been presented to simulate the longitudinal motion of vehicles, in which five general factors—namely the subject vehicle’s speed and acceleration sensitivity, safety consideration, relative speed sensitivity and gap reducing desire—have been identified to describe drivers’ preferences and the interactions between vehicles. Inspired by the similarity between vehicle interactions and particle interactions, a mechanical system with force elements has been introduced to quantify the vehicle’s acceleration. Accordingly, each of the aforementioned five factors are assumed to function as an individual trigger to alter each vehicle’s speed. Based on Newton’s second law of motion, the subject vehicle’s longitudinal behavior can be simulated by the present open mechanics-based acceleration model. By introducing feeling gap, multilane acceleration behavior is included in the presented model. The simulation results fit realistic conditions for the traffic flow and the road capacity very well, where traffic shockwaves can be observed for a certain range of the traffic density. This model can be extended to more general scenarios if other factors can be recognized and introduced into the modeling framework. Thirdly, a driver decision-based lane change execution model has been developed to describe a vehicle’s lane change execution process, which includes two steps, i.e. driver’s lane selection and lane change execution. Currently, most lane change models focus on the driver’s lane selection, and overlook the driver’s behavior during a process of lane change execution which plays a significant role in the simulation of traffic flow characteristics. In this model, a lane change execution is analyzed as a driver’s decision-making process, which consists of desire point setting, priority decision-making, corresponding actions and achievement of consensus analysis. Compared with the traditional lane change execution models, the present model describes a realistic lane change process, and it provides more accurate and detailed simulation results in the microscopic traffic simulation. Based on the presented open mechanics-based acceleration model and the driver decision- based lane change execution model, a reverse lane change model has further been developed to simulate some complex traffic situations such as reverse lane change process at a two-way-two- lane road section where one lane is blocked by a traffic incident. Based on this reverse lane change model, information on the average waiting time and road capability can be obtained. The simulation results show that the present model is able to reflect real driver behavior and the corresponding traffic phenomenon during a reverse lane change process Through a homogenization process of the microscopic vehicle motion, we can obtain the macroscopic traffic flow of the roadway network within certain time and spatial ranges, which will be integrated into the OMITS system for traffic prediction. The validation of the models through future OMITS operations will also enable them to be high fidelity models in future driverless technologies and autonomous vehicles

Políticas de Copyright de Publicações Científicas em Repositórios Institucionais: O Caso do INESC TEC

A progressiva transformação das práticas científicas, impulsionada pelo desenvolvimento das novas Tecnologias de Informação e Comunicação (TIC), têm possibilitado aumentar o acesso à informação, caminhando gradualmente para uma abertura do ciclo de pesquisa. Isto permitirá resolver a longo prazo uma adversidade que se tem colocado aos investigadores, que passa pela existência de barreiras que limitam as condições de acesso, sejam estas geográficas ou financeiras. Apesar da produção científica ser dominada, maioritariamente, por grandes editoras comerciais, estando sujeita às regras por estas impostas, o Movimento do Acesso Aberto cuja primeira declaração pública, a Declaração de Budapeste (BOAI), é de 2002, vem propor alterações significativas que beneficiam os autores e os leitores. Este Movimento vem a ganhar importância em Portugal desde 2003, com a constituição do primeiro repositório institucional a nível nacional. Os repositórios institucionais surgiram como uma ferramenta de divulgação da produção científica de uma instituição, com o intuito de permitir abrir aos resultados da investigação, quer antes da publicação e do próprio processo de arbitragem (preprint), quer depois (postprint), e, consequentemente, aumentar a visibilidade do trabalho desenvolvido por um investigador e a respetiva instituição. O estudo apresentado, que passou por uma análise das políticas de copyright das publicações científicas mais relevantes do INESC TEC, permitiu não só perceber que as editoras adotam cada vez mais políticas que possibilitam o auto-arquivo das publicações em repositórios institucionais, como também que existe todo um trabalho de sensibilização a percorrer, não só para os investigadores, como para a instituição e toda a sociedade. A produção de um conjunto de recomendações, que passam pela implementação de uma política institucional que incentive o auto-arquivo das publicações desenvolvidas no âmbito institucional no repositório, serve como mote para uma maior valorização da produção científica do INESC TEC.The progressive transformation of scientific practices, driven by the development of new Information and Communication Technologies (ICT), which made it possible to increase access to information, gradually moving towards an opening of the research cycle. This opening makes it possible to resolve, in the long term, the adversity that has been placed on researchers, which involves the existence of barriers that limit access conditions, whether geographical or financial. Although large commercial publishers predominantly dominate scientific production and subject it to the rules imposed by them, the Open Access movement whose first public declaration, the Budapest Declaration (BOAI), was in 2002, proposes significant changes that benefit the authors and the readers. This Movement has gained importance in Portugal since 2003, with the constitution of the first institutional repository at the national level. Institutional repositories have emerged as a tool for disseminating the scientific production of an institution to open the results of the research, both before publication and the preprint process and postprint, increase the visibility of work done by an investigator and his or her institution. The present study, which underwent an analysis of the copyright policies of INESC TEC most relevant scientific publications, allowed not only to realize that publishers are increasingly adopting policies that make it possible to self-archive publications in institutional repositories, all the work of raising awareness, not only for researchers but also for the institution and the whole society. The production of a set of recommendations, which go through the implementation of an institutional policy that encourages the self-archiving of the publications developed in the institutional scope in the repository, serves as a motto for a greater appreciation of the scientific production of INESC TEC