Radar based on automotive pedestrian detection using the micro Doppler effects

Orientador: Prof. Dr. Alessandro ZimmerDissertação (mestrado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Elétrica. Defesa : Curitiba, 29/08/2018Inclui referências: p.74-78Resumo: O desenvolvimento do carro autônomo é hoje em dia uma prática comum entre as maiores indústrias automotivas, e também em indústrias tecnológicas, como o Google e a Apple. Ao adicionar mais sensores, o veículo é capaz de se movimentar sozinho, identificar a trajetória correta, a distância para outros carros, e também a presença de objetos e seres vivos. Entretanto, existem muitos aspectos bloqueando o lançamento do carro autônomo. Como exemplo aspectos técnicos, como o caso do reconhecimento de pedestres. Embora, esse tópico seja abundantemente estudado para o uso de câmeras digitais, as mesmas não possuem confiabilidade nas medições de velocidade e distância, e ainda apresentam péssimos resultados quando há variação ou a falta de luz no ambiente. Baseado no que foi mencionado anteriormente, o foco dessa dissertação é de desenvolver e discutir a eficiência de um sistema de rápida identificação de pedestres, utilizando um novo radar de 79GHz de frequência. O principal objetivo é reconhecer o pedestre o mais rápido possível utilizando os efeitos micro Doppler do movimento humano em situações muito próximas de um acidente, junto com o método de classificação support vector machine (SVM). Objetivando essa meta algumas técnicas são usadas ao longo do trabalho. Primeiramente, a resolução de velocidade é melhorada com técnicas de otimização multiobjetivos, como algoritmos genéticos e random search para extrair o micro efeito Doppler. Então as informações de velocidade e distância são medidas pelo radar. Em sequência, um método de extração de características chamado de video temporal gradiente é aplicado. O método de machine learning SVM classifica os objetos em pedestre e não pedestres, com quadro diferentes métodos de treinamento. Por fim, é possível ver as vantagens do método de otimização que consegue atingir uma resolução de velocidade de 0,12 m/s. A comparação dos modelos de SVM mostra que o quarto modelo, utilizando kernel polinomial, apresenta os melhores resultados com uma acurácia de 99,5%. Entretanto, o tempo de processamento não é bom o suficiente, levando 72 ms para a classificação de um objeto. Palavras-Chaves: Carro autônomo. Reconhecimento de pedestres. Micro Doppler. Otimização multiobjectivos. Support vector machine.Abstract: The development of the autonomous car is nowadays a common practice in all the greatest automotive factories in the world, also in companies outside the automotive market, like Google and Apple. By adding more sensors, the vehicle is now capable of moving alone, identifying the correct path, the distance from another cars, also the presence of objects and people. However, there are still many issues blocking the autonomous car to be released. There are technical aspects to be solved, as the pedestrian recognition issues. Although, the recognition is widely studied and applied using cameras and digital images, there are issues to be improved. Like the distance and velocity reliability and the problems occurred because the lack of light in the environment. Based on the before mentioned, the focus in this presented work is to develop and discuss the efficiency of a pedestrian recognition system, using one automotive radar of 79 GHz. The main goal is to early detect the pedestrian using the micro Doppler characteristics of a human body in near to crash situations. Aiming this goal some techniques are used in the work. Firstly, the velocity resolution is improved, in order to extract the micro Doppler characteristics of the objects. The improvement of velocity resolution is reached by the use of multiobjective optimization techniques, as genetic algorithm and random search. The information about velocity and range is measured by the radar. In sequence a simple feature extraction method called video temporal gradient transform the data. The result is used in a machine learning technique called support vector machine (SVM). Which classifies the objects between pedestrians and non-pedestrians, with four different approaches. Concluding the work, it is possible to see the advantages of the multiobjective optimization in order to extract the micro Doppler effects. The optimization reached the velocity resolution of 0,12 m/s. The SVM comparison show that the fourth model with a polynomial kernel presented better result with accuracy 99,5%. However, the processing time of the system was not good enough taking 72 ms to identify an object. Keywords: Autonomous car. Pedestrian recognition. Micro Doppler. Multiobjective optimization. Support vector machine

Safe Intelligent Driver Assistance System in V2X Communication Environments based on IoT

In the modern world, power and speed of cars have increased steadily, as traffic continued to increase. At the same time highway-related fatalities and injuries due to road incidents are constantly growing and safety problems come first. Therefore, the development of Driver Assistance Systems (DAS) has become a major issue. Numerous innovations, systems and technologies have been developed in order to improve road transportation and safety. Modern computer vision algorithms enable cars to understand the road environment with low miss rates. A number of Intelligent Transportation Systems (ITSs), Vehicle Ad-Hoc Networks (VANETs) have been applied in the different cities over the world. Recently, a new global paradigm, known as the Internet of Things (IoT) brings new idea to update the existing solutions. Vehicle-to-Infrastructure communication based on IoT technologies would be a next step in intelligent transportation for the future Internet-of-Vehicles (IoV). The overall purpose of this research was to come up with a scalable IoT solution for driver assistance, which allows to combine safety relevant information for a driver from different types of in-vehicle sensors, in-vehicle DAS, vehicle networks and driver`s gadgets. This study brushed up on the evolution and state-of-the-art of Vehicle Systems. Existing ITSs, VANETs and DASs were evaluated in the research. The study proposed a design approach for the future development of transport systems applying IoT paradigm to the transport safety applications in order to enable driver assistance become part of Internet of Vehicles (IoV). The research proposed the architecture of the Safe Intelligent DAS (SiDAS) based on IoT V2X communications in order to combine different types of data from different available devices and vehicle systems. The research proposed IoT ARM structure for SiDAS, data flow diagrams, protocols. The study proposes several IoT system structures for the vehicle-pedestrian and vehicle-vehicle collision prediction as case studies for the flexible SiDAS framework architecture. The research has demonstrated the significant increase in driver situation awareness by using IoT SiDAS, especially in NLOS conditions. Moreover, the time analysis, taking into account IoT, Cloud, LTE and DSRS latency, has been provided for different collision scenarios, in order to evaluate the overall system latency and ensure applicability for real-time driver emergency notification. Experimental results demonstrate that the proposed SiDAS improves traffic safety

Development of an object detection and mask generation software for dynamic beam projection in automotive pixel lighting applications

Nowadays there are many contributions to the automotive industry and the field is developing fast. This work can be used for some real-time autonomous driving applications. The goal was to add advanced functionality to a standard light source in collaboration with electronic systems. Including advanced features may result in safer and more pleasant driving. The application fields of the work could include glare-free light sources, orientation and lane lights, marking lights, and symbol projection. On a real-time source, object detection and classification with a confidence score is implemented. The best model is obtained by intending to train the model with varying parameters. The most accurate result which is mAP value 0.572 was obtained by distributing the training dataset with learning rate 0.2 and setting the epochs to 300. Moreover, a basic implementation of a glare-free light source was done to avoid the drivers from being blinded by the illumination of the beams. The car and rectangle shape masks were generated as image files and sent as CSV files to the pixel light source device. As a result, the rectangle shaped mask functions more precisely then car shaped.Nowadays there are many contributions to the automotive industry and the field is developing fast. This work can be used for some real-time autonomous driving applications. The goal was to add advanced functionality to a standard light source in collaboration with electronic systems. Including advanced features may result in safer and more pleasant driving. The application fields of the work could include glare-free light sources, orientation and lane lights, marking lights, and symbol projection. On a real-time source, object detection and classification with a confidence score is implemented. The best model is obtained by intending to train the model with varying parameters. The most accurate result which is mAP value 0.572 was obtained by distributing the training dataset with learning rate 0.2 and setting the epochs to 300. Moreover, a basic implementation of a glare-free light source was done to avoid the drivers from being blinded by the illumination of the beams. The car and rectangle shape masks were generated as image files and sent as CSV files to the pixel light source device. As a result, the rectangle shaped mask functions more precisely then car shaped

Artificial Intelligence for the Edge Computing Paradigm.

With modern technologies moving towards the internet of things where seemingly every financial, private, commercial and medical transaction being carried out by portable and intelligent devices; Machine Learning has found its way into every smart device and application possible. However, Machine Learning cannot be used on the edge directly due to the limited capabilities of small and battery-powered modules. Therefore, this thesis aims to provide light-weight automated Machine Learning models which are applied on a standard edge device, the Raspberry Pi, where one framework aims to limit parameter tuning while automating feature extraction and a second which can perform Machine Learning classification on the edge traditionally, and can be used additionally for image-based explainable Artificial Intelligence. Also, a commercial Artificial Intelligence software have been ported to work in a client/server setups on the Raspberry Pi board where it was incorporated in all of the Machine Learning frameworks which will be presented in this thesis. This dissertation also introduces multiple algorithms that can convert images into Time-series for classification and explainability but also introduces novel Time-series feature extraction algorithms that are applied to biomedical data while introducing the concept of the Activation Engine, which is a post-processing block that tunes Neural Networks without the need of particular experience in Machine Leaning. Also, a tree-based method for multiclass classification has been introduced which outperforms the One-to-Many approach while being less complex that the One-to-One method.\par The results presented in this thesis exhibit high accuracy when compared with the literature, while remaining efficient in terms of power consumption and the time of inference. Additionally the concepts, methods or algorithms that were introduced are particularly novel technically, where they include: • Feature extraction of professionally annotated, and poorly annotated time-series. • The introduction of the Activation Engine post-processing block. • A model for global image explainability with inference on the edge. • A tree-based algorithm for multiclass classification

Deep learning based 3D object detection for automotive radar and camera fusion

La percepción en el dominio de los vehículos autónomos es una disciplina clave para lograr la automatización de los Sistemas Inteligentes de Transporte. Por ello, este Trabajo Fin de Máster tiene como objetivo el desarrollo de una técnica de fusión sensorial para RADAR y cámara que permita crear una representación del entorno enriquecida para la Detección de Objetos 3D mediante algoritmos Deep Learning. Para ello, se parte de la idea de PointPainting [1] y se adapta a un sensor en auge, el RADAR 3+1D, donde nube de puntos RADAR e información semántica de la cámara son agregadas para generar una representación enriquecida del entorno.Perception in the domain of autonomous vehicles is a key discipline to achieve the au tomation of Intelligent Transport Systems. Therefore, this Master Thesis aims to develop a sensor fusion technique for RADAR and camera to create an enriched representation of the environment for 3D Object Detection using Deep Learning algorithms. To this end, the idea of PointPainting [1] is used as a starting point and is adapted to a growing sensor, the 3+1D RADAR, in which the radar point cloud is aggregated with the semantic information from the camera.Máster Universitario en Ingeniería Industrial (M141