Towards a Common Software/Hardware Methodology for Future Advanced Driver Assistance Systems

The European research project DESERVE (DEvelopment platform for Safe and Efficient dRiVE, 2012-2015) had the aim of designing and developing a platform tool to cope with the continuously increasing complexity and the simultaneous need to reduce cost for future embedded Advanced Driver Assistance Systems (ADAS). For this purpose, the DESERVE platform profits from cross-domain software reuse, standardization of automotive software component interfaces, and easy but safety-compliant integration of heterogeneous modules. This enables the development of a new generation of ADAS applications, which challengingly combine different functions, sensors, actuators, hardware platforms, and Human Machine Interfaces (HMI). This book presents the different results of the DESERVE project concerning the ADAS development platform, test case functions, and validation and evaluation of different approaches. The reader is invited to substantiate the content of this book with the deliverables published during the DESERVE project. Technical topics discussed in this book include:Modern ADAS development platforms;Design space exploration;Driving modelling;Video-based and Radar-based ADAS functions;HMI for ADAS;Vehicle-hardware-in-the-loop validation system

Real-Time High-Resolution Multiple-Camera Depth Map Estimation Hardware and Its Applications

Depth information is used in a variety of 3D based signal processing applications such as autonomous navigation of robots and driving systems, object detection and tracking, computer games, 3D television, and free view-point synthesis. These applications require high accuracy and speed performances for depth estimation. Depth maps can be generated using disparity estimation methods, which are obtained from stereo matching between multiple images. The computational complexity of disparity estimation algorithms and the need of large size and bandwidth for the external and internal memory make the real-time processing of disparity estimation challenging, especially for high resolution images. This thesis proposes a high-resolution high-quality multiple-camera depth map estimation hardware. The proposed hardware is verified in real-time with a complete system from the initial image capture to the display and applications. The details of the complete system are presented. The proposed binocular and trinocular adaptive window size disparity estimation algorithms are carefully designed to be suitable to real-time hardware implementation by allowing efficient parallel and local processing while providing high-quality results. The proposed binocular and trinocular disparity estimation hardware implementations can process 55 frames per second on a Virtex-7 FPGA at a 1024 x 768 XGA video resolution for a 128 pixel disparity range. The proposed binocular disparity estimation hardware provides best quality compared to existing real-time high-resolution disparity estimation hardware implementations. A novel compressed-look up table based rectification algorithm and its real-time hardware implementation are presented. The low-complexity decompression process of the rectification hardware utilizes a negligible amount of LUT and DFF resources of the FPGA while it does not require the existence of external memory. The first real-time high-resolution free viewpoint synthesis hardware utilizing three-camera disparity estimation is presented. The proposed hardware generates high-quality free viewpoint video in real-time for any horizontally aligned arbitrary camera positioned between the leftmost and rightmost physical cameras. The full embedded system of the depth estimation is explained. The presented embedded system transfers disparity results together with synchronized RGB pixels to the PC for application development. Several real-time applications are developed on a PC using the obtained RGB+D results. The implemented depth estimation based real-time software applications are: depth based image thresholding, speed and distance measurement, head-hands-shoulders tracking, virtual mouse using hand tracking and face tracking integrated with free viewpoint synthesis. The proposed binocular disparity estimation hardware is implemented in an ASIC. The ASIC implementation of disparity estimation imposes additional constraints with respect to the FPGA implementation. These restrictions, their implemented efficient solutions and the ASIC implementation results are presented. In addition, a very high-resolution (82.3 MP) 360°x90° omnidirectional multiple camera system is proposed. The hemispherical camera system is able to view the target locations close to horizontal plane with more than two cameras. Therefore, it can be used in high-resolution 360° depth map estimation and its applications in the future

Towards a Common Software/Hardware Methodology for Future Advanced Driver Assistance Systems

The European research project DESERVE (DEvelopment platform for Safe and Efficient dRiVE, 2012-2015) had the aim of designing and developing a platform tool to cope with the continuously increasing complexity and the simultaneous need to reduce cost for future embedded Advanced Driver Assistance Systems (ADAS). For this purpose, the DESERVE platform profits from cross-domain software reuse, standardization of automotive software component interfaces, and easy but safety-compliant integration of heterogeneous modules. This enables the development of a new generation of ADAS applications, which challengingly combine different functions, sensors, actuators, hardware platforms, and Human Machine Interfaces (HMI). This book presents the different results of the DESERVE project concerning the ADAS development platform, test case functions, and validation and evaluation of different approaches. The reader is invited to substantiate the content of this book with the deliverables published during the DESERVE project. Technical topics discussed in this book include:Modern ADAS development platforms;Design space exploration;Driving modelling;Video-based and Radar-based ADAS functions;HMI for ADAS;Vehicle-hardware-in-the-loop validation system

Sensing and Signal Processing in Smart Healthcare

In the last decade, we have witnessed the rapid development of electronic technologies that are transforming our daily lives. Such technologies are often integrated with various sensors that facilitate the collection of human motion and physiological data and are equipped with wireless communication modules such as Bluetooth, radio frequency identification, and near-field communication. In smart healthcare applications, designing ergonomic and intuitive human–computer interfaces is crucial because a system that is not easy to use will create a huge obstacle to adoption and may significantly reduce the efficacy of the solution. Signal and data processing is another important consideration in smart healthcare applications because it must ensure high accuracy with a high level of confidence in order for the applications to be useful for clinicians in making diagnosis and treatment decisions. This Special Issue is a collection of 10 articles selected from a total of 26 contributions. These contributions span the areas of signal processing and smart healthcare systems mostly contributed by authors from Europe, including Italy, Spain, France, Portugal, Romania, Sweden, and Netherlands. Authors from China, Korea, Taiwan, Indonesia, and Ecuador are also included

Collision Avoidance on Unmanned Aerial Vehicles using Deep Neural Networks

Unmanned Aerial Vehicles (UAVs), although hardly a new technology, have recently gained a prominent role in many industries, being widely used not only among enthusiastic consumers but also in high demanding professional situations, and will have a massive societal impact over the coming years. However, the operation of UAVs is full of serious safety risks, such as collisions with dynamic obstacles (birds, other UAVs, or randomly thrown objects). These collision scenarios are complex to analyze in real-time, sometimes being computationally impossible to solve with existing State of the Art (SoA) algorithms, making the use of UAVs an operational hazard and therefore significantly reducing their commercial applicability in urban environments. In this work, a conceptual framework for both stand-alone and swarm (networked) UAVs is introduced, focusing on the architectural requirements of the collision avoidance subsystem to achieve acceptable levels of safety and reliability. First, the SoA principles for collision avoidance against stationary objects are reviewed. Afterward, a novel image processing approach that uses deep learning and optical flow is presented. This approach is capable of detecting and generating escape trajectories against potential collisions with dynamic objects. Finally, novel models and algorithms combinations were tested, providing a new approach for the collision avoidance of UAVs using Deep Neural Networks. The feasibility of the proposed approach was demonstrated through experimental tests using a UAV, created from scratch using the framework developed.Os veículos aéreos não tripulados (VANTs), embora dificilmente considerados uma nova tecnologia, ganharam recentemente um papel de destaque em muitas indústrias, sendo amplamente utilizados não apenas por amadores, mas também em situações profissionais de alta exigência, sendo expectável um impacto social massivo nos próximos anos. No entanto, a operação de VANTs está repleta de sérios riscos de segurança, como colisões com obstáculos dinâmicos (pássaros, outros VANTs ou objetos arremessados). Estes cenários de colisão são complexos para analisar em tempo real, às vezes sendo computacionalmente impossível de resolver com os algoritmos existentes, tornando o uso de VANTs um risco operacional e, portanto, reduzindo significativamente a sua aplicabilidade comercial em ambientes citadinos. Neste trabalho, uma arquitectura conceptual para VANTs autônomos e em rede é apresentada, com foco nos requisitos arquitetônicos do subsistema de prevenção de colisão para atingir níveis aceitáveis de segurança e confiabilidade. Os estudos presentes na literatura para prevenção de colisão contra objectos estacionários são revistos e uma nova abordagem é descrita. Esta tecnica usa técnicas de aprendizagem profunda e processamento de imagem, para realizar a prevenção de colisões em tempo real com objetos móveis. Por fim, novos modelos e combinações de algoritmos são propostos, fornecendo uma nova abordagem para evitar colisões de VANTs usando Redes Neurais Profundas. A viabilidade da abordagem foi demonstrada através de testes experimentais utilizando um VANT, desenvolvido a partir da arquitectura apresentada

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

Metalik yansımalı yüzeylerde otomatik çizik tespiti için görüntü işleme sistemi.

In industry, problems due to human error, mechanical flaws and transportation may occur; besides, they need to be detected in fast and efficient ways. In order to eliminate failure of human inspection, automated systems come in action, usually image processing involved. This thesis work, targets one common mass production problem on specular surfaces, i.e. scratch detection. To achieve this, we have implemented two different prototypes. The low-cost system is based on basic line detection, and the mid-end system depends on learning based detection. Both systems are implemented on embedded platforms and performance comparisons are done. Detailed analysis is carried out on computational cost and detection performance. This real-world episode is done on a mechanical prototype in laboratory environmentM.S. - Master of Scienc

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

SMART CITY MANAGEMENT USING MACHINE LEARNING TECHNIQUES

In response to the growing urban population, smart cities are designed to improve people\u27s quality of life by implementing cutting-edge technologies. The concept of a smart city refers to an effort to enhance a city\u27s residents\u27 economic and environmental well-being via implementing a centralized management system. With the use of sensors and actuators, smart cities can collect massive amounts of data, which can improve people\u27s quality of life and design cities\u27 services. Although smart cities contain vast amounts of data, only a percentage is used due to the noise and variety of the data sources. Information and communication technology (ICT) and the Internet of Things (IoT) play a far more prominent role in developing smart cities when it comes to making choices, designing policies, and executing different methods. Smart city applications have made great strides thanks to recent advances in artificial intelligence (AI), especially machine learning (ML) and deep learning (DL). The applications of ML and DL have significantly increased the accuracy aspect of decision-making in smart cities, especially in analyzing the captured data using IoT-based devices and sensors. Smart cities employ algorithms that use unlabeled and labeled data to manage resources and deliver individualized services effectively. It has instantaneous practical use in many crucial areas, including smart health, smart environment, smart transportation system, energy management, and smart water distribution system in a smart city. Hence, ML and DL have become hot research topics in AI techniques in recent years and are proving to be accurate optimization techniques in smart cities. In addition, artificial intelligence algorithms enable the processing massive datasets and identify patterns and characteristics that would otherwise go unnoticed. Despite these advantages, researchers\u27 skepticism of AI\u27s sometimes mysterious inner workings has prevented it from being widely used for smart cities. This thesis\u27s primary intent is to explore the value of employing diverse AI and ML techniques in developing smart city-centric domains and investigate the efficacy of these proposed approaches in four different aspects of the smart city such as smart energy, smart transportation system, smart water distribution system and smart environment. In addition, we use these machine learning approaches to make a data analytics and visualization unit module for the smart city testbed. Internet-of-Things-based machine learning approaches in diverse aspects have repeatedly demonstrated greater accuracy, sensitivity, cost-effectiveness, and productivity, used in the built-in Virginia Commonwealth University\u27s real-time testbed

The Pull Paradigm : foundations of user-centric advanced driver assistant systems based on bidirectional car2X communication

This thesis develops applications for vehicular ad-hoc networks that go far beyond the currently established areas of driving safety and traffic efficiency. The ad-hoc network is regarded as a dynamic information resource which is available to any vehicle at any time. In contrast to current state-of-the-art research, the proposed Pull Paradigm starts at the user\u27s vehicle rather than at an information source somewhere in the network, e.g. a braking car. To access information from highly dynamic ad-hoc networks, bidirectional communication and information discovery and retrieval play a vital role. Therefore, in the course of the work, the applicability of the Pull Paradigm to established vehicular ad-hoc networks is thoroughly examined and missing aspects are identified. It turns out that a number of enhancements to almost all layers of the network stack are necessary in order to apply the Pull Paradigm using existing technology. The central elements here are two novel algorithms for managing information flow and dissemination in ad-hoc networks, which are at first formulated from the abstract perspective of graph theory. Using the knowledge gained leads to the development of PADE, a platform that supports development of vehicular ad-hoc network applications. The designed algorithms are then implemented as a routing scheme, integrated and evaluated in large, simulated city scenarios. Furthermore, PADE combines real\u27\u27 and simulated communication technologies and abstracts from them, so that applications can be transferred from the lab into a test vehicle with minimal effort. In order to achieve this ambitious goal, PADE builds on a number of existing simulation and communication technologies. The practical applicability of the Pull approach is shown in two demonstrators that are integrated into a BMW 5 series test vehicle. The presentation module of the PADE platform was tested in the currently largest field operational test for vehicular ad-hoc communication. Over 400 drivers in 120 vehicles experienced the system on a daily basis.In dieser Doktorarbeit werden Anwendungen für Fahrzeug Ad-hoc Netzwerke erarbeitet, die weit über die derzeit etablierten Bereiche der Fahrsicherheit und Verkehrseffizienz hinausgehen. Das Ad-hoc Netzwerk wird dabei als dynamische Informationsressource angesehen, die jedem Fahrzeug zu jedem Zeitpunkt zur Verfügung steht. Im Gegensatz zum derzeitigen Stand der Forschung geht das vorgestellte Pull Paradigma vom Fahrzeug des Benutzers und nicht von der Informationsquelle aus, z.B. einem bremsenden Fahrzeug. Für den Zugriff auf Informationen aus hochdynamischen Ad-hoc Netzen, spielen bidirektionale Kommunikation, Informationssuche und -rücktransport eine entscheidende Rolle. Im Verlauf der Arbeit wird deshalb die Anwendbarkeit des Pull Paradigmas auf etablierte Fahrzeug Ad-hoc Netze untersucht und fehlende Aspekte identifiziert. Es zeigt sich, dass eine Reihe an Erweiterungen auf fast allen Ebenen des Netzwerkstapels nötig sind damit die bestehende Technologie um das Pull Paradigma erweitert werden kann. Zentraler Punkt hierbei sind zwei neuartige Algorithmen zur Informationsverwaltung und -verbreitung in Ad-hoc Netzwerken die zunächst abstrakt aus Sicht der Graphentheorie formuliert werden. Mit Hilfe der gewonnenen Erkenntnisse wird PADE, eine Plattform zur Entwicklung von Anwendungen für Fahrzeug Ad-hoc Netze, entwickelt. Die entworfenen Algorithmen werden dann als Routingverfahren im Netzwerkstapel realisiert, in diesen integriert und auf großflächigen Stadtszenarien im Simulator evaluiert. Des Weiteren vereint PADE echte\u27\u27 und simulierte Kommunikationstechnologien und abstrahiert von diesen, sodass Anwendungen mit minimalem Aufwand vom Labor in ein Testfahrzeug überführt werden können. Um dieses ambitionierte Ziel zu erreichen, wird auf einer Reihe bereits bestehender Simulations- und Kommunikationstechnologien aufgebaut. Die praktische Anwendbarkeit des Pull Paradigmas wird anschließend in zwei Demonstratoren implementiert und in ein BMW 5er Testfahrzeug integriert. Das Präsentationsmodul der PADE Plattform wurde im derzeit weltgrößten Feldversuch für Fahrzeug Ad-hoc Kommunikation von über 400 Fahrern in 120 Fahrzeugen im Alltag getestet