車載カメラを用いた障害物検出システムの研究

九州工業大学博士学位論文 学位記番号:工博甲第371号　学位授与年月日:平成26年9月26日1 Introduction||2 Obstacles Detection||3 2D and 3D Objects Classification||4 Final Experimental Results and Evaluation||5 Conclusion||ReferencesIn recent years, autonomous collision avoidance systems have been researched and developed for realizing safe driving using cameras and sensors. These systems are designed to warn the drivers the presence of obstacles on the road and help them take a necessary action in advance. In these systems, the ability to detect obstacles is essential. Although various methods of obstacles detection have already been reported, these existing obstacles detection methods have some inadequacies: Some of them can be only used to detect moving obstacles; Some of them cannot extract the shape of obstacles, and they only use a rectangular frame that surrounds an obstacle to represent a detected obstacle; Some of them can only be used to detect one kind of specific object, such as pedestrian detection or vehicle detection. In order to make up for the inadequacies of the existing obstacles detection method, in this thesis, a method is proposed for detecting obstacles on a road by the employment of the background modeling and the road region detection. In obstacles detection, true obstacles are defined as arbitrary objects which protrude from the ground plane in the road region, including static and moving objects. Road marks in the road region and objects outside the road region are considered as false obstacles. The output of this obstacles detection method is based on the obstacles’ shape. In this thesis, we also propose a method of classifying 2D objects and 3D objects. The results of 2D objects and 3D objects classification can be used in the resultant image of obstacles detection to delete 2D objects (such as road marks) and improve the accuracy of obstacles detection. The originalities of this thesis are as follows: In the first place, the proposed method can detect arbitrary objects including both static objects and moving objects. This is helpful because static objects such as boxes fallen on the road from a car are dangerous for drivers. In the second place, the output of the proposed method is the shape of obstacles. Extraction of the shape of an obstacle is important for obstacles recognition. If the detected obstacle is recognized as a pedestrian from its shape, we can foresee his/her next motion. In the third place, the proposed method can distinguish which objects are 3D objects, and which objects are 2D objects in a pile of objects using a monocular camera. It is useful in the obstacles detection and other applications, such as navigation of walking robots. In the performed experiments, it is shown that the proposed obstacles detection method is able to extract the shape of both static and moving obstacles in a frontal view from a car

On Creating Reference Data for Performance Analysis in Image Processing

This thesis investigates methods for the creation of reference datasets for image processing, especially for the dense correspondence problem. Three types of reference data can be identified: Real datasets with dense ground truth, real datasets with sparse or missing ground truth and synthetic datasets. For the creation of real datasets with ground truth a existing method based on depth map fusion was evaluated. The described method is especially suited for creating large amounts of reference data with known accuracy. The creation of reference datasets with missing ground truth was examined on the example of multiple datasets for the automotive industry. The data was used succesfully for verification and evaluation by multiple image processing projects. Finally, it was investigated how methods from computer graphics can be used for creating synthetic reference datasets. Especially the creation of photorealistic image sequences using global illumination has been examined for the task of evaluating algorithms. The results show that while such sequences can be used for evaluation, their creation is hindered by practicallity problems. As an application example, a new simulation method for Time-of-Flight depth cameras which can simulate all relevant error sources of these systems was developed

Naturalness framework for driver-car interaction

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonAutomobile dashboards are evolving into intelligent largely screen-based computer interfaces. Recent evidence suggests unnatural aspects of some secondary systems (including infotainment systems). Naturalness of interaction is a minority discipline not yet applied to the automobile; while automotive interface research is a mainly quantitative discipline that has not yet applied a naturalness approach. To advance the field, a measurement scale based on rigorous qualitative exploration of natural-feeling interaction with secondary controls was required. Study 1 used ethnographic interview with Contextual Inquiry inside 12 ordinary drivers’ cars, to investigate natural-feeling aspects of past, present and future driver-car interactions. Thematic analysis suggested a framework of ten characteristics. Half concerned control and physicality; half concerned perceived socio-intelligent behaviours of the car. Study 2 involved intensive exploratory workshops with ten drivers comprising Think Aloud, artefact modelling and focus groups, to explore natural-feeling interaction with secondary controls in different ways. The resulting thematic framework comprised 11 characteristics in four categories: familiarity/control, physical connection, low visual/cognitive demand, and humanlike intelligence and communication. Study 3 comprised two ethnographic participant observations. Eight drivers were observed interacting with their controls during long road journeys. Twenty-two drivers were observed interacting verbally with futuristic ‘intelligent’ secondary systems while driving on public roads. Design guidelines relating to physicality, usability, automation, and humanlike communication were formulated. Study 4 converted all the qualitative findings into a questionnaire comprising 46 bipolar five-point scales. Eighty-one drivers used it to rate one control in their cars. Correlation and factor analyses revealed three underlying factors and 14 items suitable for the first industrially applicable measurement scale for driver-car naturalness. These items concern perceived helpfulness, politeness, competence, predictability, control, ease, mental demands, intuitiveness, ‘realness’, instantaneousness, communication, logical location, mapping and 'affordance'. Initial testing found acceptable validity. The conclusion recommends further data collection, expanded validity testing, and potential applications to self-driving cars

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

Sistemas automáticos de informação e segurança para apoio na condução de veículos

Doutoramento em Engenharia MecânicaO objeto principal desta tese é o estudo de algoritmos de processamento e representação automáticos de dados, em particular de informação obtida por sensores montados a bordo de veículos (2D e 3D), com aplicação em contexto de sistemas de apoio à condução. O trabalho foca alguns dos problemas que, quer os sistemas de condução automática (AD), quer os sistemas avançados de apoio à condução (ADAS), enfrentam hoje em dia. O documento é composto por duas partes. A primeira descreve o projeto, construção e desenvolvimento de três protótipos robóticos, incluindo pormenores associados aos sensores montados a bordo dos robôs, algoritmos e arquitecturas de software. Estes robôs foram utilizados como plataformas de ensaios para testar e validar as técnicas propostas. Para além disso, participaram em várias competições de condução autónoma tendo obtido muito bons resultados. A segunda parte deste documento apresenta vários algoritmos empregues na geração de representações intermédias de dados sensoriais. Estes podem ser utilizados para melhorar técnicas já existentes de reconhecimento de padrões, deteção ou navegação, e por este meio contribuir para futuras aplicações no âmbito dos AD ou ADAS. Dado que os veículos autónomos contêm uma grande quantidade de sensores de diferentes naturezas, representações intermédias são particularmente adequadas, pois podem lidar com problemas relacionados com as diversas naturezas dos dados (2D, 3D, fotométrica, etc.), com o carácter assíncrono dos dados (multiplos sensores a enviar dados a diferentes frequências), ou com o alinhamento dos dados (problemas de calibração, diferentes sensores a disponibilizar diferentes medições para um mesmo objeto). Neste âmbito, são propostas novas técnicas para a computação de uma representação multi-câmara multi-modal de transformação de perspectiva inversa, para a execução de correcção de côr entre imagens de forma a obter mosaicos de qualidade, ou para a geração de uma representação de cena baseada em primitivas poligonais, capaz de lidar com grandes quantidades de dados 3D e 2D, tendo inclusivamente a capacidade de refinar a representação à medida que novos dados sensoriais são recebidos.The main object of this thesis is the study of algorithms for automatic information processing and representation, in particular information provided by onboard sensors (2D and 3D), to be used in the context of driving assistance. The work focuses on some of the problems facing todays Autonomous Driving (AD) systems and Advanced Drivers Assistance Systems (ADAS). The document is composed of two parts. The first part describes the design, construction and development of three robotic prototypes, including remarks about onboard sensors, algorithms and software architectures. These robots were used as test beds for testing and validating the developed techniques; additionally, they have participated in several autonomous driving competitions with very good results. The second part of this document presents several algorithms for generating intermediate representations of the raw sensor data. They can be used to enhance existing pattern recognition, detection or navigation techniques, and may thus benefit future AD or ADAS applications. Since vehicles often contain a large amount of sensors of different natures, intermediate representations are particularly advantageous; they can be used for tackling problems related with the diverse nature of the data (2D, 3D, photometric, etc.), with the asynchrony of the data (multiple sensors streaming data at different frequencies), or with the alignment of the data (calibration issues, different sensors providing different measurements of the same object). Within this scope, novel techniques are proposed for computing a multicamera multi-modal inverse perspective mapping representation, executing color correction between images for obtaining quality mosaics, or to produce a scene representation based on polygonal primitives that can cope with very large amounts of 3D and 2D data, including the ability of refining the representation as new information is continuously received

Exhaust system energy management of internal combustion engines

Today, the investigation of fuel economy improvements in internal combustion engines (ICEs) has become the most significant research interest among the automobile manufacturers and researchers. The scarcity of natural resources, progressively increasing oil prices, carbon dioxide taxation and stringent emission regulations all make fuel economy research relevant and compelling. The enhancement of engine performance solely using incylinder techniques is proving increasingly difficult and as a consequence the concept of exhaust energy recovery has emerged as an area of considerable interest. Three main energy recovery systems have been identified that are at various stages of investigation. Vapour power bottoming cycles and turbo-compounding devices have already been applied in commercially available marine engines and automobiles. Although the fuel economy benefits are substantial, system design implications have limited their adaptation due to the additional components and the complexity of the resulting system. In this context, thermo-electric (TE) generation systems, though still in their infancy for vehicle applications have been identified as attractive, promising and solid state candidates of low complexity. The performance of these devices is limited to the relative infancy of materials investigations and module architectures. There is great potential to be explored. The initial modelling work reported in this study shows that with current materials and construction technology, thermo-electric devices could be produced to displace the alternator of the light duty vehicles, providing the fuel economy benefits of 3.9%-4.7% for passenger cars and 7.4% for passenger buses. More efficient thermo-electric materials could increase the fuel economy significantly resulting in a substantially improved business case. The dynamic behaviour of the thermo-electric generator (TEG) applied in both, main exhaust gas stream and exhaust gas recirculation (EGR) path of light duty and heavy duty engines were studied through a series of experimental and modelling programs. The analyses of the thermo-electric generation systems have highlighted the need for advanced heat exchanger design as well as the improved materials to enhance the performance of these systems. These research requirements led to the need for a systems evaluation technique typified by hardware-in-the-loop (HIL) testing method to evaluate heat exchange and materials options. HIL methods have been used during this study to estimate both the output power and the exhaust back pressure created by the device. The work has established the feasibility of a new approach to heat exchange devices for thermo-electric systems. Based on design projections and the predicted performance of new materials, the potential to match the performance of established heat recovery methods has been demonstrated

Sensors Fault Diagnosis Trends and Applications

Fault diagnosis has always been a concern for industry. In general, diagnosis in complex systems requires the acquisition of information from sensors and the processing and extracting of required features for the classification or identification of faults. Therefore, fault diagnosis of sensors is clearly important as faulty information from a sensor may lead to misleading conclusions about the whole system. As engineering systems grow in size and complexity, it becomes more and more important to diagnose faulty behavior before it can lead to total failure. In the light of above issues, this book is dedicated to trends and applications in modern-sensor fault diagnosis

Design and semantics of form and movement : DeSForM 2007

A strong theme that has emerged in our previous two conferences in the importance of narrative to the process of generating, developing and communicating new modalities of interaction between people, things and environments. Our researches have identified aspects of importance in the design and have begun to establish orders of, priority of approach and representation for these aspects as components of interaction. We have begun to grapple with the growth in the complexity of the interaction design process for truly ‘animated’ functionality in products, especially where this manifests itself as apparent behavioural characteristics resident in or portrayed by products. The findings and experience of researchers is that this increase in complexity is likely to be exponential compared to the rigours relating to the resolution of static physical product configuration or even system operated product with screen based interfaces. The emerging sense is that narrative in the process is essential to bring meaning and to ‘touch’ our humanity or connect with human experience. ‘The science of the artificial in conversation with the poetics of human experience’! Through this conference we will once again engage in presentations, debate and demonstrations on these issues. In this respect we, the conference co-chairs, have sought to bring together researchers from academia, industry and professional design practice and related disciplines connected with interactive product service and system development to share our latest thinking in the field, to asses its outcomes and to identify further research questions, opportunities and territories for future investigation and exploration

Design and semantics of form and movement : DeSForM 2007

A strong theme that has emerged in our previous two conferences in the importance of narrative to the process of generating, developing and communicating new modalities of interaction between people, things and environments. Our researches have identified aspects of importance in the design and have begun to establish orders of, priority of approach and representation for these aspects as components of interaction. We have begun to grapple with the growth in the complexity of the interaction design process for truly ‘animated’ functionality in products, especially where this manifests itself as apparent behavioural characteristics resident in or portrayed by products. The findings and experience of researchers is that this increase in complexity is likely to be exponential compared to the rigours relating to the resolution of static physical product configuration or even system operated product with screen based interfaces. The emerging sense is that narrative in the process is essential to bring meaning and to ‘touch’ our humanity or connect with human experience. ‘The science of the artificial in conversation with the poetics of human experience’! Through this conference we will once again engage in presentations, debate and demonstrations on these issues. In this respect we, the conference co-chairs, have sought to bring together researchers from academia, industry and professional design practice and related disciplines connected with interactive product service and system development to share our latest thinking in the field, to asses its outcomes and to identify further research questions, opportunities and territories for future investigation and exploration