Nonlinear Modeling and Control of Driving Interfaces and Continuum Robots for System Performance Gains

With the rise of (semi)autonomous vehicles and continuum robotics technology and applications, there has been an increasing interest in controller and haptic interface designs. The presence of nonlinearities in the vehicle dynamics is the main challenge in the selection of control algorithms for real-time regulation and tracking of (semi)autonomous vehicles. Moreover, control of continuum structures with infinite dimensions proves to be difficult due to their complex dynamics plus the soft and flexible nature of the manipulator body. The trajectory tracking and control of automobile and robotic systems requires control algorithms that can effectively deal with the nonlinearities of the system without the need for approximation, modeling uncertainties, and input disturbances. Control strategies based on a linearized model are often inadequate in meeting precise performance requirements. To cope with these challenges, one must consider nonlinear techniques. Nonlinear control systems provide tools and methodologies for enabling the design and realization of (semi)autonomous vehicle and continuum robots with extended specifications based on the operational mission profiles. This dissertation provides an insight into various nonlinear controllers developed for (semi)autonomous vehicles and continuum robots as a guideline for future applications in the automobile and soft robotics field. A comprehensive assessment of the approaches and control strategies, as well as insight into the future areas of research in this field, are presented.First, two vehicle haptic interfaces, including a robotic grip and a joystick, both of which are accompanied by nonlinear sliding mode control, have been developed and studied on a steer-by-wire platform integrated with a virtual reality driving environment. An operator-in-the-loop evaluation that included 30 human test subjects was used to investigate these haptic steering interfaces over a prescribed series of driving maneuvers through real time data logging and post-test questionnaires. A conventional steering wheel with a robust sliding mode controller was used for all the driving events for comparison. Test subjects operated these interfaces for a given track comprised of a double lane-change maneuver and a country road driving event. Subjective and objective results demonstrate that the driver’s experience can be enhanced up to 75.3% with a robotic steering input when compared to the traditional steering wheel during extreme maneuvers such as high-speed driving and sharp turn (e.g., hairpin turn) passing. Second, a cellphone-inspired portable human-machine-interface (HMI) that incorporated the directional control of the vehicle as well as the brake and throttle functionality into a single holistic device will be presented. A nonlinear adaptive control technique and an optimal control approach based on driver intent were also proposed to accompany the mechatronic system for combined longitudinal and lateral vehicle guidance. Assisting the disabled drivers by excluding extensive arm and leg movements ergonomically, the device has been tested in a driving simulator platform. Human test subjects evaluated the mechatronic system with various control configurations through obstacle avoidance and city road driving test, and a conventional set of steering wheel and pedals were also utilized for comparison. Subjective and objective results from the tests demonstrate that the mobile driving interface with the proposed control scheme can enhance the driver’s performance by up to 55.8% when compared to the traditional driving system during aggressive maneuvers. The system’s superior performance during certain vehicle maneuvers and approval received from the participants demonstrated its potential as an alternative driving adaptation for disabled drivers. Third, a novel strategy is designed for trajectory control of a multi-section continuum robot in three-dimensional space to achieve accurate orientation, curvature, and section length tracking. The formulation connects the continuum manipulator dynamic behavior to a virtual discrete-jointed robot whose degrees of freedom are directly mapped to those of a continuum robot section under the hypothesis of constant curvature. Based on this connection, a computed torque control architecture is developed for the virtual robot, for which inverse kinematics and dynamic equations are constructed and exploited, with appropriate transformations developed for implementation on the continuum robot. The control algorithm is validated in a realistic simulation and implemented on a six degree-of-freedom two-section OctArm continuum manipulator. Both simulation and experimental results show that the proposed method could manage simultaneous extension/contraction, bending, and torsion actions on multi-section continuum robots with decent tracking performance (e.g. steady state arc length and curvature tracking error of 3.3mm and 130mm-1, respectively). Last, semi-autonomous vehicles equipped with assistive control systems may experience degraded lateral behaviors when aggressive driver steering commands compete with high levels of autonomy. This challenge can be mitigated with effective operator intent recognition, which can configure automated systems in context-specific situations where the driver intends to perform a steering maneuver. In this article, an ensemble learning-based driver intent recognition strategy has been developed. A nonlinear model predictive control algorithm has been designed and implemented to generate haptic feedback for lateral vehicle guidance, assisting the drivers in accomplishing their intended action. To validate the framework, operator-in-the-loop testing with 30 human subjects was conducted on a steer-by-wire platform with a virtual reality driving environment. The roadway scenarios included lane change, obstacle avoidance, intersection turns, and highway exit. The automated system with learning-based driver intent recognition was compared to both the automated system with a finite state machine-based driver intent estimator and the automated system without any driver intent prediction for all driving events. Test results demonstrate that semi-autonomous vehicle performance can be enhanced by up to 74.1% with a learning-based intent predictor. The proposed holistic framework that integrates human intelligence, machine learning algorithms, and vehicle control can help solve the driver-system conflict problem leading to safer vehicle operations

Advanced Control and Estimation Concepts, and New Hardware Topologies for Future Mobility

According to the National Research Council, the use of embedded systems throughout society could well overtake previous milestones in the information revolution. Mechatronics is the synergistic combination of electronic, mechanical engineering, controls, software and systems engineering in the design of processes and products. Mechatronic systems put “intelligence” into physical systems. Embedded sensors/actuators/processors are integral parts of mechatronic systems. The implementation of mechatronic systems is consistently on the rise. However, manufacturers are working hard to reduce the implementation cost of these systems while trying avoid compromising product quality. One way of addressing these conflicting objectives is through new automatic control methods, virtual sensing/estimation, and new innovative hardware topologies

Towards a Common Software/Hardware Methodology for Future Advanced Driver Assistance Systems

The European research project DESERVE (DEvelopment platform for Safe and Efficient dRiVE, 2012-2015) had the aim of designing and developing a platform tool to cope with the continuously increasing complexity and the simultaneous need to reduce cost for future embedded Advanced Driver Assistance Systems (ADAS). For this purpose, the DESERVE platform profits from cross-domain software reuse, standardization of automotive software component interfaces, and easy but safety-compliant integration of heterogeneous modules. This enables the development of a new generation of ADAS applications, which challengingly combine different functions, sensors, actuators, hardware platforms, and Human Machine Interfaces (HMI). This book presents the different results of the DESERVE project concerning the ADAS development platform, test case functions, and validation and evaluation of different approaches. The reader is invited to substantiate the content of this book with the deliverables published during the DESERVE project. Technical topics discussed in this book include:Modern ADAS development platforms;Design space exploration;Driving modelling;Video-based and Radar-based ADAS functions;HMI for ADAS;Vehicle-hardware-in-the-loop validation system

Towards a Common Software/Hardware Methodology for Future Advanced Driver Assistance Systems

The European research project DESERVE (DEvelopment platform for Safe and Efficient dRiVE, 2012-2015) had the aim of designing and developing a platform tool to cope with the continuously increasing complexity and the simultaneous need to reduce cost for future embedded Advanced Driver Assistance Systems (ADAS). For this purpose, the DESERVE platform profits from cross-domain software reuse, standardization of automotive software component interfaces, and easy but safety-compliant integration of heterogeneous modules. This enables the development of a new generation of ADAS applications, which challengingly combine different functions, sensors, actuators, hardware platforms, and Human Machine Interfaces (HMI). This book presents the different results of the DESERVE project concerning the ADAS development platform, test case functions, and validation and evaluation of different approaches. The reader is invited to substantiate the content of this book with the deliverables published during the DESERVE project. Technical topics discussed in this book include:Modern ADAS development platforms;Design space exploration;Driving modelling;Video-based and Radar-based ADAS functions;HMI for ADAS;Vehicle-hardware-in-the-loop validation system

A Context Aware Classification System for Monitoring Driver’s Distraction Levels

Understanding the safety measures regarding developing self-driving futuristic cars is a concern for decision-makers, civil society, consumer groups, and manufacturers. The researchers are trying to thoroughly test and simulate various driving contexts to make these cars fully secure for road users. Including the vehicle’ surroundings offer an ideal way to monitor context-aware situations and incorporate the various hazards. In this regard, different studies have analysed drivers’ behaviour under different case scenarios and scrutinised the external environment to obtain a holistic view of vehicles and the environment. Studies showed that the primary cause of road accidents is driver distraction, and there is a thin line that separates the transition from careless to dangerous. While there has been a significant improvement in advanced driver assistance systems, the current measures neither detect the severity of the distraction levels nor the context-aware, which can aid in preventing accidents. Also, no compact study provides a complete model for transitioning control from the driver to the vehicle when a high degree of distraction is detected. The current study proposes a context-aware severity model to detect safety issues related to driver’s distractions, considering the physiological attributes, the activities, and context-aware situations such as environment and vehicle. Thereby, a novel three-phase Fast Recurrent Convolutional Neural Network (Fast-RCNN) architecture addresses the physiological attributes. Secondly, a novel two-tier FRCNN-LSTM framework is devised to classify the severity of driver distraction. Thirdly, a Dynamic Bayesian Network (DBN) for the prediction of driver distraction. The study further proposes the Multiclass Driver Distraction Risk Assessment (MDDRA) model, which can be adopted in a context-aware driving distraction scenario. Finally, a 3-way hybrid CNN-DBN-LSTM multiclass degree of driver distraction according to severity level is developed. In addition, a Hidden Markov Driver Distraction Severity Model (HMDDSM) for the transitioning of control from the driver to the vehicle when a high degree of distraction is detected. This work tests and evaluates the proposed models using the multi-view TeleFOT naturalistic driving study data and the American University of Cairo dataset (AUCD). The evaluation of the developed models was performed using cross-correlation, hybrid cross-correlations, K-Folds validation. The results show that the technique effectively learns and adopts safety measures related to the severity of driver distraction. In addition, the results also show that while a driver is in a dangerous distraction state, the control can be shifted from driver to vehicle in a systematic manner

Contributions to road safety: from abstractions and control theory to real solutions, discussion and evaluation

This manuscript aims to describe my career in the transportation domain, putting in evidence my contributions in different levels, as for example thesis advising, teaching, research animation and coordination, projects construction and participation in expert committees, among others, besides my scientific research itself. The goal, besides the HDR diploma itself, is to show very clearly, including to myself, this 'pack' of contributions in order to look for better contributions to the transportation and control communities or to other communities in the future, and also which research directions I will define to work on in the following. I obtained my PhD degree in the Laboratoire des Signaux et Systèmes - L2S 1 in collaboration with MIT, in 2001, having worked in a purely theoretical automatic control topic scarcely known in the literature - the adaptive control of systems with nonlinear parameterization problem. Arriving in 2002 as a permanent researcher to the former LCPC (Laboratoire Central des Ponts et haussées), now called IFSTTAR (Institut Français des Sciences et Technologies des Transports, de l'Aménagement et des Réseaux), I have been faced to real problems to solve in practice, and faced to the new community of transportation, with a completely different philosophy of work. I have nowadays this double vision - of the very applied transportation domain with concrete problems to be solved that touch the citizen every day, and the vision of a very rich high-level theoretical research in automatic control with powerful tools to solve the real problems, or on the other hand, with control problems that appear because of the need for new tools to solve the real problems. I consider this as an important characteristic for my future contributions. Besides the knowledge in Transportation itself, my eleven years of career in IFSTTAR gave me as well the following new features : 1. From the individual research, I have learned also how to coordinate work (in projects for example, as in the PReVAL sub-project of the European PReVENT project, in which I co-leaded one workpackage, or for research teams, as the control team of LIVIC, coordinated by myself from 2006 to 2009). I have also learned how to animate research (by coordinating research working groups or organizing scientific events and workshops - see for example the working group RSEI and the related scientific event below that I have organized in June 2012) and how to advise students. 2. Besides the double vision I have described above, the experience gave me also the acquisition of a quite multidisciplinary view of the problems in the domain. Firstly, arriving in LIVIC, in the frame of the French consortium ARCOS, I have worked for two years in close cooperation with experts in cognitive sciences (the PsyCoTech group from IRCCyN, Nantes) on designing driving assistance systems to a human driver. After this work, I have continued the collaboration with experts in human sciences within the PReVAL subproject of PReVENT on driving assistance systems evaluation and within the French ANR PARTAGE project, that I have constructed together with the PsyCoTec team of IRCCyN and leaded the IFSTTAR partner for one year. In a dition, through my participation in PReVENT at dirent levels (in two meetings of the Core Group, in PReVAL by co-leading the workpackage 3 on Technical Evaluation of ADAS - ADAS is the shortcut for Advanced Driving Assistance Systems - and in the SAFELANE subproject), I have learned many different aspects of ITS systems. I consider this as an add-on value for my 'pack of knowledge'. 3. What I call "from abstractions to real problems : coming back and forth to solve these real problems" has been matured in my mind, and I am very grateful to my students, with whom I have learned and that helped me in this maturing process. By this sentence, I mean, with a problem to solve in hands, and after building an abstraction, or a simplified view of the problem, and the design of a solution, how to apply it, and to come back again to the theory to change it and to come back to the practice, and so on. This is exactly one of the pillars of the NoE HYCON2, for making interact the theory with the application domains. 4. Considering a problem inserted into the societal context, or inserted within its related context, has been another maturing for myself that I consider very important, notably in the transportation domain, that represents a very complex context containing many different parameters, scenarios and objectives and in addition all the uncertainties linked to the human behavior. I think that it is very important to have a very large view of the context in which the specific problem we are treating is placed. Without this, one cannot say in most of the cases, from my point of view, that the problem is solved. This point will be discussed in Chapter 9.5. 5. Another point that I consider important and where I have been contributing recently is the road mapping work. The acquisition of the multidisciplinary knowledge and a larger view of the domain that I have mentioned in the preceding items, together with my theoretical knowledge in automatic control, allowed myself to start contributing to theroad mapping work in Transportation (through my participation in the imobility forum, in HYCON2 and the in the support action T-Area-SoS on Systems of Systems - all these actions to give advice to the European Commission on the priority areas to be considered in the new Calls, notably in the frame of the H2020 program). I had also the pleasure of opening again books and thesis that I had studied in my PhD work, this time now for advising students in the frame of other very different problems. The very beautiful thesis of Mikael Johansson, Lund University, on piecewise linear systems stability theory is an example. My previous study on switched systems, and the implication of switched Lyapunov functions on stability helped me also in advising my students (Post-Docs, PhD, and M.Sc. students), this time for real applications, with very interesting results blooming up from their work. I realize also that the experience that I have described in the five items above must be put in favor of students since this kind of knowledge cannot be found in the books. Concluding, in these last eleven years, from 2002 to 2013, I could bring to the scientic community and to my students a set of contributions of different kinds. I will try to make clear these contributions for the reader in the next two chapters (written in English and in French). This document is organized in the following way : Part II contains my complete curriculum vitae (in french) where all these contributions will be described in detail. Part III contains then the scientific contributions of the manuscript. What I aim in this chapter is to describe, but further, to analyze them with a distanced look and providing a critical view, announcing perspectives, and placing and discussing the obtained results in the societal context. This is in straight relation with item 4 above. Also, I prefer to adopt, as far as possible, a form comprehensible to the non-automatic control expert, with, as far as possible as well, qualitative explanations and then appropriated references containing the theorems and the definitions corresponding to the qualitative explanations will be provided. In the case it is necessary, they are provided within the text. The Part III is structured in the following chapters. Chapter 8 contains an overview of the global transportation scenario with the associated challenges and a description of the driving assistance systems context. Chapter 9 contains my scientific contributions. These include my research results, my contributions in students advising, in the coordination of research groups, and the collaborative works. It is structured in 3 sections : Section 9.1 introduces what will be the greed for a part of the main contributions, that are described in Sections 9.2 and 9.3. Section 9.1 is also dedicated to showing to the reader how theory and abstractions can be very important for solving real problems. Chapter 9.4 describes other contributions that are the result of collaborative works. A discussion from a multidisciplinary view is provided in Chapter 9.5 based on a survey paper of myself. Chapter 10 will be finally dedicated to the perspectives and the general conclusions. Then last Part contains as annexes a selection of the publications that I consider the most illustrative of my contributions described in Chapter 9. Finally, since the described work is in the intersection of two communities - the transportation and the control theory communities - I decided to write a part of the document dedicated to the non control experts readers. This is Part VI of the document whose aim is to provide some fundamental notions on control theory in a very simple qualitative description whose understanding will help the different readers to understand the contributions

Predicting Safety Benefits of Automated Emergency Braking at Intersections - Virtual simulations based on real-world accident data

Introduction: Intersections are a global traffic safety concern. In the United States, around half of all fatal road traffic accidents take place at intersections or were related to them. In the European Union, about one fifth of road traffic fatalities occur at intersections.Intersection Automated Emergency Braking (AEB) seems to be a promising technology with which to address intersection accidents, as information retrieval by on-board sensing is operational on its own, and, in critical situations, braking is initiated independent of driver reaction. This is not the case for Vehicle-to-Everything (V2X) communication, which requires all conflict-involved vehicles to be equipped with this technology and drivers to respond to an initiated warning. The objective of this thesis is to evaluate the effectiveness of a theoretical Intersection AEB system in avoiding accidents and mitigating injuries. As it will take several decades for a new safety technology to penetrate the vehicle fleet and full coverage of all vehicles may never be achieved, the technology benefit is here analyzed as a function of market penetration. Finally, this research assesses whether a set of test scenarios can be derived without compromising the variance of real-world accidents.Methods: Data from the United States National Automotive Sampling System / General Estimates System and the Fatality Analysis Reporting System was used to compare the capacity of on-board sensing and V2X communication to save lives. To investigate Intersection AEB in detail, the German In-Depth Accident Study (GIDAS) data and the related Pre-Crash Matrix (PCM) were utilized to re-simulate accidents with and without Intersection AEB using different parameter settings of technical aspects and driver comfort boundaries. Machine learning techniques were used to identify opportunities for data clustering.Result: On-board sensing has a substantially higher capability to save lives than V2X communication during the period before full market penetration of both is reached. The analysis of GIDAS and PCM data indicate that about two thirds of left-turn across path accidents with oncoming traffic (LTAP/OD) and about 80 percent of straight crossing path (SCP) accidents can be avoid by an idealized Intersection AEB. Moderate to fatal injuries could be avoided to an even higher extent. Key parameters impacting effectiveness are vehicle speed and potential path choice; to increase effectiveness, these should be limited and narrowed down, respectively.Conclusion and Limitations: Intersection AEB is effective in reducing LTAP/OD and SCP accidents and mitigating injuries However, intersection accidents are highly diverse and accurate performance evaluation requires taking variations into account. The simulations were conducted using ideal sensing without processing delays and an ideal coefficient of friction estimation

Proceedings of the Seventeenth Annual Conference on Manual Control

Manual control is considered, with concentration on perceptive/cognitive man-machine interaction and interface

2nd Symposium on Management of Future motorway and urban Traffic Systems (MFTS 2018): Booklet of abstracts: Ispra, 11-12 June 2018

The Symposium focuses on future traffic management systems, covering the subjects of traffic control, estimation, and modelling of motorway and urban networks, with particular emphasis on the presence of advanced vehicle communication and automation technologies. As connectivity and automation are being progressively introduced in our transport and mobility systems, there is indeed a growing need to understand the implications and opportunities for an enhanced traffic management as well as to identify innovative ways and tools to optimise traffic efficiency. In particular the debate on centralised versus decentralised traffic management in the presence of connected and automated vehicles has started attracting the attention of the research community. In this context, the Symposium provides a remarkable opportunity to share novel ideas and discuss future research directions.JRC.C.4-Sustainable Transpor

Operating cycle representations for road vehicles

This thesis discusses different ways to represent road transport operations mathematically. The intention is to make more realistic predictions of longitudinal performance measures for road vehicles, such as the CO2 emissions. It is argued that a driver and vehicle independent description of relevant transport operations increase the chance that a predicted measure later coincides with the actual measure from the vehicle in its real-world application. This allows for fair comparisons between vehicle designs and, by extension, effective product development. Three different levels of representation are introduced, each with its own purpose and application. The first representation, called the bird\u27s eye view, is a broad, high-level description with few details. It can be used to give a rough picture of the collection of all transport operations that a vehicle executes during its lifetime. It is primarily useful as a classification system to compare different applications and assess their similarity. The second representation, called the stochastic operating cycle (sOC) format, is a statistical, mid-level description with a moderate amount of detail. It can be used to give a comprehensive statistical picture of transport operations, either individually or as a collection. It is primarily useful to measure and reproduce variation in operating conditions, as it describes the physical properties of the road as stochastic processes subject to a hierarchical structure.The third representation, called the deterministic operating cycle (dOC) format, is a physical, low-level description with a great amount of detail. It describes individual operations and contains information about the road, the weather, the traffic and the mission. It is primarily useful as input to dynamic simulations of longitudinal vehicle dynamics.Furthermore, it is discussed how to build a modular, dynamic simulation model that can use data from the dOC format to predict energy usage. At the top level, the complete model has individual modules for the operating cycle, the driver and the vehicle. These share information only through the same interfaces as in reality but have no components in common otherwise and can therefore be modelled separately. Implementations are briefly presented for each module, after which the complete model is showcased in a numerical example.The thesis ends with a discussion, some conclusions, and an outlook on possible ways to continue