Sim-to-real transfer and reality gap modeling in model predictive control for autonomous driving

The main challenge for the adoption of autonomous driving is to ensure an adequate level of safety. Considering the almost infinite variability of possible scenarios that autonomous vehicles would have to face, the use of autonomous driving simulators is becoming of utmost importance. Simulation suites allow the used of automated validation techniques in a wide variety of scenarios, and enable the development of closed-loop validation methods, such as machine learning and reinforcement learning approaches. However, simulation tools suffer from a standing flaw in that there is a noticeable gap between the simulation conditions and real-world scenarios. Although the use of simulators powers most of the research around autonomous driving, and is generally used within all domains it is divided into, there is an inherent source of error given the stochastic nature of activities performed in real world, which are unreplicable in computer environments. This paper proposes a new approach to assess the real-to-sim gap for path tracking systems. The aim is to narrow down the sources of error between simulation results and real-world conditions, and to evaluate the performance of the simulation suite in the design process by employing the information extracted from gap analysis, which adds a new dimension of development against other approaches for autonomous driving. A real-time model predictive controller (MPC) based on adaptive potential fields was developed and validated using the CARLA simulator. Both the path planning and vehicle control systems where tested in real traffic conditions. The error between the simulator and the real data acquisition was evaluated using the Pearson correlation coefficient (PCC) and the max normalized cross-correlation (MNCC). The controller was further evaluated on a process of sim-to-real transfer, and was finally tested both in simulation and real traffic conditions. A comparison was performed against an optimal-control ILQR-based model predictive controller was carried out to further showcase the validity of this approach

Reconfigurable Integrated Vehicle Stability Control Using Optimal Control Techniques

The motivation for the development of vehicle stability control systems comes from the fact that vehicle dynamic behavior in unfavorable driving conditions such as low road-tire adhesion and high speed differs greatly from its nominal behavior. Due to this unexpected behavior, a driver may not be successful in controlling the vehicle in challenging driving situations based only on her/his everyday driving experience. Several noteworthy research works have been conducted on stability control systems over the last two decades to prevent car accidents due to human error. Most of the resultant stability controllers contain individual modules, where each perform a particular task such as yaw tracking, sideslip control, or wheel slip control. These design requirements may contradict each other in some driving scenarios. In such situations, inconsistent control actions can be generated with individual modules. The development of a stability controller that can satisfy diverse and often contradictory requirements is a great challenge. In general, transferring a control structure from one vehicle to another with a different drivetrain layout and actuation system configuration requires remarkable rectifications and repetition of tuning processes from the beginning to achieve a similar performance. This can be considered to be a serious drawback for car manufacturing companies since it results in extra effort, time, and expenses in redesigning and retuning the controller. In this thesis, an integrated controller with a modular structure has been designed to concurrently provide control of the vehicle chassis (yaw rate and sideslip control) and wheel stability (wheel slip ratio control). The proposed control structure incorporates longitudinal and lateral vehicle dynamics to decide on a unified control action. This control action is an outcome of solving an optimization problem that considers all the control objectives in a single cost function, so integrated wheel and vehicle stability is guaranteed. Moreover, according to the particular modular design of the proposed control structure, it can be easily reconfigured to work with different drivetrain layouts such as all-wheel-drive, front-wheel-drive, and rear-wheel-drive, as well as various actuators such as torque vectoring, differential braking, and active steering systems. The high-level control module provides a Center of Gravity (CG) based error analysis and determines the required longitudinal forces and yaw moment adjustments. The low-level control module utilizes this information to allocate control actions optimally at each vehicle corner (wheel) through a single or multi-actuator regime. In order to consider the effect of the actuator dynamics, a mathematical description of the auction system is included in distribution objective function. Therefore, a legitimate control performance is promised in situations requiring shifting from one configuration to another with minimal modifications. The performance of the proposed modular control structure is examined in simulations with a high-fidelity model of an electric GM Equinox vehicle. The high-fidelity model has been developed and provided by GM and the use of the model is to reduce the number of labor-intensive vehicle test and is to test extreme and dangerous driving conditions. Several driving scenarios with severe steering and throttle commands, then, are designed to evaluate the capability of the proposed control structure in integrated longitudinal and lateral vehicle stabilization on slippery road condition. Experimental tests also have been performed with two different electric vehicles for real-time implementation as well as validation purposes. The observations verified the performance qualifications of the proposed control structure to preserve integrated wheel and vehicle chassis stability in all track tests

Integrated Vehicle Stability Control and Power Distribution Using Model Predictive Control

There is a growing need for active safety systems to assist drivers in unfavorable driving conditions. In these conditions, the behavior of the vehicle is different than the linear response during everyday driving. Even experienced drivers usually lose control of the vehicle in such situations and that often results in a car accident. Stability control systems have been developed over the past few decades to assist drivers in keeping the vehicle under control. Most of these control systems are comprised of separate modules, each responsible for one task such as yaw rate tracking, sideslip control, traction control or power distribution. These objectives may be in conflict in some driving situations. In such cases, individual controllers fight over priority and produce conflicting control commands, to the detriment of the vehicle performance. In addition, in most stability control systems, transferring the controller from one vehicle to another with a different driveline and actuator configuration requires significant modifications in the controller and major re-tuning to obtain a similar performance. This is a major disadvantage for auto companies and increases the controller design and tuning costs. In this thesis, an integrated control system has been designed to address vehicle stability, traction control and power distribution objectives at the same time. The proposed controller casts all of these objectives in a single objective function and chooses control actions to optimize this objective function. Therefore, the output of the integrated controller is not altered by another module and the optimality of the solution is not compromised. Furthermore, the designed controller can be easily reconfigured to work with various driveline configurations such as all-wheel drive, front or rear-wheel drive. In addition, it can also work with various actuator configurations such as torque vectoring, differential braking or any combination of them on the front or rear axles. Moving from one configuration to another does not change the stability control performance and major re-tuning can be avoided. The performance of the designed model predictive controller is evaluated in software simulations with a high fidelity model of an electric Equinox vehicle. The stability and wheel slip control performance of the controller is evaluated in various driving and road conditions. In addition, the effect of integrated power distribution is studied. Experimental tests with two different electric vehicles are also carried out to evaluate the real-time performance of the MPC controller. It is observed that the controller is able to maintain vehicle and wheel stability in all of the driving scenarios considered. The power distribution system is able to improve vehicle efficiency by approximately 1.5% and acts in cooperation with the stability control objectives

High-Speed Obstacle Avoidance at the Dynamic Limits for Autonomous Ground Vehicles

Enabling autonomy of passenger-size and larger vehicles is becoming increasingly important in both military and commercial applications. For large autonomous ground vehicles (AGVs), the vehicle dynamics are critical to consider to ensure vehicle safety during obstacle avoidance maneuvers especially at high speeds. This research is concerned with large-size high-speed AGVs with high center of gravity that operate in unstructured environments. The term `unstructured' in this context denotes that there are no lanes or traffic rules to follow. No map of the environment is available a priori. The environment is perceived through a planar light detection and ranging sensor. The mission of the AGV is to move from its initial position to a given target position safely and as fast as possible. In this dissertation, a model predictive control (MPC)-based obstacle avoidance algorithm is developed to achieve the objectives through an iterative simultaneous optimization of the path and the corresponding control commands. MPC is chosen because it offers a rigorous and systematic approach for taking vehicle dynamics and safety constraints into account. Firstly, this thesis investigates the level of model fidelity needed for an MPC-based obstacle avoidance algorithm to be able to safely and quickly avoid obstacles even when the vehicle is close to its dynamic limits. Five different representations of vehicle dynamics models are considered. It is concluded that the two Degrees-of-Freedom (DoF) representation that accounts for tire nonlinearities and longitudinal load transfer is necessary for the MPC-based obstacle avoidance algorithm to operate the vehicle at its limits within an environment that includes large obstacles. Secondly, existing MPC formulations for passenger vehicles in structured environments do not readily apply to this context. Thus, a novel nonlinear MPC formulation is developed. First, a new cost function formulation is used that aims to find the shortest path to the target position. Second, a region partitioning approach is used in conjunction with a multi-phase optimal control formulation to accommodate the complicated forms of obstacle-free regions from an unstructured environment. Third, the no-wheel-lift-off condition is established offline using a fourteen DoF vehicle dynamics model and is included in the MPC formulation. The formulation can simultaneous optimize both steering angle and reference longitudinal speed commands. Simulation results show that the proposed algorithm is capable of safely exploiting the dynamic limits of the vehicle while navigating the vehicle through sensed obstacles of different size and number. Thirdly, in the algorithm, a model of the vehicle is used explicitly to predict and optimize future actions, but in practice, the model parameter values are not exactly known. It is demonstrated that using nominal parameter values in the algorithm leads to safety issues in about one fourth of the evaluated scenarios with the considered parametric uncertainty distributions. To improve the robustness of the algorithm, a novel double-worst-case formulation is developed. Results from simulations with stratified random scenarios and worst-case scenarios show that the double-worst-case formulation considering both the most likely and less likely worst-case scenarios renders the algorithm robust to all uncertainty realizations tested. The trade-off between the robustness and the task completion performance of the algorithm is also quantified. Finally, in addition to simulation-based validation, preliminary experimental validation is also performed. These results demonstrate that the developed algorithm is promising in terms of its capability of avoiding obstacles. Limitations and potential improvements of the algorithm are discussed.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/135770/1/ljch_1.pd

Integrated Planning and Control for Collision Avoidance Systems

Collision avoidance systems like emergency braking assist systems have demonstrated their effectiveness in increasing the safety of vehicle passengers in various studies. To further increase the effectiveness of collision avoidance systems, the exploitation of the lateral free space by evasive maneuvers is being investigated in this book. This work focuses on methods for integrated trajectory planning and vehicle dynamics control in collision avoidance scenarios by combined evasion and braking

Linear Parameter-Varying Control of Full-Vehicle Vertical Dynamics using Semi-Active Dampers

Semi-aktive Fahrwerke bergen im Vergleich zu passiven großes Potential zur Verbesserung wesentlicher Fahrzeugeigenschaften, wie Fahrkomfort, Straßenhaftung und Fahrverhalten. Die Ausnutzung dieses Potentials verlangt nach geeigneten Regelungsalgorithmen,welche das nichtlineare Eingangssignal-zu-Dämpferkraft Verhalten und die Passivitätsbeschränkung semi-aktiver Dämpfer berücksichtigen. Im Besonderen die Passivitätsbeschränkung impliziert enge, zustandsabhängige Aktuatorkraftbegrenzungen und sollte daher im Regelungsentwurf direkt berücksichtigt werden. Der Entwurf performanter semi-aktiver Fahrwerkregelungen stellt eine große Herausforderung dar, da Störungen aufgrund von Straßenunebenheiten und Lastwechseln unterschiedliche Anforderungen an die Regelung stellen, und zusätzlich in einer Gesamtfahrzeuganwendung auch ein Regelungsentwurf basierend auf einem Gesamtfahrzeugmodell benötigt wird. Im Gegensatz zu konventionellen viertelfahrzeug-basierten Fahrwerkregelungsansätzen, welche häufig in der Literatur zu finden sind, zielt der Gesamtfahrzeugregelungsansatz dieser Dissertation auf die explizite Berücksichtigung der Hub-, Wank und Nickbewegung des Aufbaus. Darüber hinaus ermöglicht der Gesamtfahrzeugansatz die Entwicklung von fehlertoleranten Reglern, welche die schwache Aktuatorredundanz der vier Dämpfer nutzen. Die vorliegende Dissertation befasst sich mit linear parameter-variablen (LPV) Regelungsmethoden zur Lösung des oben beschriebenen komplexen Regelungsproblems. Die Kraftbegrenzungen der semi-aktiven Dämpfer werden mittels Sättigungsindikatoren modelliert und diese dann als variable Parameter in den LPV Regelungsentwurf integriert. Zusätzlich wird der LPV Regler um eine Dämpferkraftrekonfiguration erweitert, so dass der Regler den Dämpferkraftverlust im Falle einer Dämpferfehlfunktion mit den verbleibenden gesunden Dämpfern kompensiert. Der Regelungsentwurf begegnet den unterschiedlichen Anforderungen von Straßen- und Lastwechselstörungen durch eine Zweifreiheitsgradregelung bestehend aus einem LPV Regler und einer LPV Vorsteuerung.Dabei fokussiert sich der LPV Regler auf die Verminderung des Effekts der Straßenunebenheiten und die LPV Vorsteuerung verringert den Effekt der Lastwechselstörungen. Auf diese Weise zeigt die Zweifreiheitsgradregelung das gewünschte Verhalten trotz dieser beiden konträren Störungen. Die Wirksamkeit der vorgeschlagenen Zweifreiheitsgradregelung wird durch Experimente auf einem Stempelprüfstand und durch Straßenversuche validiert. Die Ergebnisse zeigen eine Verbesserung des klassischen Zielkonflikts der Fahrwerksregelung zwischen Fahrkomfort und Straßenhaftung durch die LPV Gesamtfahrzeugregelung. Insbesondere erzielt die LPV Gesamtfahrzeugregelung eine 10 % ige Verbesserung von Fahrkomfort und Straßenhaftung im Vergleich zu einer Skyhook-Groundhook Gesamtfahrzeugregelung. Des Weiteren verdeutlicht ein Experiment mit einem simulierten Dämpferfehler die Vorteile der fehlertoleranten LPV Regelung. Abschließend wird anhand von Spurwechselversuchen die Wirksamkeit der LPV Vorsteuerung zur Verbesserung von Fahrkomfort, Straßenhaftung und Fahrverhalten bei dynamischen Lenkwinkeleingaben des Fahrers demonstriert

Dynamic Analysis and Obstacle Avoidance of Autonomous Tractor Semi-Trailers

This thesis fills the research gap of the tractor semi-trailer left turn problem at a city intersection. Although obstacle avoidance is a classic topic in the field of autonomous driving, however, most research is focused on passenger cars or single body vehicles. For an autonomous driving tractor semi-trailer, obstacle avoidance is an essential function. This thesis develops an obstacle avoidance algorithm for tractor semi-trailers

Real-time Trajectory Planning to Enable Safe and Performant Automated Vehicles Operating in Unknown Dynamic Environments

Need for increased automated vehicle safety and performance will exist until control systems can fully exploit the vehicle's maneuvering capacity to avoid collisions with both static and moving obstacles in unknown environments. A safe and performance-based trajectory planning algorithm exists that can operate an automated vehicle in unknown static environments. However, this algorithm cannot be used safely in unknown dynamic environments; furthermore, it is not real-time. Accordingly, this thesis addresses two overarching research questions: * How should a trajectory planning algorithm be formulated to enable automated ground vehicle safety and performance in unknown dynamic environments? * How can such an algorithm be solved in real-time? Safe trajectory planning for high-performance automated vehicles with both static and moving obstacles is a challenging problem. Part of the challenge is developing a formulation that can be solved in real-time while including the following set of specifications: minimum time to goal, a dynamic vehicle model, minimum control effort, both static and moving obstacle avoidance, simultaneous optimization of speed and steering, and a short execution horizon. This thesis presents a nonlinear model predictive control-based trajectory planning formulation, tailored for a high mobility multipurpose wheeled vehicle (HMMWV), that includes the above set of specifications. This formulation is tested then with various sets of these specifications in a known dynamic environment. In particular, a parametric study relating execution horizon and obstacle speed reveals that the moving obstacle avoidance specification is not needed for safety when the planner has a short execution horizon (< 0.375 s), and the obstacles are slow (< 2.11 m/s). However, a moving obstacle avoidance specification is needed when the obstacles move faster, and this specification improves safety without, in most cases, increasing solve-times. Overall, results indicate that trajectory planners for high-performance automated vehicles should include the entire set of specifications mentioned above unless a static or low-speed environment permits a less comprehensive planner. Then, this thesis combines this comprehensive planning algorithm with a suitable perception algorithm to enable safe and performant control of automated ground vehicles in unknown dynamic environments. A high-fidelity, ROS-based proving ground with a 2D LiDAR model, in Gazebo, and a 145 degree of freedom model of the HMMWV, in Chrono, is developed to combine these algorithms. Six-hundred tests, realized with various obstacle speeds and sizes, are performed in this proving ground in both known and unknown dynamic environments. Results from this comparison demonstrate that operating in an unknown environment, as opposed to a known environment, significantly increases collisions, steering effort, throttle effort, braking effort, orientation and tracking error, time to goal, and planner solve times. To avoid this deterioration of safety and performance factors in unknown environments, the use of more accurate perception systems should be explored. Ultimately, however, these results demonstrate that the comprehensive trajectory planning formulation developed in this thesis enables safe and performant control of automated vehicles in unknown dynamic environments among small (< 2 m) obstacles traveling at speeds up to high (20 m/s). To solve this formulation in real-time, an open-source, direct-collocation-based optimal control problem modeling language, called NLOptControl, is established in this thesis. Results demonstrate that NLOptControl can solve the formulation in real-time in both known and unknown environments. NLOptControl holds great potential for not only improving existing off-line and on-line control systems but also engendering a wide variety of new ones.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/149859/1/febbo_1.pd

Benelux meeting on systems and control, 23rd, March 17-19, 2004, Helvoirt, The Netherlands

Book of abstract