A Deep-Learning Framework to Predict the Dynamics of a Human-Driven Vehicle Based on the Road Geometry

Many trajectory forecasting methods, implementing deterministic and stochastic models, have been presented in the last decade for automotive applications. In this work, a deep-learning framework is proposed to model and predict the evolution of the coupled driver-vehicle system dynamics. Particularly, we aim to describe how the road geometry affects the actions performed by the driver. Differently from other works, the problem is formulated in such a way that the user may specify the features of interest. Nonetheless, we propose a set of features that is commonly used for automotive control applications to practically show the functioning of the algorithm. To solve the prediction problem, a deep recurrent neural network based on Long Short-Term Memory autoencoders is designed. It fuses the information on the road geometry and the past driver-vehicle system dynamics to produce context-aware predictions. Also, the complexity of the neural network is constrained to favour its use in online control tasks. The efficacy of the proposed approach was verified in a case study centered on motion cueing algorithms, using a dataset collected during test sessions of a non-professional driver on a dynamic driving simulator. A 3D track with complex geometry was employed as driving environment to render the prediction task challenging. Finally, the robustness of the neural network to changes in the driver and track was investigated to set guidelines for future works.Comment: 10 pages, 9 figures, 3 tables. This work has been submitted to the IEEE Transactions on Intelligent Transportation Systems for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Future Integrated Systems Concept for Preventing Aircraft Loss-of-Control Accidents

Loss of control remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft loss-of-control accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or (more often) in combination. Hence, there is no single intervention strategy to prevent these accidents. This paper presents future system concepts and research directions for preventing aircraft loss-of-control accidents

Transfer of Training on the Vertical Motion Simulator

This paper describes a transfer-of-training study in the NASA Ames Vertical Motion Simulator (VMS). Sixty-one general aviation pilots, divided in four groups, trained on four challenging commercial transport tasks with four motion conditions: no motion, small hexapod, large hexapod, and VMS motion. Then, every pilot repeated the tasks with VMS motion to determine if training with different motion conditions had an effect. New objective motion criteria guided the selection of the motion parameters for the small and large hexapod conditions. Considering results that were statistically significant, or marginally, the motion condition used in training affected 1) longitudinal and lateral touchdown location; 2) the number of secondary stall warnings in a stall recovery; 3) pilot ratings of motion utility and maximum load factor obtained in an overbanked upset recovery; and 4) pilot ratings of motion utility and pedal input reaction time in the engine-out-on-takeoff task. Since the motion condition revealed statistical differences on objective measures in all the tasks, even with some in the direction not predicted, trainers should be cautious not to oversimplify the effects of platform motion. Evidence suggests that the new objective motion criteria may offer valid standardization benefits, as instances arose when the higher-fidelity hexapod motion, as predicted by the criteria, provided better cues in training than the lower-fidelity hexapod motion

Algorithms and Applications for Nonlinear Model Predictive Control with Long Prediction Horizon

Fast implementations of NMPC are important when addressing real-time control of systems exhibiting features like fast dynamics, large dimension, and long prediction horizon, as in such situations the computational burden of the NMPC may limit the achievable control bandwidth. For that purpose, this thesis addresses both algorithms and applications. First, fast NMPC algorithms for controlling continuous-time dynamic systems using a long prediction horizon have been developed. A bridge between linear and nonlinear MPC is built using partial linearizations or sensitivity update. In order to update the sensitivities only when necessary, a Curvature-like measure of nonlinearity (CMoN) for dynamic systems has been introduced and applied to existing NMPC algorithms. Based on CMoN, intuitive and advanced updating logic have been developed for different numerical and control performance. Thus, the CMoN, together with the updating logic, formulates a partial sensitivity updating scheme for fast NMPC, named CMoN-RTI. Simulation examples are used to demonstrate the effectiveness and efficiency of CMoN-RTI. In addition, a rigorous analysis on the optimality and local convergence of CMoN-RTI is given and illustrated using numerical examples. Partial condensing algorithms have been developed when using the proposed partial sensitivity update scheme. The computational complexity has been reduced since part of the condensing information are exploited from previous sampling instants. A sensitivity updating logic together with partial condensing is proposed with a complexity linear in prediction length, leading to a speed up by a factor of ten. Partial matrix factorization algorithms are also proposed to exploit partial sensitivity update. By applying splitting methods to multi-stage problems, only part of the resulting KKT system need to be updated, which is computationally dominant in on-line optimization. Significant improvement has been proved by giving floating point operations (flops). Second, efficient implementations of NMPC have been achieved by developing a Matlab based package named MATMPC. MATMPC has two working modes: the one completely relies on Matlab and the other employs the MATLAB C language API. The advantages of MATMPC are that algorithms are easy to develop and debug thanks to Matlab, and libraries and toolboxes from Matlab can be directly used. When working in the second mode, the computational efficiency of MATMPC is comparable with those software using optimized code generation. Real-time implementations are achieved for a nine degree of freedom dynamic driving simulator and for multi-sensory motion cueing with active seat

Motion Generation and Planning System for a Virtual Reality Motion Simulator: Development, Integration, and Analysis

In the past five years, the advent of virtual reality devices has significantly influenced research in the field of immersion in a virtual world. In addition to the visual input, the motion cues play a vital role in the sense of presence and the factor of engagement in a virtual environment. This thesis aims to develop a motion generation and planning system for the SP7 motion simulator. SP7 is a parallel robotic manipulator in a 6RSS-R configuration. The motion generation system must be able to produce accurate motion data that matches the visual and audio signals. In this research, two different system workflows have been developed, the first for creating custom visual, audio, and motion cues, while the second for extracting the required motion data from an existing game or simulation. Motion data from the motion generation system are not bounded, while motion simulator movements are limited. The motion planning system commonly known as the motion cueing algorithm is used to create an effective illusion within the limited capabilities of the motion platform. Appropriate and effective motion cues could be achieved by a proper understanding of the perception of human motion, in particular the functioning of the vestibular system. A classical motion cueing has been developed using the model of the semi-circular canal and otoliths. A procedural implementation of the motion cueing algorithm has been described in this thesis. We have integrated all components together to make this robotic mechanism into a VR motion simulator. In general, the performance of the motion simulator is measured by the quality of the motion perceived on the platform by the user. As a result, a novel methodology for the systematic subjective evaluation of the SP7 with a pool of juries was developed to check the quality of motion perception. Based on the results of the evaluation, key issues related to the current configuration of the SP7 have been identified. Minor issues were rectified on the flow, so they were not extensively reported in this thesis. Two major issues have been addressed extensively, namely the parameter tuning of the motion cueing algorithm and the motion compensation of the visual signal in virtual reality devices. The first issue was resolved by developing a tuning strategy with an abstraction layer concept derived from the outcome of the novel technique for the objective assessment of the motion cueing algorithm. The origin of the second problem was found to be a calibration problem of the Vive lighthouse tracking system. So, a thorough experimental study was performed to obtain the optimal calibrated environment. This was achieved by benchmarking the dynamic position tracking performance of the Vive lighthouse tracking system using an industrial serial robot as a ground truth system. With the resolution of the identified issues, a general-purpose virtual reality motion simulator has been developed that is capable of creating custom visual, audio, and motion cues and of executing motion planning for a robotic manipulator with a human motion perception constraint

Optimizing The Design Of Multimodal User Interfaces

Due to a current lack of principle-driven multimodal user interface design guidelines, designers may encounter difficulties when choosing the most appropriate display modality for given users or specific tasks (e.g., verbal versus spatial tasks). The development of multimodal display guidelines from both a user and task domain perspective is thus critical to the achievement of successful human-system interaction. Specifically, there is a need to determine how to design task information presentation (e.g., via which modalities) to capitalize on an individual operator\u27s information processing capabilities and the inherent efficiencies associated with redundant sensory information, thereby alleviating information overload. The present effort addresses this issue by proposing a theoretical framework (Architecture for Multi-Modal Optimization, AMMO) from which multimodal display design guidelines and adaptive automation strategies may be derived. The foundation of the proposed framework is based on extending, at a functional working memory (WM) level, existing information processing theories and models with the latest findings in cognitive psychology, neuroscience, and other allied sciences. The utility of AMMO lies in its ability to provide designers with strategies for directing system design, as well as dynamic adaptation strategies (i.e., multimodal mitigation strategies) in support of real-time operations. In an effort to validate specific components of AMMO, a subset of AMMO-derived multimodal design guidelines was evaluated with a simulated weapons control system multitasking environment. The results of this study demonstrated significant performance improvements in user response time and accuracy when multimodal display cues were used (i.e., auditory and tactile, individually and in combination) to augment the visual display of information, thereby distributing human information processing resources across multiple sensory and WM resources. These results provide initial empirical support for validation of the overall AMMO model and a sub-set of the principle-driven multimodal design guidelines derived from it. The empirically-validated multimodal design guidelines may be applicable to a wide range of information-intensive computer-based multitasking environments

Impact of Human-Centered Vestibular System Model for Motion Control in a Driving Simulator

This study presents a driving simulator experiment to evaluate three different motion cueing algorithms based on model predictive control. The difference among these motion strategies lies in the type of mathematical model used. The first one contains only the dynamic model of the platform, while the others integrate additionally two different vestibular system models. We compare these three strategies to discuss the tradeoffs when including a vestibular system model in the control loop from the user's viewpoint. The study is conducted in autonomous mode and in free driving mode, as both play an important role in motion cueing validation. A total of 38 individuals participated in the experiment; 19 drove the simulator in free driving mode and the remaining using the autonomous driving mode. For both driving modes, substantial differences is observed. The analysis shows that one of the vestibular system models is suitable for driving simulators, as it thoroughly restores high-frequency accelerations and is well noted by the participants, especially those in the free driving mode. Further tests are needed to analyze the advantages of integrating the chosen vestibular system model in the control design for motion cuieng algorithms. Regarding the autonomous mode, further research is needed to examine the influence of the vestibular system model on the motion performance, as the behavior of the autonomous model may implicitly interfere with subjective assessments.This work was supported in part by Renault’s group and the ANRT (National Research and Technology Agency)

Model-Based Control Techniques for Automotive Applications

Two different topics are covered in the thesis. Model Predictive Control applied to the Motion Cueing Problem In the last years the interest about dynamic driving simulators is increasing and new commercial solutions are arising. Driving simulators play an important role in the development of new vehicles and advanced driver assistance devices: in fact, on the one hand, having a human driver on a driving simulator allows automotive manufacturers to bridge the gap between virtual prototyping and on-road testing during the vehicle development phase; on the other hand, novel driver assistance systems (such as advanced accident avoidance systems) can be safely tested by having the driver operating the vehicle in a virtual, highly realistic environment, while being exposed to hazardous situations. In both applications, it is crucial to faithfully reproduce in the simulator the driver's perception of forces acting on the vehicle and its acceleration. This has to be achieved while keeping the platform within its limited operation space. Such strategies go under the name of Motion Cueing Algorithms. In this work, a particular implementation of a Motion Cueing algorithm is described, that is based on Model Predictive Control technique. A distinctive feature of such approach is that it exploits a detailed model of the human vestibular system, and consequently differs from standard Motion Cueing strategies based on Washout Filters: such feature allows for better implementation of tilt coordination and more efficient handling of the platform limits. The algorithm has been evaluated in practice on a small-size, innovative platform, by performing tests with professional drivers. Results show that the MPC-based motion cueing algorithm allows to effectively handle the platform working area, to limit the presence of those platform movements that are typically associated with driver motion sickness, and to devise simple and intuitive tuning procedures. Moreover, the availability of an effective virtual driver allows the development of effective predictive strategies, and first simulation results are reported in the thesis. Control Techniques for a Hybrid Sport Motorcycle Reduction of the environmental impact of transportation systems is a world wide priority. Hybrid propulsion vehicles have proved to have a strong potential to this regard, and different four-wheels solutions have spread out in the market. Differently from cars, and even if they are considered the ideal solution for urban mobility, motorbikes and mopeds have not seen a wide application of hybrid propulsion yet, mostly due to the more strict constraints on available space and driving feeling. In the thesis, the problem of providing a commercial 125cc motorbike with a hybrid propulsion system is considered, by adding an electric engine to its standard internal combustion engine. The aim for the prototype is to use the electrical machine (directly keyed on the drive shaft) to obtain a torque boost during accelerations, improving and regularizing the supplied power while reducing the emissions. Two different control algorithms are proposed 1) the first is based on a standard heuristic with adaptive features, simpler to implement on the ECU for the prototype; 2) the second is a torque-split optimal-control strategy, managing the different contributions from the two engines. A crucial point is the implementation of a Simulink virtual environment, realized starting from a commercial tool, VI-BikeRealTime, to test the algorithms. The hybrid engine model has been implemented in the tool from scratch, as well as a simple battery model, derived directly from data-sheet characteristics by using polynomial interpolation. The simulation system is completed by a virtual rider and a tool for build test circuits. Results of the simulations on a realistic track are included, to evaluate the different performance of the two strategies in a closed loop environment (thanks to the virtual rider). The results from on-track tests of the real prototype, using the first control strategy, are reported too

Driving simulator study of the relationship between motion strategy preference and self-reported driving behavior

Faithful motion restitution in driving simulators normally focuses on track monitoring and maximizing the platform workspace by leaving aside the principal component—the driver. Therefore, in this work we investigated the role of the motion perception model on motion cueing algorithms from a user’s viewpoint. We focused on the driving behavior influence regarding motion perception in a driving simulator. Participants drove a driving simulator with two different configurations: (a) using the platform dynamic model and (b) using a supplementary motion perception model. Both strategies were compared and the participants’ data were classified according to the strategy they preferred. To this end, we developed a driving behavior questionnaire aiming at evaluating the self-reported driving behavior influence on participants’ motion cueing preferences. The results showed significant differences between the participants who chose different strategies and the scored driving behavior in the hostile and violations factors. In order to support these findings, we compared participants’ behaviors and actual motion driving simulator indicators such as speed, jerk, and lateral position. The analysis revealed that motion preferences arise from different reasons linked to the realism or smoothness in motion. Also, strong positive correlations were found between hostile and violation behaviors of the group who preferred the strategy with the supplementary motion perception model, and objective measures such as jerk and speed on different road segments. This indicates that motion perception in driving simulators may depend not only on the type of motion cueing strategy, but may also be influenced by users’ self-reported driving behaviors.This work was supported in part by Renault group and a PhD grant from ANRT (National Research and Technology Agency)