Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

Lane change in dense traffic is considered a challenging problem that typically requires the recognition of an opportune and appropriate opportunity for maneuvers. In this work, we propose a chance-aware lane-change strategy with high-level model predictive control (MPC) through curriculum reinforcement learning (CRL). The embodied MPC in our framework is parameterized with augmented decision variables, where full-state references and regulatory factors concerning their relative importance are introduced. Furthermore, to improve the convergence speed and ensure a high-quality policy, effective curriculum design is integrated into the reinforcement learning (RL) framework with policy transfer and enhancement. Then the proposed framework is deployed to numerical simulations towards dense and dynamic traffic. It is noteworthy that, given a narrow chance, the proposed approach generates high-quality lane-change maneuvers such that the vehicle merges into the traffic flow with a high success rate of 96%

Spatiotemporal Receding Horizon Control with Proactive Interaction Towards Autonomous Driving in Dense Traffic

In dense traffic scenarios, ensuring safety while keeping high task performance for autonomous driving is a critical challenge. To address this problem, this paper proposes a computationally-efficient spatiotemporal receding horizon control (ST-RHC) scheme to generate a safe, dynamically feasible, energy-efficient trajectory in control space, where different driving tasks in dense traffic can be achieved with high accuracy and safety in real time. In particular, an embodied spatiotemporal safety barrier module considering proactive interactions is devised to mitigate the effects of inaccuracies resulting from the trajectory prediction of other vehicles. Subsequently, the motion planning and control problem is formulated as a constrained nonlinear optimization problem, which favorably facilitates the effective use of off-the-shelf optimization solvers in conjunction with multiple shooting. The effectiveness of the proposed ST-RHC scheme is demonstrated through comprehensive comparisons with state-of-the-art algorithms on synthetic and real-world traffic datasets under dense traffic, and the attendant outcome of superior performance in terms of accuracy, efficiency and safety is achieved.Comment: 16 pages, 13 figures, accepted for publication in IEEE Transactions on Intelligent Vehicle

Curriculum Proximal Policy Optimization with Stage-Decaying Clipping for Self-Driving at Unsignalized Intersections

Unsignalized intersections are typically considered as one of the most representative and challenging scenarios for self-driving vehicles. To tackle autonomous driving problems in such scenarios, this paper proposes a curriculum proximal policy optimization (CPPO) framework with stage-decaying clipping. By adjusting the clipping parameter during different stages of training through proximal policy optimization (PPO), the vehicle can first rapidly search for an approximate optimal policy or its neighborhood with a large parameter, and then converges to the optimal policy with a small one. Particularly, the stage-based curriculum learning technology is incorporated into the proposed framework to improve the generalization performance and further accelerate the training process. Moreover, the reward function is specially designed in view of different curriculum settings. A series of comparative experiments are conducted in intersection-crossing scenarios with bi-lane carriageways to verify the effectiveness of the proposed CPPO method. The results show that the proposed approach demonstrates better adaptiveness to different dynamic and complex environments, as well as faster training speed over baseline methods.Comment: 7 pages, 4 figure

Learning the References of Online Model Predictive Control for Urban Self-Driving

In this work, we propose a novel learning-based model predictive control (MPC) framework for motion planning and control of urban self-driving. In this framework, instantaneous references and cost functions of online MPC are learned from raw sensor data without relying on any oracle or predicted states of traffic. Moreover, driving safety conditions are latently encoded via the introduction of a learnable instantaneous reference vector. In particular, we implement a deep reinforcement learning (DRL) framework for policy search, where practical and lightweight raw observations are processed to reason about the traffic and provide the online MPC with instantaneous references. The proposed approach is validated in a high-fidelity simulator, where our development manifests remarkable adaptiveness to complex and dynamic traffic. Furthermore, sim-to-real deployments are also conducted to evaluate the generalizability of the proposed framework in various real-world applications. Also, we provide the open-source code and video demonstrations at the project website: https://latent-mpc.github.io/

Real-Time Parallel Trajectory Optimization with Spatiotemporal Safety Constraints for Autonomous Driving in Congested Traffic

Multi-modal behaviors exhibited by surrounding vehicles (SVs) can typically lead to traffic congestion and reduce the travel efficiency of autonomous vehicles (AVs) in dense traffic. This paper proposes a real-time parallel trajectory optimization method for the AV to achieve high travel efficiency in dynamic and congested environments. A spatiotemporal safety module is developed to facilitate the safe interaction between the AV and SVs in the presence of trajectory prediction errors resulting from the multi-modal behaviors of the SVs. By leveraging multiple shooting and constraint transcription, we transform the trajectory optimization problem into a nonlinear programming problem, which allows for the use of optimization solvers and parallel computing techniques to generate multiple feasible trajectories in parallel. Subsequently, these spatiotemporal trajectories are fed into a multi-objective evaluation module considering both safety and efficiency objectives, such that the optimal feasible trajectory corresponding to the optimal target lane can be selected. The proposed framework is validated through simulations in a dense and congested driving scenario with multiple uncertain SVs. The results demonstrate that our method enables the AV to safely navigate through a dense and congested traffic scenario while achieving high travel efficiency and task accuracy in real time.Comment: 8 pages, 7 figures, accepted for publication in the 26th IEEE International Conference on Intelligent Transportation Systems (ITSC 2023

Incremental Bayesian Learning for Fail-Operational Control in Autonomous Driving

Abrupt maneuvers by surrounding vehicles (SVs) can typically lead to safety concerns and affect the task efficiency of the ego vehicle (EV), especially with model uncertainties stemming from environmental disturbances. This paper presents a real-time fail-operational controller that ensures the asymptotic convergence of an uncertain EV to a safe state, while preserving task efficiency in dynamic environments. An incremental Bayesian learning approach is developed to facilitate online learning and inference of changing environmental disturbances. Leveraging disturbance quantification and constraint transformation, we develop a stochastic fail-operational barrier based on the control barrier function (CBF). With this development, the uncertain EV is able to converge asymptotically from an unsafe state to a defined safe state with probabilistic stability. Subsequently, the stochastic fail-operational barrier is integrated into an efficient fail-operational controller based on quadratic programming (QP). This controller is tailored for the EV operating under control constraints in the presence of environmental disturbances, with both safety and efficiency objectives taken into consideration. We validate the proposed framework in connected cruise control (CCC) tasks, where SVs perform aggressive driving maneuvers. The simulation results demonstrate that our method empowers the EV to swiftly return to a safe state while upholding task efficiency in real time, even under time-varying environmental disturbances.Comment: 8 pages, 8 figures, accepted for publication in the 22nd European Control Conference (ECC 2024

Research Progress on the Formation Mechanism of Imidazoline Quinoline Heterocyclic Amines and the Inhibition Mechanism of Exogenous Substances on Them

The imidazoquinoline type (IQ-type) is recognized as the most mutagenic activity in heterocyclic amines currently, and closely associated with a variety of cancers and neurodegenerative diseases of human. The precursors of IQ-type HAs are free amino acids, reducing sugars, and creatine/creatinine, and these precursors can undergo a series of chemical reactions at high temperature, and intermediate products such as free radicals and reactive carbonyl compounds are formed, which are the basis for the formation of IQ-type HAs. The exogenous substances added to the meat products during the thermal processing affected the IQ-type HAs formation to varying degrees. There are three main action mechanisms of the exogenous substances inhibiting IQ-type HAs, including competitive inhibition mechanism of precursors, free radical scavenging mechanisms, and active carbonyl compounds capturing mechanism. This article focuses on the formation process of IQ-type HAs and the inhibitory mechanism of exogenous substances, and clarified the reaction pathway of IQ-type HAs generation, and elaborated the action mechanism of exogenous substances inhibiting the formation of IQ-type HAs in detail. This paper provides theoretical guidance for effectively controlling the IQ-type HAs formation during the thermal processing of meat products

A preliminary study: saltiness and sodium content of aqueous extracts from plants and marine animal shells

To develop a salt substitute with low sodium content, the water-soluble components of seaweed, kelp, clamshell, oyster shell, semen cassiae, cuttlefish bone, inula flower, Arabia cowry shell, and sanna leaf were extracted with water. The aqueous extracts of nine species of plants and marine animal shells were obtained after drying the plants and shells at 105 °C until achieving a constant weight. The hedonic scale test revealed that the clamshell and cuttlefish bone aqueous extracts tasted distinctly salty. The result of the degree of difference test showed that the 1 % clamshell extract solution (m/v) and 0.6 % cuttlefish bone extract solution (m/v) both had equivalent saltiness of 0.6 % NaCl (m/v). In contrast, the sodium content in the cuttlefish bone extract solution was 27 % less than that in a NaCl solution of the same degree of saltiness. Therefore, a novel salt substitute will be developed in future studies in accordance with the principles of bionics and a deep understanding of the salty taste interactions among key salty components in the cuttlefish bone extract