Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

Lane change in dense traffic is considered a challenging problem that typically requires the recognition of an opportune and appropriate opportunity for maneuvers. In this work, we propose a chance-aware lane-change strategy with high-level model predictive control (MPC) through curriculum reinforcement learning (CRL). The embodied MPC in our framework is parameterized with augmented decision variables, where full-state references and regulatory factors concerning their relative importance are introduced. Furthermore, to improve the convergence speed and ensure a high-quality policy, effective curriculum design is integrated into the reinforcement learning (RL) framework with policy transfer and enhancement. Then the proposed framework is deployed to numerical simulations towards dense and dynamic traffic. It is noteworthy that, given a narrow chance, the proposed approach generates high-quality lane-change maneuvers such that the vehicle merges into the traffic flow with a high success rate of 96%

Curriculum Proximal Policy Optimization with Stage-Decaying Clipping for Self-Driving at Unsignalized Intersections

Unsignalized intersections are typically considered as one of the most representative and challenging scenarios for self-driving vehicles. To tackle autonomous driving problems in such scenarios, this paper proposes a curriculum proximal policy optimization (CPPO) framework with stage-decaying clipping. By adjusting the clipping parameter during different stages of training through proximal policy optimization (PPO), the vehicle can first rapidly search for an approximate optimal policy or its neighborhood with a large parameter, and then converges to the optimal policy with a small one. Particularly, the stage-based curriculum learning technology is incorporated into the proposed framework to improve the generalization performance and further accelerate the training process. Moreover, the reward function is specially designed in view of different curriculum settings. A series of comparative experiments are conducted in intersection-crossing scenarios with bi-lane carriageways to verify the effectiveness of the proposed CPPO method. The results show that the proposed approach demonstrates better adaptiveness to different dynamic and complex environments, as well as faster training speed over baseline methods.Comment: 7 pages, 4 figure

Learning the References of Online Model Predictive Control for Urban Self-Driving

In this work, we propose a novel learning-based model predictive control (MPC) framework for motion planning and control of urban self-driving. In this framework, instantaneous references and cost functions of online MPC are learned from raw sensor data without relying on any oracle or predicted states of traffic. Moreover, driving safety conditions are latently encoded via the introduction of a learnable instantaneous reference vector. In particular, we implement a deep reinforcement learning (DRL) framework for policy search, where practical and lightweight raw observations are processed to reason about the traffic and provide the online MPC with instantaneous references. The proposed approach is validated in a high-fidelity simulator, where our development manifests remarkable adaptiveness to complex and dynamic traffic. Furthermore, sim-to-real deployments are also conducted to evaluate the generalizability of the proposed framework in various real-world applications. Also, we provide the open-source code and video demonstrations at the project website: https://latent-mpc.github.io/

Real-Time Parallel Trajectory Optimization with Spatiotemporal Safety Constraints for Autonomous Driving in Congested Traffic

Multi-modal behaviors exhibited by surrounding vehicles (SVs) can typically lead to traffic congestion and reduce the travel efficiency of autonomous vehicles (AVs) in dense traffic. This paper proposes a real-time parallel trajectory optimization method for the AV to achieve high travel efficiency in dynamic and congested environments. A spatiotemporal safety module is developed to facilitate the safe interaction between the AV and SVs in the presence of trajectory prediction errors resulting from the multi-modal behaviors of the SVs. By leveraging multiple shooting and constraint transcription, we transform the trajectory optimization problem into a nonlinear programming problem, which allows for the use of optimization solvers and parallel computing techniques to generate multiple feasible trajectories in parallel. Subsequently, these spatiotemporal trajectories are fed into a multi-objective evaluation module considering both safety and efficiency objectives, such that the optimal feasible trajectory corresponding to the optimal target lane can be selected. The proposed framework is validated through simulations in a dense and congested driving scenario with multiple uncertain SVs. The results demonstrate that our method enables the AV to safely navigate through a dense and congested traffic scenario while achieving high travel efficiency and task accuracy in real time.Comment: 8 pages, 7 figures, accepted for publication in the 26th IEEE International Conference on Intelligent Transportation Systems (ITSC 2023

A preliminary study: saltiness and sodium content of aqueous extracts from plants and marine animal shells

To develop a salt substitute with low sodium content, the water-soluble components of seaweed, kelp, clamshell, oyster shell, semen cassiae, cuttlefish bone, inula flower, Arabia cowry shell, and sanna leaf were extracted with water. The aqueous extracts of nine species of plants and marine animal shells were obtained after drying the plants and shells at 105 °C until achieving a constant weight. The hedonic scale test revealed that the clamshell and cuttlefish bone aqueous extracts tasted distinctly salty. The result of the degree of difference test showed that the 1 % clamshell extract solution (m/v) and 0.6 % cuttlefish bone extract solution (m/v) both had equivalent saltiness of 0.6 % NaCl (m/v). In contrast, the sodium content in the cuttlefish bone extract solution was 27 % less than that in a NaCl solution of the same degree of saltiness. Therefore, a novel salt substitute will be developed in future studies in accordance with the principles of bionics and a deep understanding of the salty taste interactions among key salty components in the cuttlefish bone extract

Soft Rehabilitation and Nursing-Care Robots: A Review and Future Outlook

Rehabilitation and nursing-care robots have become one of the prevalent methods for assistant treatment of motor disorder patients in the field of medical rehabilitation. Traditional rehabilitation robots are mostly made of rigid materials, which significantly limits their application for medical rehabilitation and nursing-care. Soft robots show great potential in the field of rehabilitation robots because of their inherent compliance and safety when they interact with humans. In this paper, we conduct a systematic summary and discussion on the soft rehabilitation and nursing-care robots. This study reviews typical mechanical structures, modeling methods, and control strategies of soft rehabilitation and nursing-care robots in recent years. We classify soft rehabilitation and nursing-care robots into two categories according to their actuation technology, one is based on tendon-driven actuation and the other is based on soft intelligent material actuation. Finally, we analyze and discuss the future directions and work about soft rehabilitation and nursing-care robots, which can provide useful guidance and help on the development of advanced soft rehabilitation and nursing-care robots

Influence of salt substitute containing KCl, L-histidine and L-lysine on the secondary structure and gel properties of myosin

The effect of a salt substitute (SS) containing L-lysine (Lys) and L-histidine (His) on secondary structure and gel properties of myosin from grass carp was examined. The results indicated that the α-helix content of myosin treated with SS was 29.00%, 29.03% and 35.93% more than that with NaCl at 0.4, 0.6 and 0.8 mol/L (P < 0.05), respectively, suggesting that some fractions of the β-sheet, β-turn and random coil were transformed into α-helix. A similar pattern of storage modulus (G’) was found between NaCl and SS treatments, and the G’ of SS treatments at the end of gelation completion was higher than that of NaCl treatments. The salt substitute improved the gel strength and hardness of myosin at 0.4, 0.6 and 0.8 mol/L. The results indicated that the changes in secondary structure and gel properties of myosin may be mainly due to the L-lys and/or L-his in salt substitute

Effects of substitution of NaCl with KCl, L-histidine, and L-lysine on instrumental quality attributes of cured and cooked pork loin

This study evaluated the effect of substituting of NaCl with varying amounts of a combination of KCl, L-histidine (L-his), and L-lysine (L-lys) on the instrumental characteristics of cooked loin. Fifteen cooked loins were produced by replacing 0%, 25%, 50%, 75% and 100% of NaCl with the salt substitute. Experiments were conducted in triplicate to determine the water-holding capacity (WHC), and T2 relaxation time, and a texture profile analysis (TPA) and color determination test were also performed. T2 relaxation time analysis indicated that substitution affected the distribution of water by increasing the proportion of immobilized water and improving the WHC. results of TPA and color tests showed no adverse effect from using the experimental treatments. Results showed that 50% was the most suitable substitution ratio (actual 30.15% subsitution level of NaCl), for which the Na content was approximately 27% lower than that of the control for cooked loin