Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

The State-of-the-art of Coordinated Ramp Control with Mixed Traffic Conditions

Ramp metering, a traditional traffic control strategy for conventional vehicles, has been widely deployed around the world since the 1960s. On the other hand, the last decade has witnessed significant advances in connected and automated vehicle (CAV) technology and its great potential for improving safety, mobility and environmental sustainability. Therefore, a large amount of research has been conducted on cooperative ramp merging for CAVs only. However, it is expected that the phase of mixed traffic, namely the coexistence of both human-driven vehicles and CAVs, would last for a long time. Since there is little research on the system-wide ramp control with mixed traffic conditions, the paper aims to close this gap by proposing an innovative system architecture and reviewing the state-of-the-art studies on the key components of the proposed system. These components include traffic state estimation, ramp metering, driving behavior modeling, and coordination of CAVs. All reviewed literature plot an extensive landscape for the proposed system-wide coordinated ramp control with mixed traffic conditions.Comment: 8 pages, 1 figure, IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE - ITSC 201

Vision-Based Lane-Changing Behavior Detection Using Deep Residual Neural Network

Accurate lane localization and lane change detection are crucial in advanced driver assistance systems and autonomous driving systems for safer and more efficient trajectory planning. Conventional localization devices such as Global Positioning System only provide road-level resolution for car navigation, which is incompetent to assist in lane-level decision making. The state of art technique for lane localization is to use Light Detection and Ranging sensors to correct the global localization error and achieve centimeter-level accuracy, but the real-time implementation and popularization for LiDAR is still limited by its computational burden and current cost. As a cost-effective alternative, vision-based lane change detection has been highly regarded for affordable autonomous vehicles to support lane-level localization. A deep learning-based computer vision system is developed to detect the lane change behavior using the images captured by a front-view camera mounted on the vehicle and data from the inertial measurement unit for highway driving. Testing results on real-world driving data have shown that the proposed method is robust with real-time working ability and could achieve around 87% lane change detection accuracy. Compared to the average human reaction to visual stimuli, the proposed computer vision system works 9 times faster, which makes it capable of helping make life-saving decisions in time

A study on wear evaluation of railway wheels based on multibody dynamics and wear computation

The wear evolution of railway wheels is a very important issue in railway engineering. In the past, the reprofiling intervals of railway vehicle steel wheels have been scheduled according to designers' experience. Today, more reliable and accurate tools in predicting wheel wear evolution and wheelset lifetime can be used in order to achieve economical and safety benefits. In this work, a computational tool that is able to predict the evolution of the wheel profiles for a given railway system, as a function of the distance run, is presented. The strategy adopted consists of using a commercial multibody software to study the railway dynamic problem and a purpose-built code for managing its pre- and post-processing data in order to compute the wear. The tool is applied here to realistic operation scenarios in order to assess the effect of some service conditions on the wheel wear progression

A multisensing setup for the intelligent tire monitoring

The present paper offers the chance to experimentally measure, for the first time, the internal tire strain by optical fiber sensors during the tire rolling in real operating conditions. The phenomena that take place during the tire rolling are in fact far from being completely understood. Despite several models available in the technical literature, there is not a correspondently large set of experimental observations. The paper includes the detailed description of the new multi-sensing technology for an ongoing vehicle measurement, which the research group has developed in the context of the project OPTYRE. The experimental apparatus is mainly based on the use of optical fibers with embedded Fiber Bragg Gratings sensors for the acquisition of the circumferential tire strain. Other sensors are also installed on the tire, such as a phonic wheel, a uniaxial accelerometer, and a dynamic temperature sensor. The acquired information is used as input variables in dedicated algorithms that allow the identification of key parameters, such as the dynamic contact patch, instantaneous dissipation and instantaneous grip. The OPTYRE project brings a contribution into the field of experimental grip monitoring of wheeled vehicles, with implications both on passive and active safety characteristics of cars and motorbikes

A verification library for multibody simulation software

A multibody dynamics verification library, that maintains and manages test and validation data is proposed, based on RRC Robot arm and CASE backhoe validation and a comparitive study of DADS, DISCOS, and CONTOPS that are existing public domain and commercial multibody dynamic simulation programs. Using simple representative problems, simulation results from each program are cross checked, and the validation results are presented. Functionalities of the verification library are defined, in order to automate validation procedure

The dynamic control of robotic manipulators in space

Described briefly is the work done during the first half year of a three-year study on dynamic control of robotic manipulators in space. The research focused on issues for advanced control of space manipulators including practical issues and new applications for the Virtual Manipulator. In addition, the development of simulations and graphics software for space manipulators, begun during the first NASA proposal in the area, has continued. The fabrication of the Vehicle Emulator System (VES) is completed and control algorithms are in process of development

Development of Eco-Friendly Ramp Control for Connected and Automated Electric Vehicles

With on-board sensors such as camera, radar, and Lidar, connected and automated vehicles (CAVs) can sense the surrounding environment and be driven autonomously and safely by themselves without colliding into other objects on the road. CAVs are also able to communicate with each other and roadside infrastructure via vehicle-to-vehicle and vehicle-to-infrastructure communications, respectively, sharing information on the vehicles’ states, signal phase and timing (SPaT) information, enabling CAVs to make decisions in a collaborative manner. As a typical scenario, ramp control attracts wide attention due to the concerns of safety and mobility in the merging area. In particular, if the line-of-the-sight is blocked (because of grade separation), then neither mainline vehicles nor on-ramp vehicles may well adapt their own dynamics to perform smoothed merging maneuvers. This may lead to speed fluctuations or even shockwave propagating upstream traffic along the corridor, thus potentially increasing the traffic delays and excessive energy consumption. In this project, the research team proposed a hierarchical ramp merging system that not only allowed microscopic cooperative maneuvers for connected and automated electric vehicles on the ramp to merge into mainline traffic flow, but also had controllability of ramp inflow rate, which enabled macroscopic traffic flow control. A centralized optimal control-based approach was proposed to both smooth the merging flow and improve the system-wide mobility of the network. Linear quadratic trackers in both finite horizon and receding horizon forms were developed to solve the optimization problem in terms of path planning and sequence determination, and a microscopic electric vehicle (EV) energy consumption model was applied to estimate the energy consumption. The simulation results confirmed that under the regulated inflow rate, the proposed system was able to avoid potential traffic congestion and improve the mobility (in terms of average speed) as much as 115%, compared to the conventional ramp metering and the ramp without any control approach. Interestingly, for EVs (connected and automated EVs in this study), the improved mobility may not necessarily result in the reduction of energy consumption. The “sweet spot” of average speed ranges from 27–34 mph for the EV models in this study.View the NCST Project Webpag

Robust Distributed Control Protocols for Large Vehicular Platoons with Prescribed Transient and Steady State Performance

In this paper, we study the longitudinal control problem for a platoon of vehicles with unknown nonlinear dynamics under both the predecessor-following and the bidirectional control architectures. The proposed control protocols are fully distributed in the sense that each vehicle utilizes feedback from its relative position with respect to its preceding and following vehicles as well as its own velocity, which can all be easily obtained by onboard sensors. Moreover, no previous knowledge of model nonlinearities/disturbances is incorporated in the control design, enhancing in that way the robustness of the overall closed loop system against model imperfections. Additionally, certain designer-specified performance functions determine the transient and steady-state response, thus preventing connectivity breaks due to sensor limitations as well as inter-vehicular collisions. Finally, extensive simulation studies and a real-time experiment conducted with mobile robots clarify the proposed control protocols and verify their effectiveness.Comment: IEEE Transactions on Control Systems Technology, accepte