Air-Combat Strategy Using Approximate Dynamic Programming

Unmanned Aircraft Systems (UAS) have the potential to perform many of the dangerous missions currently own by manned aircraft. Yet, the complexity of some tasks, such as air combat, have precluded UAS from successfully carrying out these missions autonomously. This paper presents a formulation of a level flight, fixed velocity, one-on-one air combat maneuvering problem and an approximate dynamic programming (ADP) approach for computing an efficient approximation of the optimal policy. In the version of the problem formulation considered, the aircraft learning the optimal policy is given a slight performance advantage. This ADP approach provides a fast response to a rapidly changing tactical situation, long planning horizons, and good performance without explicit coding of air combat tactics. The method's success is due to extensive feature development, reward shaping and trajectory sampling. An accompanying fast and e ffective rollout-based policy extraction method is used to accomplish on-line implementation. Simulation results are provided that demonstrate the robustness of the method against an opponent beginning from both off ensive and defensive situations. Flight results are also presented using micro-UAS own at MIT's Real-time indoor Autonomous Vehicle test ENvironment (RAVEN).Defense University Research Instrumentation Program (U.S.) (grant number FA9550-07-1-0321)United States. Air Force Office of Scientific Research (AFOSR # FA9550-08-1-0086)American Society for Engineering Education (National Defense Science and Engineering Graduate Fellowship

A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectories

The problem of maneuvering a vehicle through a race course in minimum time requires computation of both longitudinal (brake and throttle) and lateral (steering wheel) control inputs. Unfortunately, solving the resulting nonlinear optimal control problem is typically computationally expensive and infeasible for real-time trajectory planning. This paper presents an iterative algorithm that divides the path generation task into two sequential subproblems that are significantly easier to solve. Given an initial path through the race track, the algorithm runs a forward-backward integration scheme to determine the minimum-time longitudinal speed profile, subject to tire friction constraints. With this fixed speed profile, the algorithm updates the vehicle's path by solving a convex optimization problem that minimizes the resulting path curvature while staying within track boundaries and obeying affine, time-varying vehicle dynamics constraints. This two-step process is repeated iteratively until the predicted lap time no longer improves. While providing no guarantees of convergence or a globally optimal solution, the approach performs very well when validated on the Thunderhill Raceway course in Willows, CA. The predicted lap time converges after four to five iterations, with each iteration over the full 4.5 km race course requiring only thirty seconds of computation time on a laptop computer. The resulting trajectory is experimentally driven at the race circuit with an autonomous Audi TTS test vehicle, and the resulting lap time and racing line is comparable to both a nonlinear gradient descent solution and a trajectory recorded from a professional racecar driver. The experimental results indicate that the proposed method is a viable option for online trajectory planning in the near future

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Trajectory Planning for Autonomous High-Speed Overtaking in Structured Environments using Robust MPC

Automated vehicles are increasingly getting mainstreamed and this has pushed development of systems for autonomous manoeuvring (e.g., lane-change, merge, overtake, etc.) to the forefront. A novel framework for situational awareness and trajectory planning to perform autonomous overtaking in high-speed structured environments (e.g., highway, motorway) is presented in this paper. A combination of a potential field like function and reachability sets of a vehicle are used to identify safe zones on a road that the vehicle can navigate towards. These safe zones are provided to a tube-based robust model predictive controller as reference to generate feasible trajectories for combined lateral and longitudinal motion of a vehicle. The strengths of the proposed framework are: (i) it is free from nonconvex collision avoidance constraints, (ii) it ensures feasibility of trajectory even if decelerating or accelerating while performing lateral motion, and (iii) it is real-time implementable. The ability of the proposed framework to plan feasible trajectories for highspeed overtaking is validated in a high-fidelity IPG CarMaker and Simulink co-simulation environment

Efficient Automated Driving Strategies Leveraging Anticipation and Optimal Control

Automated vehicles and advanced driver assistance systems bring computation, sensing, and communication technologies that exceed human abilities in some ways. For example, automated vehicles may sense a panorama all at once, do not suffer from human impairments and distractions, and could wirelessly communicate precise data with neighboring vehicles. Prototype and commercial deployments have demonstrated the capability to relieve human operators of some driving tasks up to and including fully autonomous taxi rides in some areas. The ultimate impact of this technology’s large-scale market penetration on energy efficiency remains unclear, with potential negative factors like road use by empty vehicles competing with positive ones like automatic eco-driving. Fundamentally enabled by historic and look-ahead data, this dissertation addresses the use of automated driving and driver assistance to optimize vehicle motion for energy efficiency. Facets of this problem include car following, co-optimized acceleration and lane change planning, and collaborative multi-agent guidance. Optimal control, especially model predictive control, is used extensively to improve energy efficiency while maintaining safe and timely driving via constraints. Techniques including chance constraints and mixed integer programming help overcome uncertainty and non-convexity challenges. Extensions of these techniques to tractor trailers on sloping roads are provided by making use of linear parameter-varying models. To approach the wheel-input energy eco-driving problem over generally shaped sloping roads with the computational potential for closed-loop implementation, a linear programming formulation is constructed. Distributed and collaborative techniques that enable connected and automated vehicles to accommodate their neighbors in traffic are also explored and compared to centralized control. Using simulations and vehicle-in-the-loop car following experiments, the proposed algorithms are benchmarked against others that do not make use of look-ahead information

A Human Driver Model for Autonomous Lane Changing in Highways: Predictive Fuzzy Markov Game Driving Strategy

This study presents an integrated hybrid solution to mandatory lane changing problem to deal with accident avoidance by choosing a safe gap in highway driving. To manage this, a comprehensive treatment to a lane change active safety design is proposed from dynamics, control, and decision making aspects. My effort first goes on driver behaviors and relating human reasoning of threat in driving for modeling a decision making strategy. It consists of two main parts; threat assessment in traffic participants, (TV s) states, and decision making. The first part utilizes an complementary threat assessment of TV s, relative to the subject vehicle, SV , by evaluating the traffic quantities. Then I propose a decision strategy, which is based on Markov decision processes (MDPs) that abstract the traffic environment with a set of actions, transition probabilities, and corresponding utility rewards. Further, the interactions of the TV s are employed to set up a real traffic condition by using game theoretic approach. The question to be addressed here is that how an autonomous vehicle optimally interacts with the surrounding vehicles for a gap selection so that more effective performance of the overall traffic flow can be captured. Finding a safe gap is performed via maximizing an objective function among several candidates. A future prediction engine thus is embedded in the design, which simulates and seeks for a solution such that the objective function is maximized at each time step over a horizon. The combined system therefore forms a predictive fuzzy Markov game (FMG) since it is to perform a predictive interactive driving strategy to avoid accidents for a given traffic environment. I show the effect of interactions in decision making process by proposing both cooperative and non-cooperative Markov game strategies for enhanced traffic safety and mobility. This level is called the higher level controller. I further focus on generating a driver controller to complement the automated car’s safe driving. To compute this, model predictive controller (MPC) is utilized. The success of the combined decision process and trajectory generation is evaluated with a set of different traffic scenarios in dSPACE virtual driving environment. Next, I consider designing an active front steering (AFS) and direct yaw moment control (DYC) as the lower level controller that performs a lane change task with enhanced handling performance in the presence of varying front and rear cornering stiffnesses. I propose a new control scheme that integrates active front steering and the direct yaw moment control to enhance the vehicle handling and stability. I obtain the nonlinear tire forces with Pacejka model, and convert the nonlinear tire stiffnesses to parameter space to design a linear parameter varying controller (LPV) for combined AFS and DYC to perform a commanded lane change task. Further, the nonlinear vehicle lateral dynamics is modeled with Takagi-Sugeno (T-S) framework. A state-feedback fuzzy H∞ controller is designed for both stability and tracking reference. Simulation study confirms that the performance of the proposed methods is quite satisfactory

Impacts of Connected and Automated Vehicles on Energy and Traffic Flow: Optimal Control Design and Verification Through Field Testing

This dissertation assesses eco-driving effectiveness in several key traffic scenarios that include passenger vehicle transportation in highway driving and urban driving that also includes interactions with traffic signals, as well as heavy-duty line-haul truck transportation in highway driving with significant road grade. These studies are accomplished through both traffic microsimulation that propagates individual vehicle interactions to synthesize large-scale traffic patterns that emerge from the eco-driving strategies, and through experimentation in which real prototyped connected and automated vehicles (CAVs) are utilized to directly measure energy benefits from the designed eco-driving control strategies. In particular, vehicle-in-the-loop is leveraged for the CAVs driven on a physical test track to interact with surrounding traffic that is virtually realized through said microsimulation software in real time. In doing so, model predictive control is designed and implemented to create performative eco-driving policies and to select vehicle lane, as well as enforce safety constraints while autonomously driving a real vehicle. Ultimately, eco-driving policies are both simulated and experimentally vetted in a variety of typical driving scenarios to show up to a 50% boost in fuel economy when switching to CAV drivers without compromising traffic flow. The first part of this dissertation specifically assesses energy efficiency of connected and automated passenger vehicles that exploit intention-sharing sourced from both neighboring vehicles in a highway scene and from traffic lights in an urban scene. Linear model predictive control is implemented for CAV motion planning, whereby chance constraints are introduced to balance between traffic compactness and safety, and integer decision variables are introduced for lane selection and collision avoidance in multi-lane environments. Validation results are shown from both large-scale microsimulation and through experimentation of real prototyped CAVs. The second part of this dissertation then assesses energy efficiency of automated line-haul trucks when tasked to aerodynamically platoon. Nonlinear model predictive control is implemented for motion planning, and simulation and experimentation are conducted for platooning verification under highway conditions with traffic. Then, interaction-aware and intention-sharing cooperative control is further introduced to eliminate experimentally measured platoon disengagements that occur on real highways when using only status-sharing control. Finally, the performance of automated drivers versus human drivers are compared in a point-to-point scenario to verify fundamental eco-driving impacts -- experimentally showing eco-driving to boost energy economy by 11% on average even in simple driving scenarios