A vision-guided parallel parking system for a mobile robot using approximate policy iteration

Reinforcement Learning (RL) methods enable autonomous robots to learn skills from scratch by interacting with the environment. However, reinforcement learning can be very time consuming. This paper focuses on accelerating the reinforcement learning process on a mobile robot in an unknown environment. The presented algorithm is based on approximate policy iteration with a continuous state space and a fixed number of actions. The action-value function is represented by a weighted combination of basis functions. Furthermore, a complexity analysis is provided to show that the implemented approach is guaranteed to converge on an optimal policy with less computational time. A parallel parking task is selected for testing purposes. In the experiments, the efficiency of the proposed approach is demonstrated and analyzed through a set of simulated and real robot experiments, with comparison drawn from two well known algorithms (Dyna-Q and Q-learning)

Driving a car with custom-designed fuzzy inferencing VLSI chips and boards

Vehicle control in a-priori unknown, unpredictable, and dynamic environments requires many calculational and reasoning schemes to operate on the basis of very imprecise, incomplete, or unreliable data. For such systems, in which all the uncertainties can not be engineered away, approximate reasoning may provide an alternative to the complexity and computational requirements of conventional uncertainty analysis and propagation techniques. Two types of computer boards including custom-designed VLSI chips were developed to add a fuzzy inferencing capability to real-time control systems. All inferencing rules on a chip are processed in parallel, allowing execution of the entire rule base in about 30 microseconds, and therefore, making control of 'reflex-type' of motions envisionable. The use of these boards and the approach using superposition of elemental sensor-based behaviors for the development of qualitative reasoning schemes emulating human-like navigation in a-priori unknown environments are first discussed. Then how the human-like navigation scheme implemented on one of the qualitative inferencing boards was installed on a test-bed platform to investigate two control modes for driving a car in a-priori unknown environments on the basis of sparse and imprecise sensor data is described. In the first mode, the car navigates fully autonomously, while in the second mode, the system acts as a driver's aid providing the driver with linguistic (fuzzy) commands to turn left or right and speed up or slow down depending on the obstacles perceived by the sensors. Experiments with both modes of control are described in which the system uses only three acoustic range (sonar) sensor channels to perceive the environment. Simulation results as well as indoors and outdoors experiments are presented and discussed to illustrate the feasibility and robustness of autonomous navigation and/or safety enhancing driver's aid using the new fuzzy inferencing hardware system and some human-like reasoning schemes which may include as little as six elemental behaviors embodied in fourteen qualitative rules

Control of autonomous multibody vehicles using artificial intelligence

The field of autonomous driving has been evolving rapidly within the last few years and a lot of research has been dedicated towards the control of autonomous vehicles, especially car-like ones. Due to the recent successes of artificial intelligence techniques, even more complex problems can be solved, such as the control of autonomous multibody vehicles. Multibody vehicles can accomplish transportation tasks in a faster and cheaper way compared to multiple individual mobile vehicles or robots. But even for a human, driving a truck-trailer is a challenging task. This is because of the complex structure of the vehicle and the maneuvers that it has to perform, such as reverse parking to a loading dock. In addition, the detailed technical solution for an autonomous truck is challenging and even though many single-domain solutions are available, e.g. for pathplanning, no holistic framework exists. Also, from the control point of view, designing such a controller is a high complexity problem, which makes it a widely used benchmark. In this thesis, a concept for a plurality of tasks is presented. In contrast to most of the existing literature, a holistic approach is developed which combines many stand-alone systems to one entire framework. The framework consists of a plurality of modules, such as modeling, pathplanning, training for neural networks, controlling, jack-knife avoidance, direction switching, simulation, visualization and testing. There are model-based and model-free control approaches and the system comprises various pathplanning methods and target types. It also accounts for noisy sensors and the simulation of whole environments. To achieve superior performance, several modules had to be developed, redesigned and interlinked with each other. A pathplanning module with multiple available methods optimizes the desired position by also providing an efficient implementation for trajectory following. Classical approaches, such as optimal control (LQR) and model predictive control (MPC) can safely control a truck with a given model. Machine learning based approaches, such as deep reinforcement learning, are designed, implemented, trained and tested successfully. Furthermore, the switching of the driving direction is enabled by continuous analysis of a cost function to avoid collisions and improve driving behavior. This thesis introduces a working system of all integrated modules. The system proposed can complete complex scenarios, including situations with buildings and partial trajectories. In thousands of simulations, the system using the LQR controller or the reinforcement learning agent had a success rate of >95 % in steering a truck with one trailer, even with added noise. For the development of autonomous vehicles, the implementation of AI at scale is important. This is why a digital twin of the truck-trailer is used to simulate the full system at a much higher speed than one can collect data in real life.Tesi

Neuro-fuzzy techniques to optimize an FPGA embedded controller for robot navigation

This paper describes how low-cost embedded controllers for robot navigation can be obtained by using a small number of if-then rules (exploiting the connection in cascade of rule bases) that apply Takagi-Sugeno fuzzy inference method and employ fuzzy sets represented by normalized triangular functions. The rules comprise heuristic and fuzzy knowledge together with numerical data obtained from a geometric analysis of the control problem that considers the kinematic and dynamic constraints of the robot. Numerical data allow tuning the fuzzy symbols used in the rules to optimize the controller performance. From the implementation point of view, very few computational and memory resources are required: standard logical, addition, and multiplication operations and a few data that can be represented by integer values. This is illustrated with the design of a controller for the safe navigation of an autonomous car-like robot among possible obstacles toward a goal configuration. Implementation results of an FPGA embedded system based on a general-purpose soft processor confirm that percentage reduction in clock cycles is drastic thanks to applying the proposed neuro-fuzzy techniques. Simulation and experimental results obtained with the robot confirm the efficiency of the controller designed. Design methodology has been supported by the CAD tools of the environment Xfuzzy 3 and by the Embedded System Tools from Xilinx. © 2014 Elsevier B.V.Peer Reviewe

Design and Development of Fuzzy-PID Controller for Four-Wheeled Mobile Robotic Stability: A Case Study on the Uphill Road

Design intelligent control to maximize the performance of the Four-Wheeled Mobile Robot (FWMR) in case of uphill road problems. The variation of slope angle and load variation on the uphill road is applied to know the performance of the automatic control system on FWMR. The research is divided into three two covering, system identification, design system and simulation testing. The Rotary speed control system response with fuzzy-PID control method has a good performance by anticipating various tilt angle 5o. The system can through uphill with loads 11N with a steady time and fast travel time. The best travel time in the 5o tilt angle condition is 2 seconds with a 4m circuit length. This research concludes that fuzzy-PID control can be implemented and increase the dynamic response of the FWMR effectively on the uphill road problems

Hardware Implementation of Soft Computing Approaches for an Intelligent Wall-following Vehicle

Soft computing techniques are generally well-suited for vehicular control systems that are usually modeled by highly nonlinear differential equations and working in unstructured environment. To demonstrate their applicability, two intelligent controllers based upon fuzzy logic theories and neural network paradigms are designed for performing a wall-following task and an autonomous parking task. Based on performance and flexibility considerations, the two controllers are implemented onto a reconfigurable hardware platform, namely a Field Programmable Gate Array (FPGA). As the number of comparative studies of these two embedded controllers designed for the same application is limited in the literature, one of the main goals of this research work has been to evaluate and compare the two controllers in terms of hardware resource requirements, operational speeds and trajectory tracking errors in following different pre-defined trajectories. The main advantages and disadvantages of each of the controllers are presented and discussed in details. Challenging issues for implementation of the controllers on the FPGA platform are also highlighted. As the two controllers exhibit benefits and drawbacks under different circumstances, this research suggests as well a hybrid controller scheme as an attempt to integrate the benefits of both control units. To evaluate its performance, the hybrid controller is tested on the same pre-defined trajectories and the corresponding results are compared to that of the fuzzy logic and the neural network based controllers. For further demonstration of the capabilities of the wall-following controllers in other applications, the fuzzy logic and the neural network controllers are used in a parallel parking system. We see this work to be a stepping stone for further research work aiming at real world implementation of the controllers on Application Specified Integrated Circuit (ASIC) type of environment