Exploration of genetic network programming with two-stage reinforcement learning for mobile robot

This paper observes the exploration of Genetic Network Programming Two-Stage Reinforcement Learning for mobile robot navigation. The proposed method aims to observe its exploration when inexperienced environments used in the implementation. In order to deal with this situation, individuals are trained firstly in the training phase, that is, they learn the environment with ϵ-greedy policy and learning rate α parameters. Here, two cases are studied, i.e., case A for low exploration and case B for high exploration. In the implementation, the individuals implemented to get experience and learn a new environment on-line. Then, the performance of learning processes are observed due to the environmental changes

Discussion on Different Controllers Used for the Navigation of Mobile Robot

Robots that can comprehend and navigate their surroundings independently on their own are considered intelligent mobile robots (MR). Using a sophisticated set of controllers, artificial intelligence (AI), deep learning (DL), machine learning (ML), sensors, and computation for navigation, MR\u27s can understand and navigate around their environments without even being connected to a cabled source of power. Mobility and intelligence are fundamental drivers of autonomous robots that are intended for their planned operations. They are becoming popular in a variety of fields, including business, industry, healthcare, education, government, agriculture, military operations, and even domestic settings, to optimize everyday activities. We describe different controllers, including proportional integral derivative (PID) controllers, model predictive controllers (MPCs), fuzzy logic controllers (FLCs), and reinforcement learning controllers used in robotics science. The main objective of this article is to demonstrate a comprehensive idea and basic working principle of controllers utilized by mobile robots (MR) for navigation. This work thoroughly investigates several available books and literature to provide a better understanding of the navigation strategies taken by MR. Future research trends and possible challenges to optimizing the MR navigation system are also discussed

A Consolidated Review of Path Planning and Optimization Techniques: Technical Perspectives and Future Directions

In this paper, a review on the three most important communication techniques (ground, aerial, and underwater vehicles) has been presented that throws light on trajectory planning, its optimization, and various issues in a summarized way. This kind of extensive research is not often seen in the literature, so an effort has been made for readers interested in path planning to fill the gap. Moreover, optimization techniques suitable for implementing ground, aerial, and underwater vehicles are also a part of this review. This paper covers the numerical, bio-inspired techniques and their hybridization with each other for each of the dimensions mentioned. The paper provides a consolidated platform, where plenty of available research on-ground autonomous vehicle and their trajectory optimization with the extension for aerial and underwater vehicles are documented

Adaptive and intelligent navigation of autonomous planetary rovers - A survey

The application of robotics and autonomous systems in space has increased dramatically. The ongoing Mars rover mission involving the Curiosity rover, along with the success of its predecessors, is a key milestone that showcases the existing capabilities of robotic technology. Nevertheless, there has still been a heavy reliance on human tele-operators to drive these systems. Reducing the reliance on human experts for navigational tasks on Mars remains a major challenge due to the harsh and complex nature of the Martian terrains. The development of a truly autonomous rover system with the capability to be effectively navigated in such environments requires intelligent and adaptive methods fitting for a system with limited resources. This paper surveys a representative selection of work applicable to autonomous planetary rover navigation, discussing some ongoing challenges and promising future research directions from the perspectives of the authors

A Multiagent Approach to Qualitative Navigation in Robotics

Navigation in unknown unstructured environments is still a difficult open problem in the field of robotics. In this PhD thesis we present a novel approach for robot navigation based on the combination of landmark-based navigation, fuzzy distances and angles representation and multiagent coordination based on a bidding mechanism. The objective has been to have a robust navigation system with orientation sense for unstructured environments using visual information. To achieve such objective we have focused our efforts on two main threads: navigation and mapping methods, and control architectures for autonomous robots. Regarding the navigation and mapping task, we have extended the work presented by Prescott, so that it can be used with fuzzy information about the locations of landmarks in the environment. Together with this extension, we have also developed methods to compute diverting targets, needed by the robot when it gets blocked. Regarding the control architecture, we have proposed a general architecture that uses a bidding mechanism to coordinate a group of systems that control the robot. This mechanism can be used at different levels of the control architecture. In our case, we have used it to coordinate the three systems of the robot (Navigation, Pilot and Vision systems) and also to coordinate the agents that compose the Navigation system itself. Using this bidding mechanism the action actually being executed by the robot is the most valued one at each point in time, so, given that the agents bid rationally, the dynamics of the biddings would lead the robot to execute the necessary actions in order to reach a given target. The advantage of using such mechanism is that there is no need to create a hierarchy, such in the subsumption architecture, but it is dynamically changing depending on the specific situation of the robot and the characteristics of the environment. We have obtained successful results, both on simulation and on real experimentation, showing that the mapping system is capable of building a map of an unknown environment and use this information to move the robot from a starting point to a given target. The experimentation also showed that the bidding mechanism we designed for controlling the robot produces the overall behavior of executing the proper action at each moment in order to reach the target

A Systematic Literature Review of Path-Planning Strategies for Robot Navigation in Unknown Environment

The Many industries, including ports, space, surveillance, military, medicine and agriculture have benefited greatly from mobile robot technology.  An autonomous mobile robot navigates in situations that are both static and dynamic. As a result, robotics experts have proposed a range of strategies. Perception, localization, path planning, and motion control are all required for mobile robot navigation. However, Path planning is a critical component of a quick and secure navigation. Over the previous few decades, many path-planning algorithms have been developed. Despite the fact that the majority of mobile robot applications take place in static environments, there is a scarcity of algorithms capable of guiding robots in dynamic contexts. This review compares qualitatively mobile robot path-planning systems capable of navigating robots in static and dynamic situations. Artificial potential fields, fuzzy logic, genetic algorithms, neural networks, particle swarm optimization, artificial bee colonies, bacterial foraging optimization, and ant-colony are all discussed in the paper. Each method's application domain, navigation technique and validation context are discussed and commonly utilized cutting-edge methods are analyzed. This research will help researchers choose appropriate path-planning approaches for various applications including robotic cranes at the sea ports as well as discover gaps for optimization

Adaptive dynamic programming with eligibility traces and complexity reduction of high-dimensional systems

This dissertation investigates the application of a variety of computational intelligence techniques, particularly clustering and adaptive dynamic programming (ADP) designs especially heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Moreover, a one-step temporal-difference (TD(0)) and n-step TD (TD(λ)) with their gradients are utilized as learning algorithms to train and online-adapt the families of ADP. The dissertation is organized into seven papers. The first paper demonstrates the robustness of model order reduction (MOR) for simulating complex dynamical systems. Agglomerative hierarchical clustering based on performance evaluation is introduced for MOR. This method computes the reduced order denominator of the transfer function by clustering system poles in a hierarchical dendrogram. Several numerical examples of reducing techniques are taken from the literature to compare with our work. In the second paper, a HDP is combined with the Dyna algorithm for path planning. The third paper uses DHP with an eligibility trace parameter (λ) to track a reference trajectory under uncertainties for a nonholonomic mobile robot by using a first-order Sugeno fuzzy neural network structure for the critic and actor networks. In the fourth and fifth papers, a stability analysis for a model-free action-dependent HDP(λ) is demonstrated with batch- and online-implementation learning, respectively. The sixth work combines two different gradient prediction levels of critic networks. In this work, we provide a convergence proofs. The seventh paper develops a two-hybrid recurrent fuzzy neural network structures for both critic and actor networks. They use a novel n-step gradient temporal-difference (gradient of TD(λ)) of an advanced ADP algorithm called value-gradient learning (VGL(λ)), and convergence proofs are given. Furthermore, the seventh paper is the first to combine the single network adaptive critic with VGL(λ). --Abstract, page iv