Control and guidance systems for the navigation of a biomimetic autonomous underwater vehicle

The field of Autonomous Underwater Vehicles (AUVs) has increased dramatically in size and scope over the past three decades. Application areas for AUVs are numerous and varied, from deep sea exploration, to pipeline surveillance to mine clearing. The main concept behind this work was the design and the implementation of a control and guidance system for the navigation of a biomimetic AUV. In particular, the AUV analysed in this project tries to imitate the appearance and approximate the swimming method of an Atlantic Salmon and, for this reason, has been called RoboSalmo

Bio-Inspired Robotics

Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

Neuroinspired control strategies with applications to flapping flight

This dissertation is centered on a theoretical, simulation, and experimental study of control strategies which are inspired by biological systems. Biological systems, along with sufficiently complicated engineered systems, often have many interacting degrees of freedom and need to excite large-displacement oscillations in order to locomote. Combining these factors can make high-level control design difficult. This thesis revolves around three different levels of abstraction, providing tools for analysis and design. First, we consider central pattern generators (CPGs) to control flapping-flight dynamics. The key idea here is dimensional reduction - we want to convert complicated interactions of many degrees of freedom into a handful of parameters which have intuitive connections to the overall system behavior, leaving the control designer unconcerned with the details of particular motions. A rigorous mathematical and control theoretic framework to design complex three-dimensional wing motions is presented based on phase synchronization of nonlinear oscillators. In particular, we show that flapping-flying dynamics without a tail or traditional aerodynamic control surfaces can be effectively controlled by a reduced set of central pattern generator parameters that generate phase-synchronized or symmetry-breaking oscillatory motions of two main wings. Furthermore, by using a Hopf bifurcation, we show that tailless aircraft (inspired by bats) alternating between flapping and gliding can be effectively stabilized by smooth wing motions driven by the central pattern generator network. Results of numerical simulation with a full six-degree-of-freedom flight dynamic model validate the effectiveness of the proposed neurobiologically inspired control approach. Further, we present experimental micro aerial vehicle (MAV) research with low-frequency flapping and articulated wing gliding. The importance of phase difference control via an abstract mathematical model of central pattern generators is confirmed with a robotic bat on a 3-DOF pendulum platform. An aerodynamic model for the robotic bat based on the complex wing kinematics is presented. Closed loop experiments show that control dimension reduction is achievable - unstable longitudinal modes are stabilized and controlled using only two control parameters. A transition of flight modes, from flapping to gliding and vice-versa, is demonstrated within the CPG control scheme. The second major thrust is inspired by this idea that mode switching is useful. Many bats and birds adopt a mixed strategy of flapping and gliding to provide agility when necessary and to increase overall efficiency. This work explores dwell time constraints on switched systems with multiple, possibly disparate invariant limit sets. We show that, under suitable conditions, trajectories globally converge to a superset of the limit sets and then remain in a second, larger superset. We show the effectiveness of the dwell-time conditions by using examples of nonlinear switching limit cycles from our work on flapping flight. This level of abstraction has been found to be useful in many ways, but it also produces its own challenges. For example, we discuss death of oscillation which can occur for many limit-cycle controllers and the difficulty in incorporating fast, high-displacement reflex feedback. This leads us to our third major thrust - considering biologically realistic neuron circuits instead of a limit cycle abstraction. Biological neuron circuits are incredibly diverse in practice, giving us a convincing rationale that they can aid us in our quest for flexibility. Nevertheless, that flexibility provides its own challenges. It is not currently known how most biological neuron circuits work, and little work exists that connects the principles of a neuron circuit to the principles of control theory. We begin the process of trying to bridge this gap by considering the simplest of classical controllers, PD control. We propose a simple two-neuron, two-synapse circuit based on the concept that synapses provide attenuation and a delay. We present a simulation-based method of analysis, including a smoothing algorithm, a steady-state response curve, and a system identification procedure for capturing differentiation. There will never be One True Control Method that will solve all problems. Nature's solution to a diversity of systems and situations is equally diverse. This will inspire many strategies and require a multitude of analysis tools. This thesis is my contribution of a few

Intelligent Control Strategies for an Autonomous Underwater Vehicle

The dynamic characteristics of autonomous underwater vehicles (AUVs) present a control problem that classical methods cannot often accommodate easily. Fundamentally, AUV dynamics are highly non-linear, and the relative similarity between the linear and angular velocities about each degree of freedom means that control schemes employed within other flight vehicles are not always applicable. In such instances, intelligent control strategies offer a more sophisticated approach to the design of the control algorithm. Neurofuzzy control is one such technique, which fuses the beneficial properties of neural networks and fuzzy logic in a hybrid control architecture. Such an approach is highly suited to development of an autopilot for an AUV. Specifically, the adaptive network-based fuzzy inference system (ANFIS) is discussed in Chapter 4 as an effective new approach for neurally tuning course-changing fuzzy autopilots. However, the limitation of this technique is that it cannot be used for developing multivariable fuzzy structures. Consequently, the co-active ANFIS (CANFIS) architecture is developed and employed as a novel multi variable AUV autopilot within Chapter 5, whereby simultaneous control of the AUV yaw and roll channels is achieved. Moreover, this structure is flexible in that it is extended in Chapter 6 to perform on-line control of the AUV leading to a novel autopilot design that can accommodate changing vehicle pay loads and environmental disturbances. Whilst the typical ANFIS and CANFIS structures prove effective for AUV control system design, the well known properties of radial basis function networks (RBFN) offer a more flexible controller architecture. Chapter 7 presents a new approach to fuzzy modelling and employs both ANFIS and CANFIS structures with non-linear consequent functions of composite Gaussian form. This merger of CANFIS and a RBFN lends itself naturally to tuning with an extended form of the hybrid learning rule, and provides a very effective approach to intelligent controller development.The Sea Systems and Platform Integration Sector, Defence Evaluation and Research Agency, Winfrit

Flagellate Underwater Robotics at Macroscale: Design, Modeling, and Characterization

Prokaryotic flagellum is considered as the only known example of a biological “wheel,” a system capable of converting the action of rotatory actuator into a continuous propulsive force. For this reason, flagella are an interesting case study in soft robotics and they represent an appealing source of inspiration for the design of underwater robots. A great number of flagellum-inspired devices exists, but these are all characterized by a size ranging in the micrometer scale and mostly realized with rigid materials. Here, we present the design and development of a novel generation of macroscale underwater propellers that draw their inspiration from flagellated organisms. Through a simple rotatory actuation and exploiting the capability of the soft material to store energy when interacting with the surrounding fluid, the propellers attain different helical shapes that generate a propulsive thrust. A theoretical model is presented, accurately describing and predicting the kinematic and the propulsive capabilities of the proposed solution. Different experimental trials are presented to validate the accuracy of the model and to investigate the performance of the proposed design. Finally, an underwater robot prototype propelled by four flagellar modules is presented

Aquatic escape for micro-aerial vehicles

As our world is experiencing climate changes, we are in need of better monitoring technologies. Most of our planet is covered with water and robots will need to move in aquatic environments. A mobile robotic platform that possesses efficient locomotion and is capable of operating in diverse scenarios would give us an advantage in data collection that can validate climate models, emergency relief and experimental biological research. This field of application is the driving vector of this robotics research which aims to understand, produce and demonstrate solutions of aerial-aquatic autonomous vehicles. However, small robots face major challenges in operating both in water and in air, as well as transition between those fluids, mainly due to the difference of density of the media. This thesis presents the developments of new aquatic locomotion strategies at small scales that further enlarge the operational domain of conventional platforms. This comprises flight, shallow water locomotion and the transition in-between. Their operating principles, manufacturing methods and control methods are discussed and evaluated in detail. I present multiple unique aerial-aquatic robots with various water escape mechanisms, spanning over different scales. The five robotic platforms showcased share similarities that are compared. The take-off methods are analysed carefully and the underlying physics principles put into light. While all presented research fulfils a similar locomotion objective - i.e aerial and aquatic motion - their relevance depends on the environmental conditions and supposed mission. As such, the performance of each vehicle is discussed and characterised in real, relevant conditions. A novel water-reactive fuel thruster is developed for impulsive take-off, allowing consecutive and multiple jump-gliding from the water surface in rough conditions. At a smaller scale, the escape of a milligram robotic bee is achieved. In addition, a new robot class is demonstrated, that employs the same wings for flying as for passive surface sailing. This unique capability allows the flexibility of flight to be combined with long-duration surface missions, enabling autonomous prolonged aquatic monitoring.Open Acces

Bioinspired sensing and control for underwater pursuit

Fish in nature have several distinct advantages over traditional propeller driven underwater vehicles including maneuverability and flow sensing capabilities. Taking inspiration from biology, this work seeks to answer three questions related to bioinspired pursuit and apply the knowledge gained therein to the control of a novel, reaction-wheel driven autonomous fish robot. Which factors are most important to a successful pursuit? How might we guarantee capture with underwater pursuit? How might we track the wake of a flapping fish or vehicle? A technique called probabilistic analytical modeling (PAM) is developed and illustrated by the interactions between predator and prey fish in two case studies that draw on recent experiments. The technique provides a method for investigators to analyze kinematics time series of pursuit to determine which parameters (e.g. speed, flush distance, and escape angles) have the greatest impact on metrics such as probability of survival. Providing theoretical guarantees of capture become complicated in the case of a swimming fish or bioinspired fish robot because of the oscillatory nature fish motion. A feedback control law is shown to result in forward swimming motion in a desired direction. Analysis of this law in a pursuit scenario yields a condition stating whether capture is guaranteed provided some basic information about the motion of the prey. To address wake tracking inspiration is taken from the lateral line sensing organ in fish, which is sensitive to hydrodynamic forces in the local flow field. In experiment, an array of pressure sensors on a Joukowski foil estimates and controls flow-relative position in a Karman vortex street using potential flow theory, recursive Bayesian filtering, and trajectory-tracking, feedback control. The work in this dissertation pushes the state of the art in bioinspired underwater vehicles closer to what can be found in nature. A modeling technique provides a means to determine what is most important to pursuit when designing a vehicle, analysis of a control law shows that a robotic fish is capable of pursuit engagements with capture guarantees, and an estimation framework demonstrates how the wake of a swimming fish or obstacle in the flow can be tracked

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Physics-based Machine Learning Methods for Control and Sensing in Fish-like Robots

Underwater robots are important for the construction and maintenance of underwater infrastructure, underwater resource extraction, and defense. However, they currently fall far behind biological swimmers such as fish in agility, efficiency, and sensing capabilities. As a result, mimicking the capabilities of biological swimmers has become an area of significant research interest. In this work, we focus specifically on improving the control and sensing capabilities of fish-like robots. Our control work focuses on using the Chaplygin sleigh, a two-dimensional nonholonomic system which has been used to model fish-like swimming, as part of a curriculum to train a reinforcement learning agent to control a fish-like robot to track a prescribed path. The agent is first trained on the Chaplygin sleigh model, which is not an accurate model of the swimming robot but crucially has similar physics; having learned these physics, the agent is then trained on a simulated swimming robot, resulting in faster convergence compared to only training on the simulated swimming robot. Our sensing work separately considers using kinematic data (proprioceptive sensing) and using surface pressure sensors. The effect of a swimming body\u27s internal dynamics on proprioceptive sensing is investigated by collecting time series of kinematic data of both a flexible and rigid body in a water tunnel behind a moving obstacle performing different motions, and using machine learning to classify the motion of the upstream obstacle. This revealed that the flexible body could more effectively classify the motion of the obstacle, even if only one if its internal states is used. We also consider the problem of using time series data from a `lateral line\u27 of pressure sensors on a fish-like body to estimate the position of an upstream obstacle. Feature extraction from the pressure data is attempted with a state-of-the-art convolutional neural network (CNN), and this is compared with using the dominant modes of a Koopman operator constructed on the data as features. It is found that both sets of features achieve similar estimation performance using a dense neural network to perform the estimation. This highlights the potential of the Koopman modes as an interpretable alternative to CNNs for high-dimensional time series. This problem is also extended to inferring the time evolution of the flow field surrounding the body using the same surface measurements, which is performed by first estimating the dominant Koopman modes of the surrounding flow, and using those modes to perform a flow reconstruction. This strategy of mapping from surface to field modes is more interpretable than directly constructing a mapping of unsteady fluid states, and is found to be effective at reconstructing the flow. The sensing frameworks developed as a result of this work allow better awareness of obstacles and flow patterns, knowledge which can inform the generation of paths through the fluid that the developed controller can track, contributing to the autonomy of swimming robots in challenging environments

Actor-critic reinforcement learning algorithms for yaw control of an Autonomous Underwater Vehicle

An Autonomous Underwater Vehicle (AUV) poses unique challenges that must be solved in order to achieve persistent autonomy. The requirement of persistent autonomy entails that a control solution must be capable of controlling a vehicle that is operating in an environment with complex non-linear dynamics and adapt to changes in those dynamics. In essence, artificial intelligence is required so that the vehicle can learn from its experience operating in the domain. In this thesis, reinforcement learning is the chosen machine learning mechanism. This learning paradigm is investigated by applying multiple actor-critic temporal difference learning algorithms to the yaw degree-of-freedom of a simulated model and the physical hardware of the Nessie VII AUV in a closed-loop feedback control problem. Additionally, results are also presented for path planning and path optimisation problems. These control problems are solved by modelling the AUV’s interaction with its environment as an optimal decision-making problem using a Markov Decision Process (MDP). Two novel actor-critic temporal difference learning algorithms called Linear True Online Continuous Learning Automation (Linear TOCLA) and Non-linear True Online Continuous Learning Automation (Non-linear TOCLA) are also presented and serve as new contributions to the reinforcement learning research community. These algorithms have been applied to the real Nessie vehicle and its simulated model. The proposed algorithms hold theoretical and practical advantages over previous state-of-the-art temporal difference learning algorithms. A new genetic algorithm is also presented and developed specifically for the optimisation of the continuous-valued reinforcement learning algorithms’. This genetic algorithm is used to find the optimal hyperparameters for four actor-critic algorithms in the well-known continuous-valued mountain car reinforcement learning benchmark problem. The results of this benchmark show that the Non-linear TOCLA algorithm achieves a similar performance to the state-of-the-art forward actor-critic algorithm it extends while significantly reducing the sensitivity of the hyperparameter selection. This reduction in hyperparameter sensitivity is shown using the distribution of optimal hyperparameters from ten separate optimisation runs. The actor learning rate of the forward actor-critic algorithm had a standard deviation of 0.00088, while the Non-linear TOCLA algorithm demonstrated a standard deviation of 0.00186. An even greater improvement is observed in the multi-step target weight, λ, which increased from a standard deviation of 0.036 for the forward actor-critic to 0.266 for the Non-linear TOCLA algorithm. All of the sourcecode used to generate the results in this thesis has been made available as open-source software.ARchaeological RObot systems for the Worlds Seas (ARROWS) EU FP7 project under grant agreement ID 30872