Deep reinforcement learning-based pitch attitude control of a beaver-like underwater robot

The foot paddling of an underwater robot causes continuous changes of the water flow field, which results in the unbalanced hydrodynamic force to change the robot's posture continuously. As the water environment and robot swimming are nonlinear and strongly coupled systems, it is difficult to establish an accurate model. This paper presents an underwater robot, which adopts the synchronous and alternate swimming trajectory of a beaver. Its pitch stability control model is established by using deep reinforcement learning algorithm and its self-learning control system is constructed for stable control of pitch attitude. Experiments are conducted to show that the pitch attitude of the beaver-like underwater robot can be stabilized while maintaining a certain swimming speed. The control method does not need to establish a complex and high-order model of webbed paddling hydrodynamics, which provides a new idea for stable swimming control of underwater robots. This work aims to find an excellent control method for underwater bionic robots. The ocean has the richest natural resources and the most diverse species on Earth. The underwater environment is complex and variable, imposing higher demands on the performance of underwater robots. Increasingly, new concept marine equipment is being researched for scientific exploration, and among these, underwater robots designed based on bionic principles are a growing trend. Currently, most underwater robots still use propellers as their propulsion system. Propellers have advantages such as simple control, high mechanical efficiency, and powerful propulsion, but they also have drawbacks including severe water flow disturbance during operation, high noise, poor concealment, and limited adaptability in complex water environments. Finding a propulsion system with better overall performance is a crucial way to enhance the motion capabilities of underwater robots. Underwater robots often have complex structures, and there are numerous factors influencing their movement in the underwater environment, making fluid dynamics modeling and optimization challenging. Reinforcement learning, as an optimization algorithm, can circumvent the aforementioned difficulties

Locomotion of Low-DoF Multi-legged Robots

Multi-legged robots inspired by insects and other arthropods have unique advantages when compared with bipedal and quadrupedal robots. Their sprawled posture provides stability, and allows them to utilize low-DoF legs which are easier to build and control. With low-DoF legs and multiple contacts with the environment, low-DoF multi-legged robots are usually over constrained if no slipping is allowed. This makes them intrinsically different from the classic bipedal and quadrupedal robots which have high-DoF legs and fewer contacts with the environment. Here we study the unique characteristics of low-DoF multi-legged robots, in terms of design, mobility and modeling. One key observation we prove is that 1-DoF multi-legged robots must slip to be able to steer in the plane. Slipping with multiple contacts makes it difficult to model these robots and their locomotion. Therefore, instead of relying on models, our primary strategy has been careful experimental study. We designed and built our own customized robots which are easily reconfigurable to accommodate a variety of research requirements. In this dissertation we present two robot platforms, BigAnt and Multipod, which demonstrate our design and fabrication methods for low-cost rapidly fabricated modular robotic platforms. BigAnt is a hexapedal robot with 1-DoF legs, whose chassis is constructed from foam board and fiber tape, and costs less than 20 USD in total; Multipod is a highly modular multi-legged robot that can be easily assembled to have different numbers of 2-DoF legs (4 to 12 legs discussed here). We conducted a detailed analysis of steering, including proposing a formal definition of steering gaits grounded in geometric mechanics, and demonstrated the intrinsic difference between legged steering and wheeled steering. We designed gaits for walking, steering, undulating, stair climbing, turning in place, and more, and experimentally tested all these gaits on our robot platforms with detailed motion tracking. Through the theoretical analyses and the experimental tests, we proved that allowing slipping is beneficial for improving the steering in our robots. Where conventional modeling strategies struggle due to multi-contact slipping, we made a significant scientific discovery: that multi-legged locomotion with slipping is often geometric in the sense known from the study of low Reynolds number swimmers and non-holonomic wheeled snake robots which have continuous contact with the environment. We noted that motion can be geometric ``on average'', i.e. stride to stride, and can be truly instantaneously geometric. For each of these we developed a data-driven modeling approach that allowed us to analyze the degree to which a motion is geometric, and applied the analysis to BigAnt and Multipod. These models can also be used for robot motion planning. To explore the mechanism behind the geometric motion characteristics of these robots, we proposed a spring supported multi-legged model. We tested the simulation based on this model against experimental data for all the systems we studied: BigAnt, Multipod, Mechapod (a variant of 6-legged Multipod) and cockroaches. The model prediction results captures many key features of system velocity profiles, but still showed some systematic errors (which can be alleviated ad-hoc). Our work shows the promise of low-DoF multi-legged robots as a class of robotic platforms that are easy to build and simulate, and have many of the mobility advantages of legged systems without the difficulties in stability and control that appear in robots with four or fewer legs.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169985/1/danzhaoy_1.pd