Learning-based adaption of robotic friction models

In the Fourth Industrial Revolution, wherein artificial intelligence and the automation of machines occupy a central role, the deployment of robots is indispensable. However, the manufacturing process using robots, especially in collaboration with humans, is highly intricate. In particular, modeling the friction torque in robotic joints is a longstanding problem due to the lack of a good mathematical description. This motivates the usage of data-driven methods in recent works. However, model-based and data-driven models often exhibit limitations in their ability to generalize beyond the specific dynamics they were trained on, as we demonstrate in this paper. To address this challenge, we introduce a novel approach based on residual learning, which aims to adapt an existing friction model to new dynamics using as little data as possible. We validate our approach by training a base neural network on a symmetric friction data set to learn an accurate relation between the velocity and the friction torque. Subsequently, to adapt to more complex asymmetric settings, we train a second network on a small dataset, focusing on predicting the residual of the initial network's output. By combining the output of both networks in a suitable manner, our proposed estimator outperforms the conventional model-based approach and the base neural network significantly. Furthermore, we evaluate our method on trajectories involving external loads and still observe a substantial improvement, approximately 60-70\%, over the conventional approach. Our method does not rely on data with external load during training, eliminating the need for external torque sensors. This demonstrates the generalization capability of our approach, even with a small amount of data-only 43 seconds of a robot movement-enabling adaptation to diverse scenarios based on prior knowledge about friction in different settings

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Models for reinforcement learning and design of a soft robot inspired by Drosophila larvae

Designs for robots are often inspired by animals, as they are designed mimicking animals’ mechanics, motions, behaviours and learning. The Drosophila, known as the fruit fly, is a well-studied model animal. In this thesis, the Drosophila larva is studied and the results are applied to robots. More specifically: a part of the Drosophila larva’s neural circuit for operant learning is modelled, based on which a synaptic plasticity model and a neural circuit model for operant learning, as well as a dynamic neural network for robot reinforcement learning, are developed; then Drosophila larva’s motor system for locomotion is studied, and based on it a soft robot system is designed. Operant learning is a concept similar to reinforcement learning in computer science, i.e. learning by reward or punishment for behaviour. Experiments have shown that a wide range of animals is capable of operant learning, including animal with only a few neurons, such as Drosophila. The fact implies that operant learning can establish without a large number of neurons. With it as an assumption, the structure and dynamics of synapses are investigated, and a synaptic plasticity model is proposed. The model includes nonlinear dynamics of synapses, especially receptor trafficking which affects synaptic strength. Tests of this model show it can enable operant learning at the neuron level and apply to a broad range of NNs, including feedforward, recurrent and spiking NNs. The mushroom body is a learning centre of the insect brain known and modelled for associative learning, but not yet for operant learning. To investigate whether it participates in operant learning, Drosophila larvae are studied with a transgenic tool by my collaborators. Based on the experiment and the results, a mushroom body model capable of operant learning is modelled. The proposed neural circuit model can reproduce the operant learning of the turning behaviour of Drosophila larvae. Then the synaptic plasticity model is simplified for robot learning. With the simplified model, a recurrent neural network with internal neural dynamics can learn to control a planar bipedal robot in a benchmark reinforcement learning task which is called bipedal walker by OpenAI. Benefiting efficiency in parameter space exploration instead of action space exploration, it is the first known solution to the task with reinforcement learning approaches. Although existing pneumatic soft robots can have multiple muscles embedded in a component, it is far less than the muscles in the Drosophila larva, which are well-organised in a tiny space. A soft robot system is developed based on the muscle pattern of the Drosophila larva, to explore the possibility to embed a high density of muscles in a limited space. Three versions of the body wall with pneumatic muscles mimicking the muscle pattern are designed. A pneumatic control system and embedded control system are also developed for controlling the robot. With a bioinspired body wall will a large number of muscles, the robot performs lifelike motions in experiments

Kinematic Analysis of Multi-Fingered, Anthropomorphic Robotic Hands

The ability of stable grasping and fine manipulation with the multi-fingered robot hand with required precision and dexterity is playing an increasingly important role in the applications like service robots, rehabilitation, humanoid robots, entertainment robots, industries etc.. A number of multi-fingered robotic hands have been developed by various researchers in the past. The distinct advantages of a multi-fingered robot hand having structural similarity with human hand motivate the need for an anthropomorphic robot hand. Such a hand provides a promising base for supplanting human hand in execution of tedious, complicated and dangerous tasks, especially in situations such as manufacturing, space, undersea etc. These can also be used in orthopaedic rehabilitation of humans for improving the quality of the life of people having orthopedically and neurological disabilities. The developments so far are mostly driven by the application requirements. There are a number of bottlenecks with industrial grippers as regards to the stability of grasping objects of irregular geometries or complex manipulation operations. A multi-fingered robot hand can be made to mimic the movements of a human hand. The present piece of research work attempts to conceptualize and design a multi-fingered, anthropomorphic robot hand by structurally imitating the human hand. In the beginning, a brief idea about the history, types of robotic hands and application of multi-fingered hands in various fields are presented. A review of literature based on different aspects of the multi-fingered hand like structure, control, optimization, gasping etc. is made. Some of the important and more relevant literatures are elaborately discussed and a brief analysis is made on the outcomes and shortfalls with respect to multi-fingered hands. Based on the analysis of the review of literature, the research work aims at developing an improved anthropomorphic robot hand model in which apart from the four fingers and a thumb, the palm arch effect of human hand is also considered to increase its dexterity. A robotic hand with five anthropomorphic fingers including the thumb and palm arch effect having 25 degrees-of-freedom in all is investigated in the present work. Each individual finger is considered as an open loop kinematic chain and each finger segment is considered as a link of the manipulator. The wrist of the hand is considered as a fixed point. The kinematic analyses of the model for both forward kinematics and inverse kinematic are carried out. The trajectories of the tip positions of the thumb and the fingers with respect to local coordinate system are determined and plotted. This gives the extreme position of the fingertips which is obtained from the forward kinematic solution with the help of MATLAB. Similarly, varying all the joint iv angles of the thumb and fingers in their respective ranges, the reachable workspace of the hand model is obtained. Adaptive Neuro-Fuzzy Inference System (ANFIS) is used for solving the inverse kinematic problem of the fingers. Since the multi-fingered hand grasps the object mainly through its fingertips and the manipulation of the object is facilitated by the fingers due to their dexterity, the grasp is considered to be force-closure grasp. The grasping theory and different types of contacts between the fingertip and object are presented and the conditions for stable and equilibrium grasp are elaborately discussed. The proposed hand model is simulated to grasp five different shaped objects with equal base dimension and height. The forces applied on the fingertip during grasping are calculated. The hand model is also analysed using ANSYS to evaluate the stresses being developed at various points in the thumb and fingers. This analysis was made for the hand considering two different hand materials i.e. aluminium alloy and structural steel. The solution obtained from the forward kinematic analysis of the hand determines the maximum size for differently shaped objects while the solution to the inverse kinematic problem indicates the configurations of the thumb and the fingers inside the workspace of the hand. The solutions are predicted in which all joint angles are within their respective ranges. The results of the stress analysis of the hand model show that the structure of the fingers and the hand as a whole is capable of handling the selected objects. The robot hand under investigation can be realized and can be a very useful tool for many critical areas such as fine manipulation of objects, combating orthopaedic or neurological impediments, service robotics, entertainment robotics etc. The dissertation concludes with a summary of the contribution and the scope of further work

Evolutionary Computation

This book presents several recent advances on Evolutionary Computation, specially evolution-based optimization methods and hybrid algorithms for several applications, from optimization and learning to pattern recognition and bioinformatics. This book also presents new algorithms based on several analogies and metafores, where one of them is based on philosophy, specifically on the philosophy of praxis and dialectics. In this book it is also presented interesting applications on bioinformatics, specially the use of particle swarms to discover gene expression patterns in DNA microarrays. Therefore, this book features representative work on the field of evolutionary computation and applied sciences. The intended audience is graduate, undergraduate, researchers, and anyone who wishes to become familiar with the latest research work on this field

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

With the rapid growth and development of cities, Intelligent Traffic Management and Control (ITMC) is becoming a fundamental component to address the challenges of modern urban traffic management, where a wide range of daily problems need to be addressed in a prompt and expedited manner. Issues such as unpredictable traffic dynamics, resource constraints, and abnormal events pose difficulties to city managers. ITMC aims to increase the efficiency of traffic management by minimizing the odds of traffic problems, by providing real-time traffic state forecasts to better schedule the intersection signal controls. Reliable implementations of ITMC improve the safety of inhabitants and the quality of life, leading to economic growth. In recent years, researchers have proposed different solutions to address specific problems concerning traffic management, ranging from image-processing and deep-learning techniques to forecasting the traffic state and deriving policies to control intersection signals. This review article studies the primary public datasets helpful in developing models to address the identified problems, complemented with a deep analysis of the works related to traffic state forecast and intersection-signal-control models. Our analysis found that deep-learning-based approaches for short-term traffic state forecast and multi-intersection signal control showed reasonable results, but lacked robustness for unusual scenarios, particularly during oversaturated situations, which can be resolved by explicitly addressing these cases, potentially leading to significant improvements of the systems overall. However, there is arguably a long path until these models can be used safely and effectively in real-world scenarios

Friction Force Microscopy of Deep Drawing Made Surfaces

Aim of this paper is to contribute to micro-tribology understanding and friction in micro-scale interpretation in case of metal beverage production, particularly the deep drawing process of cans. In order to bridging the gap between engineering and trial-and-error principles, an experimental AFM-based micro-tribological approach is adopted. For that purpose, the can’s surfaces are imaged with atomic force microscopy (AFM) and the frictional force signal is measured with frictional force microscopy (FFM). In both techniques, the sample surface is scanned with a stylus attached to a cantilever. Vertical motion of the cantilever is recorded in AFM and horizontal motion is recorded in FFM. The presented work evaluates friction over a micro-scale on various samples gathered from cylindrical, bottom and round parts of cans, made of same the material but with different deep drawing process parameters. The main idea is to link the experimental observation with the manufacturing process. Results presented here can advance the knowledge in order to comprehend the tribological phenomena at the contact scales, too small for conventional tribology

Towards a Conceptual Design of an Intelligent Material Transport Based on Machine Learning and Axiomatic Design Theory

Reliable and efficient material transport is one of the basic requirements that affect productivity in sheet metal industry. This paper presents a methodology for conceptual design of intelligent material transport using mobile robot, based on axiomatic design theory, graph theory and artificial intelligence. Developed control algorithm was implemented and tested on the mobile robot system Khepera II within the laboratory model of manufacturing environment. Matlab© software package was used for manufacturing process simulation, implementation of search algorithms and neural network training. Experimental results clearly show that intelligent mobile robot can learn and predict optimal material transport flows thanks to the use of artificial neural networks. Achieved positioning error of mobile robot indicates that conceptual design approach can be used for material transport and handling tasks in intelligent manufacturing systems