Visual Servoing in Robotics

Visual servoing is a well-known approach to guide robots using visual information. Image processing, robotics, and control theory are combined in order to control the motion of a robot depending on the visual information extracted from the images captured by one or several cameras. With respect to vision issues, a number of issues are currently being addressed by ongoing research, such as the use of different types of image features (or different types of cameras such as RGBD cameras), image processing at high velocity, and convergence properties. As shown in this book, the use of new control schemes allows the system to behave more robustly, efficiently, or compliantly, with fewer delays. Related issues such as optimal and robust approaches, direct control, path tracking, or sensor fusion are also addressed. Additionally, we can currently find visual servoing systems being applied in a number of different domains. This book considers various aspects of visual servoing systems, such as the design of new strategies for their application to parallel robots, mobile manipulators, teleoperation, and the application of this type of control system in new areas

Dynamics and control of robotic systems for on-orbit objects manipulation

Multi-body systems (MSs) are assemblies composed of multiple bodies (either rigid or structurally flexible) connected among each other by means of mechanical joints. In many engineering fields (such as aerospace, aeronautics, robotics, machinery, military weapons and bio-mechanics) a large number of systems (e.g. space robots, aircraft, terrestrial vehicles, industrial machinery, launching systems) can be included in this category. The dynamic characteristics and performance of such complex systems need to be accurately and rapidly analyzed and predicted. Taking this engineering background into consideration, a new branch of study, named as Multi-body Systems Dynamics (MSD), emerged in the 1960s and has become an important research and development area in modern mechanics; it mainly addresses the theoretical modeling, numerical analysis, design optimization and control for complex MSs. The research on dynamics modeling and numerical solving techniques for rigid multi-body systems has relatively matured and perfected through the developments over the past half century. However, for many engineering problems, the rigid multi-body system model cannot meet the requirements in terms of precision. It is then necessary to consider the coupling between the large rigid motions of the MS components and their elastic displacements; thus the study of the dynamics of flexible MSs has gained increasing relevance. The flexible MSD involves many theories and methods, such as continuum mechanics, computational mechanics and nonlinear dynamics, thus implying a higher requirement on the theoretical basis. Robotic on-orbit operations for servicing, repairing or de-orbiting existing satellites are among space mission concepts expected to have a relevant role in a close future. In particular, many studies have been focused on removing significant debris objects from their orbit. While mission designs involving tethers, nets, harpoons or glues are among options studied and analyzed by the scientific and industrial community, the debris removal by means of robotic manipulators seems to be the solution with the longest space experience. In fact, robotic manipulators are now a well-established technology in space applications as they are routinely used for handling and assembling large space modules and for reducing human extravehicular activities on the International Space Station. The operations are generally performed in a tele-operated approach, where the slow motion of the robotic manipulator is controlled by specialized operators on board of the space station or at the ground control center. Grasped objects are usually cooperative, meaning they are capable to re-orient themselves or have appropriate mechanisms for engagement with the end-effectors of the manipulator (i.e. its terminal parts). On the other hand, debris removal missions would target objects which are often non-controlled and lacking specific hooking points. Moreover, there would be a distinctive advantage in terms of cost and reliability to conduct this type of mission profile in a fully autonomous manner, as issues like obstacle avoidance could be more easily managed locally than from a far away control center. Space Manipulator Systems (SMSs) are satellites made of a base platform equipped with one or more robotic arms. A SMS is a floating system because its base is not fixed to the ground like in terrestrial manipulators; therefore, the motion of the robotic arms affects the attitude and position of the base platform and vice versa. This reciprocal influence is denoted as "dynamic coupling" and makes the dynamics modeling and motion planning of a space robot much more complicated than those of fixed-base manipulators. Indeed, SMSs are complex systems whose dynamics modeling requires appropriate theoretical and mathematical tools. The growing importance SMSs are acquiring is due to their operational ductility as they are able to perform complicated tasks such as repairing, refueling, re-orbiting spacecraft, assembling articulated space structures and cleaning up the increasing amount of space debris. SMSs have also been employed in several rendezvous and docking missions. They have also been the object of many studies which verified the possibility to extend the operational life of commercial and scientific satellites by using an automated servicing spacecraft dedicated to repair, refuel and/or manage their failures (e.g. DARPA's Orbital Express and JAXA's ETS VII). Furthermore, Active Debris Removal (ADR) via robotic systems is one of the main concerns governments and space agencies have been facing in the last years. As a result, the grasping and post-grasping operations on non-cooperative objects are still open research areas facing many technical challenges: the target object identification by means of passive or active optical techniques, the estimation of its kinematic state, the design of dexterous robotic manipulators and end-effectors, the multi-body dynamics analysis, the selection of approaching and grasping maneuvers and the post-grasping mission planning are the main open research challenges in this field. The missions involving the use of SMSs are usually characterized by the following typical phases: 1. Orbital approach; 2. Rendez-vous; 3. Robotic arm(s) deployment; 4. Pre-grasping; 5. Grasping and post-grasping operations. This thesis project will focus on the last three. The manuscript is structured as follows: Chapter 1 presents the derivation of a multi-body system dynamics equations further developing them to reach their Kane's formulation; Chapter 2 investigates two different approaches (Particle Swarm Optimization and Machine Learning) dealing with a space manipulator deployment maneuver; Chapter 3 addresses the design of a combined Impedance+PD controller capable of accomplishing the pre-grasping phase goals and Chapter 4 is dedicated to the dynamic modeling of the closed-loop kinematic chain formed by the manipulator and the grasped target object and to the synthesis of a Jacobian Transpose+PD controller for a post-grasping docking maneuver. Finally, the concluding remarks summarize the overall thesis contribution

Design and Development of an Automated Mobile Manipulator for Industrial Applications

This thesis presents the modeling, control and coordination of an automated mobile manipulator. A mobile manipulator in this investigation consists of a robotic manipulator and a mobile platform resulting in a hybrid mechanism that includes a mobile platform for locomotion and a manipulator arm for manipulation. The structural complexity of a mobile manipulator is the main challenging issue because it includes several problems like adapting a manipulator and a redundancy mobile platform at non-holonomic constraints. The objective of the thesis is to fabricate an automated mobile manipulator and develop control algorithms that effectively coordinate the arm manipulation and mobility of mobile platform. The research work starts with deriving the motion equations of mobile manipulators. The derivation introduced here makes use of motion equations of robot manipulators and mobile platforms separately, and then integrated them as one entity. The kinematic analysis is performed in two ways namely forward & inverse kinematics. The motion analysis is performed for various WMPs such as, Omnidirectional WMP, Differential three WMP, Three wheeled omni-steer WMP, Tricycle WMP and Two steer WMP. From the obtained motion analysis results, Differential three WMP is chosen as the mobile platform for the developed mobile manipulator. Later motion analysis is carried out for 4-axis articulated arm. Danvit-Hartenberg representation is implemented to perform forward kinematic analysis. Because of this representation, one can easily understand the kinematic equation for a robotic arm. From the obtained arm equation, Inverse kinematic model for the 4-axis robotic manipulator is developed. Motion planning of an intelligent mobile robot is one of the most vital issues in the field of robotics, which includes the generation of optimal collision free trajectories within its work space and finally reaches its target position. For solving this problem, two evolutionary algorithms namely Particle Swarm Optimization (PSO) and Artificial Immune System (AIS) are introduced to move the mobile platform in intelligent manner. The developed algorithms are effective in avoiding obstacles, trap situations and generating optimal paths within its unknown environments. Once the robot reaches its goal (within the work space of the manipulator), the manipulator will generate its trajectories according to task assigned by the user. Simulation analyses are performed using MATLAB-2010 in order to validate the feasibility of the developed methodologies in various unknown environments. Additionally, experiments are carried out on an automated mobile manipulator. ATmega16 Microcontrollers are used to enable the entire robot system movement in desired trajectories by means of robot interface application program. The control program is developed in robot software (Keil) to control the mobile manipulator servomotors via a serial connection through a personal computer. To support the proposed control algorithms both simulation and experimental results are presented. Moreover, validation of the developed methodologies has been made with the ER-400 mobile platform

Multi-objective optimization design of parallel manipulators using a neural network and principal component analysis

In this work, a multi-objective optimization design method is proposed based on principal component analysis (PCA) and a neural network to obtain a mechanism's optimal comprehensive performance. First, multi-objective optimization mathematical modeling, including design parameters, objective functions, and constraint functions, is established. Second, the sample data are obtained through the design of the experiment (DOE) and are then standardized to eliminate the adverse effects of a non-uniform dimension of objective functions. Third, the first k principal components are established for p performance indices (k&lt;p) using the variance-based PCA method, and then the factor analysis method is employed to define its physical meaning. Fourth, the overall comprehensive performance evaluation index is established by objectively determining weight factors. Finally, the computational cost of the modeling is improved by combining the neural network and a particle swarm optimization (PSO) algorithm. Dimensional synthesis of a Sprint (3RPS) parallel manipulator (PM) is taken as a case study to implement the proposed method, and the optimization results are verified by a comprehensive performance comparison of robots before and after optimization.</p

Inverse Kinematic Analysis of Robot Manipulators

An important part of industrial robot manipulators is to achieve desired position and orientation of end effector or tool so as to complete the pre-specified task. To achieve the above stated goal one should have the sound knowledge of inverse kinematic problem. The problem of getting inverse kinematic solution has been on the outline of various researchers and is deliberated as thorough researched and mature problem. There are many fields of applications of robot manipulators to execute the given tasks such as material handling, pick-n-place, planetary and undersea explorations, space manipulation, and hazardous field etc. Moreover, medical field robotics catches applications in rehabilitation and surgery that involve kinematic, dynamic and control operations. Therefore, industrial robot manipulators are required to have proper knowledge of its joint variables as well as understanding of kinematic parameters. The motion of the end effector or manipulator is controlled by their joint actuator and this produces the required motion in each joints. Therefore, the controller should always supply an accurate value of joint variables analogous to the end effector position. Even though industrial robots are in the advanced stage, some of the basic problems in kinematics are still unsolved and constitute an active focus for research. Among these unsolved problems, the direct kinematics problem for parallel mechanism and inverse kinematics for serial chains constitute a decent share of research domain. The forward kinematics of robot manipulator is simpler problem and it has unique or closed form solution. The forward kinematics can be given by the conversion of joint space to Cartesian space of the manipulator. On the other hand inverse kinematics can be determined by the conversion of Cartesian space to joint space. The inverse kinematic of the robot manipulator does not provide the closed form solution. Hence, industrial manipulator can achieve a desired task or end effector position in more than one configuration. Therefore, to achieve exact solution of the joint variables has been the main concern to the researchers. A brief introduction of industrial robot manipulators, evolution and classification is presented. The basic configurations of robot manipulator are demonstrated and their benefits and drawbacks are deliberated along with the applications. The difficulties to solve forward and inverse kinematics of robot manipulator are discussed and solution of inverse kinematic is introduced through conventional methods. In order to accomplish the desired objective of the work and attain the solution of inverse kinematic problem an efficient study of the existing tools and techniques has been done. A review of literature survey and various tools used to solve inverse kinematic problem on different aspects is discussed. The various approaches of inverse kinematic solution is categorized in four sections namely structural analysis of mechanism, conventional approaches, intelligence or soft computing approaches and optimization based approaches. A portion of important and more significant literatures are thoroughly discussed and brief investigation is made on conclusions and gaps with respect to the inverse kinematic solution of industrial robot manipulators. Based on the survey of tools and techniques used for the kinematic analysis the broad objective of the present research work is presented as; to carry out the kinematic analyses of different configurations of industrial robot manipulators. The mathematical modelling of selected robot manipulator using existing tools and techniques has to be made for the comparative study of proposed method. On the other hand, development of new algorithm and their mathematical modelling for the solution of inverse kinematic problem has to be made for the analysis of quality and efficiency of the obtained solutions. Therefore, the study of appropriate tools and techniques used for the solution of inverse kinematic problems and comparison with proposed method is considered. Moreover, recommendation of the appropriate method for the solution of inverse kinematic problem is presented in the work. Apart from the forward kinematic analysis, the inverse kinematic analysis is quite complex, due to its non-linear formulations and having multiple solutions. There is no unique solution for the inverse kinematics thus necessitating application of appropriate predictive models from the soft computing domain. Artificial neural network (ANN) can be gainfully used to yield the desired results. Therefore, in the present work several models of artificial neural network (ANN) are used for the solution of the inverse kinematic problem. This model of ANN does not rely on higher mathematical formulations and are adept to solve NP-hard, non-linear and higher degree of polynomial equations. Although intelligent approaches are not new in this field but some selected models of ANN and their hybridization has been presented for the comparative evaluation of inverse kinematic. The hybridization scheme of ANN and an investigation has been made on accuracies of adopted algorithms. On the other hand, any Optimization algorithms which are capable of solving various multimodal functions can be implemented to solve the inverse kinematic problem. To overcome the problem of conventional tool and intelligent based method the optimization based approach can be implemented. In general, the optimization based approaches are more stable and often converge to the global solution. The major problem of ANN based approaches are its slow convergence and often stuck in local optimum point. Therefore, in present work different optimization based approaches are considered. The formulation of the objective function and associated constrained are discussed thoroughly. The comparison of all adopted algorithms on the basis of number of solutions, mathematical operations and computational time has been presented. The thesis concludes the summary with contributions and scope of the future research work

Trajectory Generation for a Multibody Robotic System: Modern Methods Based on Product of Exponentials

This work presents several trajectory generation algorithms for multibody robotic systems based on the Product of Exponentials (PoE) formulation, also known as screw theory. A PoE formulation is first developed to model the kinematics and dynamics of a multibody robotic manipulator (Sawyer Robot) with 7 revolute joints and an end-effector. In the first method, an Inverse Kinematics (IK) algorithm based on the Newton-Raphson iterative method is applied to generate constrained joint-space trajectories corresponding to straight-line and curvilinear motions of the end effector in Cartesian space with finite jerk. The second approach describes Constant Screw Axis (CSA) trajectories which are generated using Machine Learning (ML) and Artificial Neural Networks (ANNs) techniques. The CSA method smooths the trajectory in the Special Euclidean (SE(3)) space. In the third approach, a multi-objective Swarm Intelligence (SI) trajectory generation algorithm is developed, where the IK problem is tackled using a combined SI-PoE ML technique resulting in a joint trajectory that avoids obstacles in the workspace, and satisfies the finite jerk constraint on end-effector while minimizing the torque profiles. The final method is a different approach to solving the IK problem using the Deep Q-Learning (DQN) Reinforcement Learning (RL) algorithm which can generate different joint space trajectories given the Cartesian end-effector path. For all methods above, the Newton-Euler recursive algorithm is implemented to compute the inverse dynamics, which generates the joint torques profiles. The simulated torque profiles are experimentally validated by feeding the generated joint trajectories to the Sawyer robotic arm through the developed Robot Operating System (ROS) - Python environment in the Software Development Kit (SDK) mode. The developed algorithms can be used to generate various trajectories for robotic arms (e.g. spacecraft servicing missions)

Global – local population memetic algorithm for solving the forward kinematics of parallel manipulators

Memetic algorithms (MA) are evolutionary computation methods that employ local search to selected individuals of the population. This work presents global–local population MA for solving the forward kinematics of parallel manipulators. A real-coded generation algorithm with features of diversity is used in the global population and an evolutionary algorithm with parent-centric crossover operator which has local search features is used in the local population. The forward kinematics of the 3RPR and 6–6 leg manipulators are examined to test the performance of the proposed method. The results show that the proposed method improves the performance of the real-coded genetic algorithm and can obtain high-quality solutions similar to the previous methods for the 6–6 leg manipulator. The accuracy of the solutions and the optimisation time achieved by the methods in this work motivates for real-time implementation of the 3RPR parallel manipulator

Advances in Spacecraft Attitude Control

Spacecraft attitude maneuvers comply with Euler's moment equations, a set of three nonlinear, coupled differential equations. Nonlinearities complicate the mathematical treatment of the seemingly simple action of rotating, and these complications lead to a robust lineage of research. This book is meant for basic scientifically inclined readers, and commences with a chapter on the basics of spaceflight and leverages this remediation to reveal very advanced topics to new spaceflight enthusiasts. The topics learned from reading this text will prepare students and faculties to investigate interesting spaceflight problems in an era where cube satellites have made such investigations attainable by even small universities. It is the fondest hope of the editor and authors that readers enjoy this book

Advances in Spacecraft Attitude Control

Spacecraft attitude maneuvers comply with Euler's moment equations, a set of three nonlinear, coupled differential equations. Nonlinearities complicate the mathematical treatment of the seemingly simple action of rotating, and these complications lead to a robust lineage of research. This book is meant for basic scientifically inclined readers, and commences with a chapter on the basics of spaceflight and leverages this remediation to reveal very advanced topics to new spaceflight enthusiasts. The topics learned from reading this text will prepare students and faculties to investigate interesting spaceflight problems in an era where cube satellites have made such investigations attainable by even small universities. It is the fondest hope of the editor and authors that readers enjoy this book

Modeling, Control and Estimation of Reconfigurable Cable Driven Parallel Robots

The motivation for this thesis was to develop a cable-driven parallel robot (CDPR) as part of a two-part robotic device for concrete 3D printing. This research addresses specific research questions in this domain, chiefly, to present advantages offered by the addition of kinematic redundancies to CDPRs. Due to the natural actuation redundancy present in a fully constrained CDPR, the addition of internal mobility offers complex challenges in modeling and control that are not often encountered in literature. This work presents a systematic analysis of modeling such kinematic redundancies through the application of reciprocal screw theory (RST) and Lie algebra while further introducing specific challenges and drawbacks presented by cable driven actuators. It further re-contextualizes well-known performance indices such as manipulability, wrench closure quality, and the available wrench set for application with reconfigurable CDPRs. The existence of both internal redundancy and static redundancy in the joint space offers a large subspace of valid solutions that can be condensed through the selection of appropriate objective priorities, constraints or cost functions. Traditional approaches to such redundancy resolution necessitate computationally expensive numerical optimization. The control of both kinematic and actuation redundancies requires cascaded control frameworks that cannot easily be applied towards real-time control. The selected cost functions for numerical optimization of rCDPRs can be globally (and sometimes locally) non-convex. In this work we present two applied examples of redundancy resolution control that are unique to rCDPRs. In the first example, we maximize the directional wrench ability at the end-effector while minimizing the joint torque requirement by utilizing the fitness of the available wrench set as a constraint over wrench feasibility. The second example focuses on directional stiffness maximization at the end-effector through a variable stiffness module (VSM) that partially decouples the tension and stiffness. The VSM introduces an additional degrees of freedom to the system in order to manipulate both reconfigurability and cable stiffness independently. The controllers in the above examples were designed with kinematic models, but most CDPRs are highly dynamic systems which can require challenging feedback control frameworks. An approach to real-time dynamic control was implemented in this thesis by incorporating a learning-based frameworks through deep reinforcement learning. Three approaches to rCDPR training were attempted utilizing model-free TD3 networks. Robustness and safety are critical features for robot development. One of the main causes of robot failure in CDPRs is due to cable breakage. This not only causes dangerous dynamic oscillations in the workspace, but also leads to total robot failure if the controllability (due to lack of cables) is lost. Fortunately, rCDPRs can be utilized towards failure tolerant control for task recovery. The kinematically redundant joints can be utilized to help recover the lost degrees of freedom due to cable failure. This work applies a Multi-Model Adaptive Estimation (MMAE) framework to enable online and automatic objective reprioritization and actuator retasking. The likelihood of cable failure(s) from the estimator informs the mixing of the control inputs from a bank of feedforward controllers. In traditional rigid body robots, safety procedures generally involve a standard emergency stop procedure such as actuator locking. Due to the flexibility of cable links, the dynamic oscillations of the end-effector due to cable failure must be actively dampened. This work incorporates a Linear Quadratic Regulator (LQR) based feedback stabilizer into the failure tolerant control framework that works to stabilize the non-linear system and dampen out these oscillations. This research contributes to a growing, but hitherto niche body of work in reconfigurable cable driven parallel manipulators. Some outcomes of the multiple engineering design, control and estimation challenges addressed in this research warrant further exploration and study that are beyond the scope of this thesis. This thesis concludes with a thorough discussion of the advantages and limitations of the presented work and avenues for further research that may be of interest to continuing scholars in the community