Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty (RSS 2016 Best Student Paper Award)

Coordinated control strategies for multi-robot systems are necessary for tasks that cannot be executed by a single robot. This encompasses tasks where the workspace of the robot is too small or where the load is too heavy for one robot to handle. Using multiple robots makes the task feasible by extending the workspace and/or increase the payload of the overall robotic system. In this paper, we consider two instances of such task: a co-worker scenario in which a human hands over a large object to a robot; intercepting a large flying object. The problem is made difficult as the pick-up/intercept motions must take place while the object is in motion and because the object's motion is not deterministic. The challenge is then to adapt the motion of the robotic arms in coordination with one another and with the object. Determining the pick-up/intercept point is done by taking into account the workspace of the multi-arm system and is continuously recomputed to adapt to change in the object's trajectory. We propose a dynamical systems (DS) based control law to generate autonomous and synchronized motions for a multi-arm robot system in the task of reaching for a moving object. We show theoretically that the resulting DS coordinates the motion of the robots with each other and with the object, while the system remains stable. We validate our approach on a dual-arm robotic system and demonstrate that it can re-synchronize and adapt the motion of each arm in synchrony in a fraction of seconds, even when the motion of the object is fast and not accurately predictable

Tracking and Grasping of Moving Objects Using Aerial Robotic Manipulators: A Brief Survey

Unmanned Aerial Vehicles (UAV) has evolved in recent years, their features have changed to be more useful to the society, although some years ago the drones had been thought to be teleoperated by humans and to take some pictures from above, which is useful; nevertheless, nowadays the drones are capable of developing autonomous tasks like tracking a dynamic target or even grasping different kind of objects. Some task like transporting heavy loads or manipulating complex shapes are more challenging for a single UAV, but for a fleet of them might be easier. This brief survey presents a compilation of relevant works related to tracking and grasping with aerial robotic manipulators, as well as cooperation among them. Moreover, challenges and limitations are presented in order to contribute with new areas of research. Finally, some trends in aerial manipulation are foreseeing for different sectors and relevant features for these kind of systems are standing out

Aspects of an open architecture robot controller and its integration with a stereo vision sensor.

The work presented in this thesis attempts to improve the performance of industrial robot systems in a flexible manufacturing environment by addressing a number of issues related to external sensory feedback and sensor integration, robot kinematic positioning accuracy, and robot dynamic control performance. To provide a powerful control algorithm environment and the support for external sensor integration, a transputer based open architecture robot controller is developed. It features high computational power, user accessibility at various robot control levels and external sensor integration capability. Additionally, an on-line trajectory adaptation scheme is devised and implemented in the open architecture robot controller, enabling a real-time trajectory alteration of robot motion to be achieved in response to external sensory feedback. An in depth discussion is presented on integrating a stereo vision sensor with the robot controller to perform external sensor guided robot operations. Key issues for such a vision based robot system are precise synchronisation between the vision system and the robot controller, and correct target position prediction to counteract the inherent time delay in image processing. These were successfully addressed in a demonstrator system based on a Puma robot. Efforts have also been made to improve the Puma robot kinematic and dynamic performance. A simple, effective, on-line algorithm is developed for solving the inverse kinematics problem of a calibrated industrial robot to improve robot positioning accuracy. On the dynamic control aspect, a robust adaptive robot tracking control algorithm is derived that has an improved performance compared to a conventional PID controller as well as exhibiting relatively modest computational complexity. Experiments have been carried out to validate the open architecture robot controller and demonstrate the performance of the inverse kinematics algorithm, the adaptive servo control algorithm, and the on-line trajectory generation. By integrating the open architecture robot controller with a stereo vision sensor system, robot visual guidance has been achieved with experimental results showing that the integrated system is capable of detecting, tracking and intercepting random objects moving in 3D trajectory at a velocity up to 40mm/s

Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty

Sina Mirrazavi Salehian S, Figueroa N, Billard A. Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty. In: Proceedings of Robotics: Science and Systems. AnnArbor, Michigan; 2016

Compliant control of Uni/ Multi- robotic arms with dynamical systems

Accomplishment of many interactive tasks hinges on the compliance of humans. Humans demonstrate an impressive capability of complying their behavior and more particularly their motions with the environment in everyday life. In humans, compliance emerges from different facets. For example, many daily activities involve reaching for grabbing tasks, where compliance appears in a form of coordination. Humans comply their handsâ motions with each other and with that of the object not only to establish a stable contact and to control the impact force but also to overcome sensorimotor imprecisions. Even though compliance has been studied from different aspects in humans, it is primarily related to impedance control in robotics. In this thesis, we leverage the properties of autonomous dynamical systems (DS) for immediate re-planning and introduce active complaint motion generators for controlling robots in three different scenarios, where compliance does not necessarily mean impedance and hence it is not directly related to control in the force/velocity domain. In the first part of the thesis, we propose an active compliant strategy for catching objects in flight, which is less sensitive to the timely control of the interception. The soft catching strategy consists in having the robot following the object for a short period of time. This leaves more time for the fingers to close on the object at the interception and offers more robustness than a âhardâ catching method in which the hand waits for the object at the chosen interception point. We show theoretically that the resulting DS will intercept the object at the intercept point, at the right time with the desired velocity direction. Stability and convergence of the approach are assessed through Lyapunov stability theory. In the second part, we propose a unified compliant control architecture for coordinately reaching for grabbing a moving object by a multi-arm robotic system. Due to the complexity of the task and of the system, each arm complies not only with the objectâs motion but also with the motion of other arms, in both task and joint spaces. At the task-space level, we propose a unified dynamical system that endows the multi-arm system with both synchronous and asynchronous behaviors and with the capability of smoothly transitioning between the two modes. At the joint space level, the compliance between the arms is achieved by introducing a centralized inverse kinematics (IK) solver under self-collision avoidance constraints; formulated as a quadratic programming problem (QP) and solved in real-time. In the last part, we propose a compliant dynamical system for stably transitioning from free motions to contacts. In this part, by modulating the robot's velocity in three regions, we show theoretically and empirically that the robot can (I) stably touch the contact surface (II) at a desired location, and (III) leave the surface or stop on the surface at a desired point

Computer Vision Methods for Autonomous Remote Sizing in Manufacturing

In the grand scheme of Industry 4.0, the employment of modern intelligent digital technology has been utilised to facilitate industrial production, leveraging automation to elevate production efficiency. Building upon this, Industry 5.0 takes a step forward, accentuating the concept of human-machine symbiosis. It directs its focus on augmenting human performance within the industry, mitigating errors made by workers, and honing the overarching performance of human-machine systems. Across various manufacturing domains, an escalating demand for this level of automation has been noticed. One such area is the speciality steel industry, whose tasks are the primary consideration of this dissertation. Speciality steel rolling forms the backbone of industrial sectors as diverse as aerospace and oil and gas. The key to the sustained survival of steel plants hinges on the digitalisation of the rolling process. Despite this, a significant number of steel rolling plants in the present day continue to place a heavy reliance on human operators to oversee and regulate the manufacturing process. With a view to securing the safety of workers in high-risk factory environments and optimising the control of steel production, this dissertation puts forth machine vision approaches. These are aimed at supervising the direction of hot steel sections and remotely gauging their dimensions, both conducted in real-time. This dissertation further contributes a novel image registration approach founded on extrinsic features. This approach is then amalgamated with frequency domain image fusion of optical images. The resultant fused image is designated to evaluate the size of high-quality hot steel sections from a remote standpoint. With the integration of the remote imaging sizing module, operators can stay abreast of the section dimensions in real time. Concurrently, the mill stands can be pre-adjusted to facilitate quality assurance. The efficacy of the developed approaches has been tested over real data, delivering an accuracy rate exceeding 95%. This suggests that the approach not only ensures worker safety but also contributes significantly to the enhancement of production control and efficiency in the speciality steel industry

Proceedings of the European Conference on Agricultural Engineering AgEng2021

This proceedings book results from the AgEng2021 Agricultural Engineering Conference under auspices of the European Society of Agricultural Engineers, held in an online format based on the University of Évora, Portugal, from 4 to 8 July 2021. This book contains the full papers of a selection of abstracts that were the base for the oral presentations and posters presented at the conference. Presentations were distributed in eleven thematic areas: Artificial Intelligence, data processing and management; Automation, robotics and sensor technology; Circular Economy; Education and Rural development; Energy and bioenergy; Integrated and sustainable Farming systems; New application technologies and mechanisation; Post-harvest technologies; Smart farming / Precision agriculture; Soil, land and water engineering; Sustainable production in Farm buildings

Guideline for Trustworthy Artificial Intelligence -- AI Assessment Catalog

Artificial Intelligence (AI) has made impressive progress in recent years and represents a key technology that has a crucial impact on the economy and society. However, it is clear that AI and business models based on it can only reach their full potential if AI applications are developed according to high quality standards and are effectively protected against new AI risks. For instance, AI bears the risk of unfair treatment of individuals when processing personal data e.g., to support credit lending or staff recruitment decisions. The emergence of these new risks is closely linked to the fact that the behavior of AI applications, particularly those based on Machine Learning (ML), is essentially learned from large volumes of data and is not predetermined by fixed programmed rules. Thus, the issue of the trustworthiness of AI applications is crucial and is the subject of numerous major publications by stakeholders in politics, business and society. In addition, there is mutual agreement that the requirements for trustworthy AI, which are often described in an abstract way, must now be made clear and tangible. One challenge to overcome here relates to the fact that the specific quality criteria for an AI application depend heavily on the application context and possible measures to fulfill them in turn depend heavily on the AI technology used. Lastly, practical assessment procedures are needed to evaluate whether specific AI applications have been developed according to adequate quality standards. This AI assessment catalog addresses exactly this point and is intended for two target groups: Firstly, it provides developers with a guideline for systematically making their AI applications trustworthy. Secondly, it guides assessors and auditors on how to examine AI applications for trustworthiness in a structured way

Conference on Intelligent Robotics in Field, Factory, Service, and Space (CIRFFSS 1994), volume 1

The AIAA/NASA Conference on Intelligent Robotics in Field, Factory, Service, and Space (CIRFFSS '94) was originally proposed because of the strong belief that America's problems of global economic competitiveness and job creation and preservation can partly be solved by the use of intelligent robotics, which are also required for human space exploration missions. Individual sessions addressed nuclear industry, agile manufacturing, security/building monitoring, on-orbit applications, vision and sensing technologies, situated control and low-level control, robotic systems architecture, environmental restoration and waste management, robotic remanufacturing, and healthcare applications

Autonomous Visual Learning for Robotic Systems

This thesis investigates the problem of visual learning using a robotic platform. Given a set of objects the robots task is to autonomously manipulate, observe, and learn. This allows the robot to recognise objects in a novel scene and pose, or separate them into distinct visual categories. The main focus of the work is in autonomously acquiring object models using robotic manipulation. Autonomous learning is important for robotic systems. In the context of vision, it allows a robot to adapt to new and uncertain environments, updating its internal model of the world. It also reduces the amount of human supervision needed for building visual models. This leads to machines which can operate in environments with rich and complicated visual information, such as the home or industrial workspace; also, in environments which are potentially hazardous for humans. The hypothesis claims that inducing robot motion on objects aids the learning process. It is shown that extra information from the robot sensors provides enough information to localise an object and distinguish it from the background. Also, that decisive planning allows the object to be separated and observed from a variety of dierent poses, giving a good foundation to build a robust classication model. Contributions include a new segmentation algorithm, a new classication model for object learning, and a method for allowing a robot to supervise its own learning in cluttered and dynamic environments.EThOS - Electronic Theses Online ServiceGBUnited Kingdo