Visual Servoing

The goal of this book is to introduce the visional application by excellent researchers in the world currently and offer the knowledge that can also be applied to another field widely. This book collects the main studies about machine vision currently in the world, and has a powerful persuasion in the applications employed in the machine vision. The contents, which demonstrate that the machine vision theory, are realized in different field. For the beginner, it is easy to understand the development in the vision servoing. For engineer, professor and researcher, they can study and learn the chapters, and then employ another application method

Agent and object aware tracking and mapping methods for mobile manipulators

The age of the intelligent machine is upon us. They exist in our factories, our warehouses, our military, our hospitals, on our roads, and on the moon. Most of these things we call robots. When placed in a controlled or known environment such as an automotive factory or a distribution warehouse they perform their given roles with exceptional efficiency, achieving far more than is within reach of a humble human being. Despite the remarkable success of intelligent machines in such domains, they have yet to make a full-hearted deployment into our homes. The missing link between the robots we have now and the robots that are soon to come to our houses is perception. Perception as we mean it here refers to a level of understanding beyond the collection and aggregation of sensory data. Much of the available sensory information is noisy and unreliable, our homes contain many reflective surfaces, repeating textures on large flat surfaces, and many disruptive moving elements, including humans. These environments change over time, with objects frequently moving within and between rooms. This idea of change in an environment is fundamental to robotic applications, as in most cases we expect them to be effectors of such change. We can identify two particular challenges1 that must be solved for robots to make the jump to less structured environments - how to manage noise and disruptive elements in observational data, and how to understand the world as a set of changeable elements (objects) which move over time within a wider environment. In this thesis we look at one possible approach to solving each of these problems. For the first challenge we use proprioception aboard a robot with an articulated arm to handle difficult and unreliable visual data caused both by the robot and the environment. We use sensor data aboard the robot to improve the pose tracking of a visual system when the robot moves rapidly, with high jerk, or when observing a scene with little visual variation. For the second challenge, we build a model of the world on the level of rigid objects, and relocalise them both as they change location between different sequences and as they move. We use semantics, image keypoints, and 3D geometry to register and align objects between sequences, showing how their position has moved between disparate observations.Open Acces

Machine Learning and System Identification for Estimation in Physical Systems

In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear modeling power while keeping data requirements practical.The thesis covers a wide range of applications, many inspired by applications within robotics, but also extending outside this already wide field.Usage of the proposed methods and algorithms are in many cases illustrated in the real-world applications that motivated the research.Topics covered include dynamics modeling and estimation, model-based reinforcement learning, spectral estimation, friction modeling and state estimation and calibration in robotic machining.In the work on modeling and identification of dynamics, we develop regularization strategies that allow us to incorporate prior domain knowledge into flexible, overparameterized models. We make use of classical control theory to gain insight into training and regularization while using tools from modern deep learning. A particular focus of the work is to allow use of modern methods in scenarios where gathering data is associated with a high cost.In the robotics-inspired parts of the thesis, we develop methods that are practically motivated and make sure that they are implementable also outside the research setting. We demonstrate this by performing experiments in realistic settings and providing open-source implementations of all proposed methods and algorithms

Contact force and torque estimation for collaborative manipulators based on an adaptive Kalman filter with variable time period.

Contact force and torque sensing approaches enable manipulators to cooperate with humans and to interact appropriately with unexpected collisions. In this thesis, various moving averages are investigated and Weighted Moving Averages and Hull Moving Average are employed to generate a mode-switching moving average to support force sensing. The proposed moving averages with variable time period were used to reduce the effects of measured motor current noise and thus provide improved confidence in joint output torque estimation. The time period of the filter adapts continuously to achieve an optimal trade-off between response time and precision of estimation in real-time. An adaptive Kalman filter that consists of the proposed moving averages and the conventional Kalman filter is proposed. Calibration routines for the adaptive Kalman filter interpret the measured motor current noise and errors in the speed data from the individual joints into. The combination of the proposed adaptive Kalman filter with variable time period and its calibration method facilitates force and torque estimation without direct measurement via force/torque sensors. Contact force/torque sensing and response time assessments from the proposed approach are performed on both the single Universal Robot 5 manipulator and the collaborative UR5 arrangement (dual-arm robot) with differing unexpected end effector loads. The combined force and torque sensing method leads to a reduction of the estimation errors and response time in comparison with the pioneering method (55.2% and 20.8 %, respectively), and the positive performance of the proposed approach is further improved as the payload rises. The proposed method can potentially be applied to any robotic manipulators as long as the motor information (current, joint position, and joint velocities) are available. Consequently the cost of implementation will be significantly lower than methods that require load cells

Design, modelling and control of a brachiating power line inspection robot

The inspection of power lines and associated hardware is vital to ensuring the reliability of the transmission and distribution network. The repetitive nature of the inspection tasks present a unique opportunity for the introduction of robotic platforms, which offer the ability to perform more systematic and detailed inspection than traditional methods. This lends itself to improved asset management automation, cost-effectiveness and safety for the operating crew. This dissertation presents the development of a prototype industrial brachiating robot. The robot is mechanically simple and capable of dynamically negotiating obstacles by brachiating. This is an improvement over current robotic platforms, which employ slow, high power static schemes for obstacle negotiation. Mathematical models of the robot were derived to understand the underlying dynamics of the system. These models were then used in the generation of optimal trajectories, using nonlinear optimisation techniques, for brachiating past line hardware. A physical robot was designed and manufactured to validate the brachiation manoeuvre. The robot was designed following classic mechanical design principles, with emphasis on functional design and robustness. System identification was used to capture the plant uncertainty and a feedback controller was designed to track the reference trajectory allowing for energy optimal brachiation swings. Finally, the robot was tested, starting with sub-system testing and ending with testing of a brachiation manoeuvre proving the prospective viability of the robot in an industrial environment

Ein mobiler Serviceroboter zur Automatisierung der Probenahme und des Probenmanagements in einem biotechnologischen Pilotlabor

Scherer T. A mobile service robot for automisation of sample taking and sample management in a biotechnological pilot laboratory. Bielefeld (Germany): Bielefeld University; 2004.In biotechnologischen Laboratorien ist die Qualität der typischerweise pharmazeutischen Produkte ein wortwörtlich lebenswichtiges Ziel. Die Qualität der Zellkultivierungen wurde historisch nur durch off-line Messungen von physikalischen Prozessparametern wie pH und pO2 sichergestellt. Biologische Parameter wie die Zelldichte und -viabilität wurden nur off-line gemessen, weil das dazu notwendige Probenmanagement hochkomplizierte Manipulationen und Analysen beinhaltet und deshalb nicht automatisiert werden konnte. Es gibt zwar mehrere automatisierte Geräte, um einem Labortechniker zu assistieren, aber kein System, welches das gesamte Probenmanagement automatisiert. In dieser Arbeit wird ein neuer Typ von Serviceroboter präsentiert, der aus einem auf einer mobilen Plattform montierten Roboterarm besteht und diese Lücke schließt. Dieser Roboter muss eine ganze Reihe von Problemen bewältigen: Er muss seine Position im Labor bestimmen können (Lokalisation), er muss eine kollisionsfreie Bahn zu den beteiligten Geräten finden können (Bahnplanung mit Hindernisvermeidung), er darf bei seinen Bewegungen keine Menschen gefährden oder Laborausrüstung beschädigen (Kollisionsvermeidung), er muss die zu bedienenden Geräte erkennen und ihre Position präzise messen können (Bildverarbeitung), er muss sie bedienen können (Armsteuerung), er muss Objekte greifen können (Greifer und Finger) und er muss sie gefügig handhaben können, um sie nicht zu beschädigen (Kraftregelung). Er muss autonom sein, um nur die allernotwendigste Menge an Benutzereingriffen zu benötigen, und doch durch ein Laborsteuerprogramm kontrollierbar sein, um Eingriffe zu erlauben. Schließlich muss er einfach durch ungeschultes Personal zu warten sein. All diese Aspekte werden von dem in dieser Arbeit präsentierten neuen Robotersystem abgedeckt.In biotechnolgical laboratories, the quality of the typically pharmaceutical product is a literally life-important goal. Historically, the quality of the cell cultivations was ensured by on-line measurements of physical process parameters like pH and pO2 only. Biological parameters like cell density and viability were only measured off-line, because the necessary sample management involves highly complicated manipulations and analyses and could therefore not be automated. Various automated devices to assist a laboratory technician do exist, but so far no system to automate the entire sample management. In this work a novel type of service robot consisting of a robot arm mounted on a mobile platform is presented that closes this gap. This robot has to master a multitude of problems: It must be able to locate its position in the laboratory (localisation), it must be able to find a collision-free path to the involved devices (path planning with obstacle avoidance), it must not endanger humans or damage laboratory equipment while moving (collision avoidance), it must be able to recognize the devices to be manipulated and measure their precise position (computer vision), it must be able to manipulate them (arm control), it must be able to grasp objects (gripper and fingers) and it must be able to handle them with compliance in order to not damage them (force control). It must be autonomous in order to only require the least possible amount of user intervention, and yet controllable by a laboratory control program in order to allow intervention. Finally, it must be easily maintainable by non-expert personell. All these aspects are covered by the novel robot system presented in this thesis

Proceedings of the NASA Conference on Space Telerobotics, volume 3

The theme of the Conference was man-machine collaboration in space. The Conference provided a forum for researchers and engineers to exchange ideas on the research and development required for application of telerobotics technology to the space systems planned for the 1990s and beyond. The Conference: (1) provided a view of current NASA telerobotic research and development; (2) stimulated technical exchange on man-machine systems, manipulator control, machine sensing, machine intelligence, concurrent computation, and system architectures; and (3) identified important unsolved problems of current interest which can be dealt with by future research

Utilizing Compliance To Address Modern Challenges in Robotics

Mechanical compliance will be an essential component for agile robots as they begin to leave the laboratory settings and join our world. The most crucial finding of this dissertation is showing how lessons learned from soft robotics can be adapted into traditional robotics to introduce compliance. Therefore, it presents practical knowledge on how to build soft bodied sensor and actuation modules: first example being soft-bodied curvature sensors. These sensors contain both standard electronic components soldered on flexible PCBs and hyperelastic materials that cover the electronics. They are built by curing multi-material composites inside hyper elastic materials. Then it shows, via precise sensing by using magnets and Hall-effect sensors, how closed-loop control of soft actuation modules can be achieved via proprioceptive feedback. Once curvature sensing idea is verified, the dissertation describes how the same sensing methodology, along with the same multi-material manufacturing technique can be utilized to construct soft bodied tri-axial force sensors. It shows experimentally that these sensors can be used by traditional robotic grippers to increase grasping quality. At this point, I observe that compliance is an important property that robots may utilize for different types of motions. One example being Raibert\u27s 2D hopper mechanism. It uses its leg-spring to store energy while on the ground and release this energy before jumping. I observe that via soft material design, it would be possible to embed compliance directly into the linkage design itself. So I go over the design details of an extremely lightweight compliant five-bar mechanism design that can store energy when compressed via soft ligaments embedded in its joints. I experimentally show that the compliant leg design offers increased efficiency compared to a rigid counterpart. I also utilize the previously mentioned soft bodied force sensors for rapid contact detection (~5-10 Hz) in the hopper test platform. In the end, this thesis connects soft robotics with the traditional body of robotic knowledge in two aspects: a) I show that manufacturing techniques we use for soft bodied sensor/actuator designs can be utilized for creating soft ligaments that add strength and compliance to robot joints; and b) I demonstrate that soft bodied force sensing techniques can be used reliably for robotic contact detection

The Development of a Multi-arm Mobile Robot System for Nuclear Decommissioning Applications.

This PhD thesis is based in the field of robotics and introduces a case study of the design and development of a multi-arm mobile robot system for nuclear decommissioning (MARS-ND). A key premise underlying the research was to develop intelligence in the robot that is similar to the cooperation and communication between the human brain and its two arms; hence the human body was adopted as the starting point to establish the size and functionality of the proposed system. The approach adopted for this research demonstrates the development, integration and configuration of a multi-arm robot system which consists of two human armlike off-the-shelf manipulators whose joints are controlled using potentiometer sensors and hydraulic actuators. Using the manipulators' sensor feedback, a wide variety of complex tasks found in the rapidly expanding field of nuclear decommissioning can be undertaken. The thesis also considers the issue of collaboration, collision detection and collision avoidance between the two arms of MARS-ND. As part of the final stage of this research the author participated in a collaborative research project with the Sugano Laboratory at Waseda University, Tokyo, Japan. The three major research issues addressed in this thesis are: 1. The selection and integration of off-the-shelf hardware in the development of MARS-ND using the latest technology available for robotic systems 2. The creation of a suitable control system for the robot arms; and the building of an advanced, user-friendly interface between the robot system and the host computer 3. The investigation and implementation of collaboration, coordinated motion control and collision detection & avoidance techniques for the robot arms The hardware and software integration for the whole robotic system is explained with the proposed software architecture and the use of National Instruments (NI) functions and tools to control the movement of the arm joints and the performance of a selected decommissioning task. This thesis also examines the operational software applied within the research through its discussion of four interlinked areas: 1. The control software and hardware interface for the MARS-ND and the controller architecture 2. The application of an NI Compact FieldPoint controller and FieldPoint I/O modules to facilitate wireless communication between the Multi-Arm Mobile Robot system and the user interface in the host PC 3. The use of Measurement and Automation Explorer (MAX) and LabVIEW software tools for calibration and the building of user interfaces required for sending and receiving the signals needed to control the robot arm joints accurately 4. The application of a PID toolkit in LabVIEW for the design of a simple PID controller for the individual arm joints with a potentiometer sensor fitted inside each joint in order to provide a feedback signal to the controller The thesis concludes that MARS-ND is a good example of a robotic system specifically designed for hazardous nuclear decommissioning applications. It demonstrates the complexity of such a system from a number of aspects such as the need for mobility, control, sensor and system design, and integration using modem tools that are available off-the-shelf. In addition the use of these modern tools allows a single mechatronics engineer to design, integrate, interface and build a motion control system for MARS-ND as compared to the traditional way of building a similar robot by a team of specialised engineers. The contribution this research makes to the design and building of multi-arm robot system for nuclear decommissioning industry concerns its size and mobility using a mobile platform to transport the multi-arm robot system. In addition links have been made between Lancaster University and Waseda University in the context of the development of multi-arm robot systems

Force sensing enhancement of robot system

At present there is a general industrial need to improve robot performance. Force feedback, which involves sensing and actuation, is one means of improving the relative position between the workpiece and the end-effector. In this research work various causes of errors and poor robot performance are identified. Several methods of improving the performance of robotic systems are discussed. As a result of this research, a system was developed which is interposed between the wrist and the gripper of the manipulator. This system integrates a force sensor with a micro-manipulator, via an electronic control unit, with a micro-computer to enhance a robot system. The force sensor, the micromanipulator and the electronic control unit, were all designed and manufactured at the robotic centre of Middlesex Polytechnic. The force feedback is provided by means of strain gauges and the associated bridge circuitry. Control algorithms which define the relationship between the force detected and the motion required are implemented in the software. The software is capable of performing two specific tasks in real time, these are: 1- Inserting a peg into a hole 2- Following an unknown geometric path A rig was designed and manufactured to enable the robot to follow different geometric shapes and paths in which force control was achieved mainly by control of the micro-manipulator