Autonomous vision-guided bi-manual grasping and manipulation

This paper describes the implementation, demonstration and evaluation of a variety of autonomous, vision-guided manipulation capabilities, using a dual-arm Baxter robot. Initially, symmetric coordinated bi-manual manipulation based on kinematic tracking algorithm was implemented on the robot to enable a master-slave manipulation system. We demonstrate the efficacy of this approach with a human-robot collaboration experiment, where a human operator moves the master arm along arbitrary trajectories and the slave arm automatically follows the master arm while maintaining a constant relative pose between the two end-effectors. Next, this concept was extended to perform dual-arm manipulation without human intervention. To this extent, an image-based visual servoing scheme has been developed to control the motion of arms for positioning them at a desired grasp locations. Next we combine this with a dynamic position controller to move the grasped object using both arms in a prescribed trajectory. The presented approach has been validated by performing numerous symmetric and asymmetric bi-manual manipulations at different conditions. Our experiments demonstrated 80% success rate in performing the symmetric dual-arm manipulation tasks; and 73% success rate in performing asymmetric dualarm manipulation tasks

A methodology for the Lower Limb Robotic Rehabilitation system

The overall goal of this thesis is to develop a new functional lower limb robot-assisted rehabilitation system for people with a paretic lower limb. A unilateral rehabilitation method is investigated, where the robot acts as an assistive device to provide the impaired leg therapeutic training through simulating the kinematics and dynamics of the ankle and lower leg movements. Foot trajectories of healthy subjects and post-stroke patients were recorded by a dedicated optical motion tracking system in a clinical gait measurement laboratory. A prototype 6 degrees of freedom parallel robot was initially built in order to verify capability of achieving singularity-free foot trajectories of healthy subjects in various exercises. This was then followed by building and testing another larger parallel robot to investigate the real-sized foot trajectories of patients. The overall results verify the designed robot’s capability in successfully tracking foot trajectories during different exercises. The thesis finally proposes a system of bilateral rehabilitation based on the concept of self-learning, where a passive parallel mechanism follows and records motion signatures of the patient’s healthy leg, and an active parallel mechanism provides motion for the impaired leg based on the kinematic mapping of the motion produced by the passive mechanism

Free singularity path planning of hybrid parallel robot

This paper presents a singularity-free path planning approach for a hybrid parallel robot. The hybrid robot is composed of two well-known parallel robots, a hexapod and a tripod, that are serially connected. In this paper a methodology is developed to avoid singularity configurations of the hybrid parallel robot. Nominal polynomial paths are used for motion of end effector, and the strokes of each actuator is calculated by using the developed inverse kinematic. A MATLAB program has been developed to generate the designed paths, and several poses have been tested in a CAD model of the hybrid parallel robot to validate the feasibility of the path planning approach

Learning robotic milling strategies based on passive variable operational space interaction control

This paper addresses the problem of robotic cutting during disassembly of products for materials separation and recycling. Waste handling applications differ from milling in manufacturing processes, as they engender considerable variety and uncertainty in the parameters (e.g. hardness) of materials which the robot must cut. To address this challenge, we propose a learning-based approach incorporating elements of interaction control, in which the robot can adapt key parameters, such as feed rate, depth of cut, and mechanical compliance during task execution. We show how a mathematical model of cutting mechanics, embedded in a simulation environment, can be used to rapidly train the system without needing large amounts of data from physical cutting trials. The simulation approach was validated on a real robot setup based on four case study materials with varying structural and mechanical properties. We demonstrate the proposed method minimises process force and path deviations to a level similar to offline optimal planning methods, while the average time to complete a cutting task is within 25% of the optimum, at the expense of reduced volume of material removed per pass. A key advantage of our approach over similar works is that no prior knowledge about the material is required.Comment: 15 pages, 14 figures, accepted for publication in IEEE Transactions on Automation Science and Engineering (T-ASE

Semi-Autonomous Behaviour Tree-Based Framework for Sorting Electric Vehicle Batteries Components

The process of recycling electric vehicle (EV) batteries currently represents a significant challenge to the waste management automation industry. One example of it is the necessity of removing and sorting dismantled components from EV battery pack. This paper proposes a novel framework to semi-automate the process of removing and sorting different objects from an EV battery pack using a mobile manipulator. The work exploits the Behaviour Trees model for cognitive task execution and monitoring, which links different robot capabilities such as navigation, object tracking and motion planning in a modular fashion. The framework was tested in simulation, in both static and dynamic environments, and it was evaluated based on task time and the number of objects that the robot successfully placed in the respective containers. Results suggested that the robot’s success rate in accomplishing the task of sorting the battery components was 95% and 82% in static and dynamic environments, respectively

Lower Limb Rehabilitation Using Patient Data

The aim of this study is to investigate the performance of a 6-DoF parallel robot in tracking the movement of the foot trajectory of a paretic leg during a single stride. The foot trajectories of nine patients with a paretic leg including both males and females have been measured and analysed by a Vicon system in a gait laboratory. Based on kinematic and dynamic analysis of a 6-DoF UPS parallel robot, an algorithm was developed in MATLAB to calculate the length of the actuators and their required forces during all trajectories. The workspace and singularity points of the robot were then investigated in nine different cases. A 6-DoF UPS parallel robot prototype with high repeatability was designed and built in order to simulate a single stride. Results showed that the robot was capable of tracking all of the trajectories with the maximum position error of 1.2 mm

Semi-autonomous Robotic Disassembly Enhanced by Mixed Reality

In this study, we introduce "SARDiM," a modular semi-autonomous platform enhanced with mixed reality for industrial disassembly tasks. Through a case study focused on EV battery disassembly, SARDiM integrates Mixed Reality, object segmentation, teleoperation, force feedback, and variable autonomy. Utilising the ROS, Unity, and MATLAB platforms, alongside a joint impedance controller, SARDiM facilitates teleoperated disassembly. The approach combines FastSAM for real-time object segmentation, generating data which is subsequently processed through a cluster analysis algorithm to determine the centroid and orientation of the components, categorizing them by size and disassembly priority. This data guides the MoveIt platform in trajectory planning for the Franka Robot arm. SARDiM provides the capability to switch between two teleoperation modes: manual and semi-autonomous with variable autonomy. Each was evaluated using four different Interface Methods (IM): direct view, monitor feed, mixed reality with monitor feed, and point cloud mixed reality. Evaluations across the eight IMs demonstrated a 40.61% decrease in joint limit violations using Mode 2. Moreover, Mode 2-IM4 outperformed Mode 1-IM1 by achieving a 2.33%-time reduction while considerably increasing safety, making it optimal for operating in hazardous environments at a safe distance, with the same ease of use as teleoperation with a direct view of the environment

An online hyper‐volume action bounding approach for accelerating the process of deep reinforcement learning from multiple controllers

This paper fuses ideas from reinforcement learning (RL), Learning from Demonstration (LfD), and Ensemble Learning into a single paradigm. Knowledge from a mixture of control algorithms (experts) are used to constrain the action space of the agent, enabling faster RL refining of a control policy, by avoiding unnecessary explorative actions. Domain‐specific knowledge of each expert is exploited. However, the resulting policy is robust against errors of individual experts, since it is refined by a RL reward function without copying any particular demonstration. Our method has the potential to supplement existing RLfD methods when multiple algorithmic approaches are available to function as experts, specifically in tasks involving continuous action spaces. We illustrate our method in the context of a visual servoing (VS) task, in which a 7‐DoF robot arm is controlled to maintain a desired pose relative to a target object. We explore four methods for bounding the actions of the RL agent during training. These methods include using a hypercube and convex hull with modified loss functions, ignoring actions outside the convex hull, and projecting actions onto the convex hull. We compare the training progress of each method using expert demonstrators, employing one expert demonstrator with the DAgger algorithm, and without using any demonstrators. Our experiments show that using the convex hull with a modified loss function not only accelerates learning but also provides the most optimal solution compared with other approaches. Furthermore, we demonstrate faster VS error convergence while maintaining higher manipulability of the arm, compared with classical image‐based VS, position‐based VS, and hybrid‐decoupled VS