605 research outputs found
Extension of the Control Concept for a Mobile Overhead Manipulator to Whole-Body Impedance Control
At present, robots constitute a central component of contemporary factories. The application of traditional ground-based systems, however, may lead to congested floors with minimal space left for new robots or human workers. Overhead manipulators, on the other hand, aim to occupy the unutilized ceiling space, in order to manipulate the workspace located below them. The SwarmRail system is an example of such an overhead manipulator. This concept deploys mobile units driving across a passive railstructure above the ground. Additionally, equipping the mobile units with robotic arms at their bottom side enables this design to provide continuous overhead manipulation while in motion. Although a first demonstrator confirmed the functional capability of said system, the current hardware suffers from complications while traversing rail crossings. Due to uneven rails consecutive rails, said crossing points cause the robot's wheels to collide with the new rail segment it is driving towards. Additionally, the robot experiences an undesired sudden altitude change.
In this thesis, we aim to implement a hierarchical whole-body impedance tracking controller for the robots employed within the SwarmRail system. Our controller combines a kinematically controlled mobile unit with the impedance-based control of a robotic arm through an admittance interface. The focus of this thesis is set on the controller's robustness against the previously mentioned external disturbances. The performance of this controller is validated inside a simulation that incorporates the aforementioned complications. Our findings suggest, that the control strategy presented in this thesis provides a foundation for the development of a controller applicable to the physical demonstrator
ABC: Adaptive, Biomimetic, Configurable Robots for Smart Farms - From Cereal Phenotyping to Soft Fruit Harvesting
Currently, numerous factors, such as demographics, migration patterns, and economics, are leading to the critical labour shortage in low-skilled and physically demanding parts of agriculture. Thus, robotics can be developed for the agricultural sector to address these shortages. This study aims to develop an adaptive, biomimetic, and configurable modular robotics architecture that can be applied to multiple tasks (e.g., phenotyping, cutting, and picking), various crop varieties (e.g., wheat, strawberry, and tomato) and growing conditions. These robotic solutions cover the entire perception–action–decision-making loop targeting the phenotyping of cereals and harvesting fruits in a natural environment.
The primary contributions of this thesis are as follows. a) A high-throughput method for imaging field-grown wheat in three dimensions, along with an accompanying unsupervised measuring method for obtaining individual wheat spike data are presented. The unsupervised method analyses the 3D point cloud of each trial plot, containing hundreds of wheat spikes, and calculates the average size of the wheat spike and total spike volume per plot. Experimental results reveal that the proposed algorithm can effectively identify spikes from wheat crops and individual spikes. b) Unlike cereal, soft fruit is typically harvested by manual selection and picking. To enable robotic harvesting, the initial perception system uses conditional generative adversarial networks to identify ripe fruits using synthetic data. To determine whether the strawberry is surrounded by obstacles, a cluster complexity-based perception system is further developed to classify the harvesting complexity of ripe strawberries. c) Once the harvest-ready fruit is localised using point cloud data generated by a stereo camera, the platform’s action system can coordinate the arm to reach/cut the stem using the passive motion paradigm framework, as inspired by studies on neural control of movement in the brain. Results from field trials for strawberry detection, reaching/cutting the stem of the fruit with a mean error of less than 3 mm, and extension to analysing complex canopy structures/bimanual coordination (searching/picking) are presented.
Although this thesis focuses on strawberry harvesting, ongoing research is heading toward adapting the architecture to other crops. The agricultural food industry remains a labour-intensive sector with a low margin, and cost- and time-efficiency business model. The concepts presented herein can serve as a reference for future agricultural robots that are adaptive, biomimetic, and configurable
Machine Learning Meets Advanced Robotic Manipulation
Automated industries lead to high quality production, lower manufacturing
cost and better utilization of human resources. Robotic manipulator arms have
major role in the automation process. However, for complex manipulation tasks,
hard coding efficient and safe trajectories is challenging and time consuming.
Machine learning methods have the potential to learn such controllers based on
expert demonstrations. Despite promising advances, better approaches must be
developed to improve safety, reliability, and efficiency of ML methods in both
training and deployment phases. This survey aims to review cutting edge
technologies and recent trends on ML methods applied to real-world manipulation
tasks. After reviewing the related background on ML, the rest of the paper is
devoted to ML applications in different domains such as industry, healthcare,
agriculture, space, military, and search and rescue. The paper is closed with
important research directions for future works
NICOL: A Neuro-inspired Collaborative Semi-humanoid Robot that Bridges Social Interaction and Reliable Manipulation
Robotic platforms that can efficiently collaborate with humans in physical
tasks constitute a major goal in robotics. However, many existing robotic
platforms are either designed for social interaction or industrial object
manipulation tasks. The design of collaborative robots seldom emphasizes both
their social interaction and physical collaboration abilities. To bridge this
gap, we present the novel semi-humanoid NICOL, the Neuro-Inspired COLlaborator.
NICOL is a large, newly designed, scaled-up version of its well-evaluated
predecessor, the Neuro-Inspired COmpanion (NICO). NICOL adopts NICO's head and
facial expression display and extends its manipulation abilities in terms of
precision, object size, and workspace size. Our contribution in this paper is
twofold -- firstly, we introduce the design concept for NICOL, and secondly, we
provide an evaluation of NICOL's manipulation abilities by presenting a novel
extension for an end-to-end hybrid neuro-genetic visuomotor learning approach
adapted to NICOL's more complex kinematics. We show that the approach
outperforms the state-of-the-art Inverse Kinematics (IK) solvers KDL, TRACK-IK
and BIO-IK. Overall, this article presents for the first time the humanoid
robot NICOL, and contributes to the integration of social robotics and neural
visuomotor learning for humanoid robots
GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators
Imitation learning from human demonstrations is a powerful framework to teach
robots new skills. However, the performance of the learned policies is
bottlenecked by the quality, scale, and variety of the demonstration data. In
this paper, we aim to lower the barrier to collecting large and high-quality
human demonstration data by proposing GELLO, a general framework for building
low-cost and intuitive teleoperation systems for robotic manipulation. Given a
target robot arm, we build a GELLO controller that has the same kinematic
structure as the target arm, leveraging 3D-printed parts and off-the-shelf
motors. GELLO is easy to build and intuitive to use. Through an extensive user
study, we show that GELLO enables more reliable and efficient demonstration
collection compared to commonly used teleoperation devices in the imitation
learning literature such as VR controllers and 3D spacemouses. We further
demonstrate the capabilities of GELLO for performing complex bi-manual and
contact-rich manipulation tasks. To make GELLO accessible to everyone, we have
designed and built GELLO systems for 3 commonly used robotic arms: Franka, UR5,
and xArm. All software and hardware are open-sourced and can be found on our
website: https://wuphilipp.github.io/gello/
Recommended from our members
On the Creation and Use of Forward Models in Robot Motor Control
Advancements in robotics have the potential to aid humans in many realms of exploration as well as daily life: from search and rescue work, to space and deep sea exploration, to in-home assistance to improve the quality of life for those with limited mobility. One of the main milestones that needs to be met for robotics to achieve these ends is a robust ability to manipulate objects and locomote in cluttered and changing environments. A prerequisite to these skills is the ability to understand the current state of the world as well as how actions result in changes to the environment; in short, a robot needs a way to model itself and the world around it. With recent advances in machine learning and access to cheap and fast computation, one of the most promising avenues for creating robust models is to learn a neural network to approximate the dynamics of the system.
Learning a data-driven model that accurately replicates the dynamics of a robot and its environment is an active area of robotics research. This model needs to be accurate, it needs to operate using sensors that are often high dimensional, and it needs to be robust to changes within the system and the surrounding environment. In this thesis, we investigate ways to improve the processof learning data-driven dynamics models as well as ways to reduce the dimensionality of a robot’s state space.
We start by trying to improve the long-term accuracy of neural network based forward models. Learning forward models is more complicated than it appears on the surface. While it is easy to learn a model to predict the change of a system over a short horizon, it is challenging to assure this performance over a long horizon. We investigate the concept of adding temporal information into the loss function of the forward model during training; we demonstrate that this improves the accuracy of a model when it is used to predict over long horizons.
While we are currently working with low dimensional systems, we eventually want to apply our learned models to robots with high dimensional state spaces. To make learning feasible, we need to find ways to learn a lower dimensional representation of the state space (also known as a latent space) to make learning models in the real world computationally feasible. We present a method to improve the usefulness of a learned latent space using a method we call context training: we learn a latent space alongside a forward model to encourage the learned latent space to retain the variables critical to learning the dynamics of the system.
In all of our experiments, we spend significant time in analysis and evaluation. A large portion of literature demonstrating the effectiveness of data-driven forward models in robot control settings often only presents the final controller performance. We were often left curious about what the model was learning independent of the control scenario. We set out to do our own deep dive into exactly what data-driven forward models are predicting. We evaluate all of our models over long horizons. We also look deeper than just the mean and median loss values. We plot the full distribution of loss values over the entire horizon. The literature on data-driven models that do evaluate model prediction accuracy often focuses on the mean and median prediction errors; while these are important metrics, we found that looking at these metrics alone can sometimes obscure subtle but important effects. A high mean loss is often a result of poor performance on only a subset of the test dataset; one model can outperform other models with lower mean error values on a majority of the test set, but it can be skewed to look like the worst performer by having a few highly inaccurate outliers.
We observe that models often have a subset of a test dataset on which they perform best; we seek to limit the use of a model to regions of the test dataset where it has high accuracy by using an ensemble of models. We find that if we train an ensemble of forward models, the accuracy of the models is higher when they all agree on a prediction. Conversely, when the ensemble of models disagrees, the prediction is often poor. We explore this relationship and propose future ways to apply it.
Finally, we look into the application of improved model accuracy and context trained latent spaces. We start by testing the performance of our context training architecture as a method to reduce the state space dimensionality in a model-free reinforcement learning (MFRL) reaching task. We hypothesize that a policy trained with a latent space observation derived using our context trained encoder will outperform a policy trained with a latent space observation derived from a standard autoencoder. Unfortunately, we found no difference in task performance between the policies learned using either method. We end on a bright note by looking at the power of model-based control when we have access to an accurate model. We successfully use model predictive control (MPC) to generate robust locomotion for a simulated snake robot. With access to an accurate model, we are able to generate realistic snake gaits in a variety of environments with very little parameter tuning that are robust to changes in the environment
Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning
Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons
Path and Motion Planning for Autonomous Mobile 3D Printing
Autonomous robotic construction was envisioned as early as the ‘90s, and yet, con-
struction sites today look much alike ones half a century ago. Meanwhile, highly
automated and efficient fabrication methods like Additive Manufacturing, or 3D
Printing, have seen great success in conventional production. However, existing
efforts to transfer printing technology to construction applications mainly rely on
manufacturing-like machines and fail to utilise the capabilities of modern robotics.
This thesis considers using Mobile Manipulator robots to perform large-scale
Additive Manufacturing tasks. Comprised of an articulated arm and a mobile base,
Mobile Manipulators, are unique in their simultaneous mobility and agility, which
enables printing-in-motion, or Mobile 3D Printing. This is a 3D printing modality,
where a robot deposits material along larger-than-self trajectories while in motion.
Despite profound potential advantages over existing static manufacturing-like large-
scale printers, Mobile 3D printing is underexplored. Therefore, this thesis tack-
les Mobile 3D printing-specific challenges and proposes path and motion planning
methodologies that allow this printing modality to be realised. The work details
the development of Task-Consistent Path Planning that solves the problem of find-
ing a valid robot-base path needed to print larger-than-self trajectories. A motion
planning and control strategy is then proposed, utilising the robot-base paths found
to inform an optimisation-based whole-body motion controller. Several Mobile 3D
Printing robot prototypes are built throughout this work, and the overall path and
motion planning strategy proposed is holistically evaluated in a series of large-scale
3D printing experiments
Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions
Perceiving and manipulating 3D articulated objects in diverse environments is
essential for home-assistant robots. Recent studies have shown that point-level
affordance provides actionable priors for downstream manipulation tasks.
However, existing works primarily focus on single-object scenarios with
homogeneous agents, overlooking the realistic constraints imposed by the
environment and the agent's morphology, e.g., occlusions and physical
limitations. In this paper, we propose an environment-aware affordance
framework that incorporates both object-level actionable priors and environment
constraints. Unlike object-centric affordance approaches, learning
environment-aware affordance faces the challenge of combinatorial explosion due
to the complexity of various occlusions, characterized by their quantities,
geometries, positions and poses. To address this and enhance data efficiency,
we introduce a novel contrastive affordance learning framework capable of
training on scenes containing a single occluder and generalizing to scenes with
complex occluder combinations. Experiments demonstrate the effectiveness of our
proposed approach in learning affordance considering environment constraints.
Project page at https://chengkaiacademycity.github.io/EnvAwareAfford/Comment: In 37th Conference on Neural Information Processing Systems (NeurIPS
2023). Website at https://chengkaiacademycity.github.io/EnvAwareAfford
- …