511 research outputs found
Direct 3D servoing using dense depth maps
International audienceThis paper proposes a novel 3D servoing approach using dense depth maps to achieve robotic tasks. With respect to position-based approaches, our method does not require the estimation of the 3D pose (direct), nor the extraction and matching of 3D features (dense) and only requires dense depth maps provided by 3D sensors. Our approach has been validated in servoing experiments using the depth information from a low cost RGB-D sensor. Positioning tasks are properly achieved despite the noisy measurements, even when partial occlusions or scene modifications occur
3D Spectral Domain Registration-Based Visual Servoing
This paper presents a spectral domain registration-based visual servoing
scheme that works on 3D point clouds. Specifically, we propose a 3D model/point
cloud alignment method, which works by finding a global transformation between
reference and target point clouds using spectral analysis. A 3D Fast Fourier
Transform (FFT) in R3 is used for the translation estimation, and the real
spherical harmonics in SO(3) are used for the rotations estimation. Such an
approach allows us to derive a decoupled 6 degrees of freedom (DoF) controller,
where we use gradient ascent optimisation to minimise translation and
rotational costs. We then show how this methodology can be used to regulate a
robot arm to perform a positioning task. In contrast to the existing
state-of-the-art depth-based visual servoing methods that either require dense
depth maps or dense point clouds, our method works well with partial point
clouds and can effectively handle larger transformations between the reference
and the target positions. Furthermore, the use of spectral data (instead of
spatial data) for transformation estimation makes our method robust to
sensor-induced noise and partial occlusions. We validate our approach by
performing experiments using point clouds acquired by a robot-mounted depth
camera. Obtained results demonstrate the effectiveness of our visual servoing
approach.Comment: Accepted to 2023 IEEE International Conference on Robotics and
Automation (ICRA'23
Exploring Convolutional Networks for End-to-End Visual Servoing
Present image based visual servoing approaches rely on extracting hand
crafted visual features from an image. Choosing the right set of features is
important as it directly affects the performance of any approach. Motivated by
recent breakthroughs in performance of data driven methods on recognition and
localization tasks, we aim to learn visual feature representations suitable for
servoing tasks in unstructured and unknown environments. In this paper, we
present an end-to-end learning based approach for visual servoing in diverse
scenes where the knowledge of camera parameters and scene geometry is not
available a priori. This is achieved by training a convolutional neural network
over color images with synchronised camera poses. Through experiments performed
in simulation and on a quadrotor, we demonstrate the efficacy and robustness of
our approach for a wide range of camera poses in both indoor as well as outdoor
environments.Comment: IEEE ICRA 201
Robotic Crop Interaction in Agriculture for Soft Fruit Harvesting
Autonomous tree crop harvesting has been a seemingly attainable, but elusive, robotics goal for the past several decades. Limiting grower reliance on uncertain seasonal labour is an economic driver of this, but the ability of robotic systems to treat each plant individually also has environmental benefits, such as reduced emissions and fertiliser use. Over the same time period, effective grasping and manipulation (G&M) solutions to warehouse product handling, and more general robotic interaction, have been demonstrated.
Despite research progress in general robotic interaction and harvesting of some specific crop types, a commercially successful robotic harvester has yet to be demonstrated. Most crop varieties, including soft-skinned fruit, have not yet been addressed. Soft fruit, such as plums, present problems for many of the techniques employed for their more robust relatives and require special focus when developing autonomous harvesters. Adapting existing robotics tools and techniques to new fruit types, including soft skinned varieties, is not well explored. This thesis aims to bridge that gap by examining the challenges of autonomous crop interaction for the harvesting of soft fruit.
Aspects which are known to be challenging include mixed obstacle planning with both hard and soft obstacles present, poor outdoor sensing conditions, and the lack of proven picking motion strategies. Positioning an actuator for harvesting requires solving these problems and others specific to soft skinned fruit. Doing so effectively means addressing these in the sensing, planning and actuation areas of a robotic system. Such areas are also highly interdependent for grasping and manipulation tasks, so solutions need to be developed at the system level.
In this thesis, soft robotics actuators, with simplifying assumptions about hard obstacle planes, are used to solve mixed obstacle planning. Persistent target tracking and filtering is used to overcome challenging object detection conditions, while multiple stages of object detection are applied to refine these initial position estimates. Several picking motions are developed and tested for plums, with varying degrees of effectiveness. These various techniques are integrated into a prototype system which is validated in lab testing and extensive field trials on a commercial plum crop.
Key contributions of this thesis include
I. The examination of grasping & manipulation tools, algorithms, techniques and challenges for harvesting soft skinned fruit
II. Design, development and field-trial evaluation of a harvester prototype to validate these concepts in practice, with specific design studies of the gripper type, object detector architecture and picking motion for this
III. Investigation of specific G&M module improvements including:
o Application of the autocovariance least squares (ALS) method to noise covariance matrix estimation for visual servoing tasks, where both simulated and real experiments demonstrated a 30% improvement in state estimation error using this technique.
o Theory and experimentation showing that a single range measurement is sufficient for disambiguating scene scale in monocular depth estimation for some datasets.
o Preliminary investigations of stochastic object completion and sampling for grasping, active perception for visual servoing based harvesting, and multi-stage fruit localisation from RGB-Depth data.
Several field trials were carried out with the plum harvesting prototype. Testing on an unmodified commercial plum crop, in all weather conditions, showed promising results with a harvest success rate of 42%. While a significant gap between prototype performance and commercial viability remains, the use of soft robotics with carefully chosen sensing and planning approaches allows for robust grasping & manipulation under challenging conditions, with both hard and soft obstacles
Modelling the Xbox 360 Kinect for visual servo control applications
A research report submitted to the faculty of Engineering and the built environment, University of the Witwatersrand, Johannesburg, in partial fulfilment of the requirements for the degree of Master of Science in Engineering.
Johannesburg, August 2016There has been much interest in using the Microsoft Xbox 360 Kinect
cameras for visual servo control applications. It is a relatively cheap
device with expected shortcomings. This work contributes to the practical
considerations of using the Kinect for visual servo control applications.
A comprehensive characterisation of the Kinect is synthesised
from existing literature and results from a nonlinear calibration procedure.
The Kinect reduces computational overhead on image processing
stages, such as pose estimation or depth estimation. It is limited
by its 0.8m to 3.5m practical depth range and quadratic depth resolution
of 1.8mm to 35mm, respectively. Since the Kinect uses an
infra-red (IR) projector, a class one laser, it should not be used outdoors,
due to IR saturation, and objects belonging to classes of non-
IR-friendly surfaces should be avoided, due to IR refraction, absorption,
or specular reflection. Problems of task stability due to invalid
depth measurements in Kinect depth maps and practical depth range
limitations can be reduced by using depth map preprocessing and
activating classical visual servoing techniques when Kinect-based approaches
are near task failure.MT201
Perceptual Context in Cognitive Hierarchies
Cognition does not only depend on bottom-up sensor feature abstraction, but
also relies on contextual information being passed top-down. Context is higher
level information that helps to predict belief states at lower levels. The main
contribution of this paper is to provide a formalisation of perceptual context
and its integration into a new process model for cognitive hierarchies. Several
simple instantiations of a cognitive hierarchy are used to illustrate the role
of context. Notably, we demonstrate the use context in a novel approach to
visually track the pose of rigid objects with just a 2D camera
- …