51 research outputs found
Incremental Learning for Robot Perception through HRI
Scene understanding and object recognition is a difficult to achieve yet
crucial skill for robots. Recently, Convolutional Neural Networks (CNN), have
shown success in this task. However, there is still a gap between their
performance on image datasets and real-world robotics scenarios. We present a
novel paradigm for incrementally improving a robot's visual perception through
active human interaction. In this paradigm, the user introduces novel objects
to the robot by means of pointing and voice commands. Given this information,
the robot visually explores the object and adds images from it to re-train the
perception module. Our base perception module is based on recent development in
object detection and recognition using deep learning. Our method leverages
state of the art CNNs from off-line batch learning, human guidance, robot
exploration and incremental on-line learning
Bridging Low-level Geometry to High-level Concepts in Visual Servoing of Robot Manipulation Task Using Event Knowledge Graphs and Vision-Language Models
In this paper, we propose a framework of building knowledgeable robot control
in the scope of smart human-robot interaction, by empowering a basic
uncalibrated visual servoing controller with contextual knowledge through the
joint usage of event knowledge graphs (EKGs) and large-scale pretrained
vision-language models (VLMs). The framework is expanded in twofold: first, we
interpret low-level image geometry as high-level concepts, allowing us to
prompt VLMs and to select geometric features of points and lines for motor
control skills; then, we create an event knowledge graph (EKG) to conceptualize
a robot manipulation task of interest, where the main body of the EKG is
characterized by an executable behavior tree, and the leaves by semantic
concepts relevant to the manipulation context. We demonstrate, in an
uncalibrated environment with real robot trials, that our method lowers the
reliance of human annotation during task interfacing, allows the robot to
perform activities of daily living more easily by treating low-level
geometric-based motor control skills as high-level concepts, and is beneficial
in building cognitive thinking for smart robot applications
Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach
We present a robot eye-hand coordination learning method that can directly
learn visual task specification by watching human demonstrations. Task
specification is represented as a task function, which is learned using inverse
reinforcement learning(IRL) by inferring differential rewards between state
changes. The learned task function is then used as continuous feedbacks in an
uncalibrated visual servoing(UVS) controller designed for the execution phase.
Our proposed method can directly learn from raw videos, which removes the need
for hand-engineered task specification. It can also provide task
interpretability by directly approximating the task function. Besides,
benefiting from the use of a traditional UVS controller, our training process
is efficient and the learned policy is independent from a particular robot
platform. Various experiments were designed to show that, for a certain DOF
task, our method can adapt to task/environment variances in target positions,
backgrounds, illuminations, and occlusions without prior retraining.Comment: Accepted in ICRA 201
- …