2,041 research outputs found
Reliable Real-Time Ball Tracking for Robot Table Tennis
Robot table tennis systems require a vision system that can track the ball position with
low latency and high sampling rate. Altering the ball to simplify the tracking using, for instance,
infrared coating changes the physics of the ball trajectory. As a result, table tennis systems use custom
tracking systems to track the ball based on heuristic algorithms respecting the real-time constrains
applied to RGB images captured with a set of cameras. However, these heuristic algorithms often
report erroneous ball positions, and the table tennis policies typically need to incorporate additional
heuristics to detect and possibly correct outliers. In this paper, we propose a vision system for
object detection and tracking that focuses on reliability while providing real-time performance.
Our assumption is that by using multiple cameras, we can find and discard the errors obtained in
the object detection phase by checking for consistency with the positions reported by other cameras.
We provide an open source implementation of the proposed tracking system to simplify future
research in robot table tennis or related tracking applications with strong real-time requirements.
We evaluate the proposed system thoroughly in simulation and in the real system, outperforming
previous work. Furthermore, we show that the accuracy and robustness of the proposed system
increases as more cameras are added. Finally, we evaluate the table tennis playing performance of an
existing method in the real robot using the proposed vision system. We measure a slight increase in
performance compared to a previous vision system even after removing all the heuristics previously
present to filter out erroneous ball observations
Tracking a table tennis ball for umpiring purposes
This study investigates tracking a table-tennis ball rapidly from video captured using low-cost equipment for umpiring purposes. A number of highly efficient algorithms have been developed for this purpose. The proposed system was tested using sequences capture from real match scenes. The preliminary results of experiments show that accurate and rapid tracking can be achieved even under challenging conditions, including occlusion and colour merging. This work can contribute to the development of an automatic umpiring system and also has the potential to provide amateur users open access to a detection tool for fast-moving, small, round objects
Probabilistic movement modeling for intention inference in human-robot interaction.
Intention inference can be an essential step toward efficient humanrobot interaction. For this purpose, we propose the Intention-Driven Dynamics Model (IDDM) to probabilistically model the generative process of movements that are directed by the intention. The IDDM allows to infer the intention from observed movements using Bayes ’ theorem. The IDDM simultaneously finds a latent state representation of noisy and highdimensional observations, and models the intention-driven dynamics in the latent states. As most robotics applications are subject to real-time constraints, we develop an efficient online algorithm that allows for real-time intention inference. Two human-robot interaction scenarios, i.e., target prediction for robot table tennis and action recognition for interactive humanoid robots, are used to evaluate the performance of our inference algorithm. In both intention inference tasks, the proposed algorithm achieves substantial improvements over support vector machines and Gaussian processes.
Robotic Table Tennis: A Case Study into a High Speed Learning System
We present a deep-dive into a real-world robotic learning system that, in
previous work, was shown to be capable of hundreds of table tennis rallies with
a human and has the ability to precisely return the ball to desired targets.
This system puts together a highly optimized perception subsystem, a high-speed
low-latency robot controller, a simulation paradigm that can prevent damage in
the real world and also train policies for zero-shot transfer, and automated
real world environment resets that enable autonomous training and evaluation on
physical robots. We complement a complete system description, including
numerous design decisions that are typically not widely disseminated, with a
collection of studies that clarify the importance of mitigating various sources
of latency, accounting for training and deployment distribution shifts,
robustness of the perception system, sensitivity to policy hyper-parameters,
and choice of action space. A video demonstrating the components of the system
and details of experimental results can be found at
https://youtu.be/uFcnWjB42I0.Comment: Published and presented at Robotics: Science and Systems (RSS2023
A multi-modal table tennis robot system
In recent years, robotic table tennis has become a popular research challenge
for perception and robot control. Here, we present an improved table tennis
robot system with high accuracy vision detection and fast robot reaction. Based
on previous work, our system contains a KUKA robot arm with 6 DOF, with four
frame-based cameras and two additional event-based cameras. We developed a
novel calibration approach to calibrate this multimodal perception system. For
table tennis, spin estimation is crucial. Therefore, we introduced a novel, and
more accurate spin estimation approach. Finally, we show how combining the
output of an event-based camera and a Spiking Neural Network (SNN) can be used
for accurate ball detection.Comment: Accepted for RoboLetics: Workshop on Robot Learning in Athletics
@CoRL 202
Robot Composite Learning and the Nunchaku Flipping Challenge
Advanced motor skills are essential for robots to physically coexist with
humans. Much research on robot dynamics and control has achieved success on
hyper robot motor capabilities, but mostly through heavily case-specific
engineering. Meanwhile, in terms of robot acquiring skills in a ubiquitous
manner, robot learning from human demonstration (LfD) has achieved great
progress, but still has limitations handling dynamic skills and compound
actions. In this paper, we present a composite learning scheme which goes
beyond LfD and integrates robot learning from human definition, demonstration,
and evaluation. The method tackles advanced motor skills that require dynamic
time-critical maneuver, complex contact control, and handling partly soft
partly rigid objects. We also introduce the "nunchaku flipping challenge", an
extreme test that puts hard requirements to all these three aspects. Continued
from our previous presentations, this paper introduces the latest update of the
composite learning scheme and the physical success of the nunchaku flipping
challenge
Recommended from our members
A Novel Multi-View Table Tennis Umpiring Framework
This research investigates the development of a low-cost multi-view umpiring framework, as an alternative to the current expensive systems that are almost exclusively restricted to elite professional sports. Table tennis has been selected as the testbed because, while automating the process is challenging, it has many different complex match elements including the service, return and rallies, which are governed by a strict set of regulations. The focus is mainly on the rally element rather than the whole match. Ball detection and tracking in video frames are undertaken to determine reliably the ball position relative to key reference objects like the table surface and net, and the ball’s flight path is used to determine the rally’s status.
While a low-cost option has benefits, it is technically challenging due to the limited number of cameras and generally low video resolution used. This thesis presents a portable multi-view umpiring framework that identifies each state change in a rally. It makes three significant contributions to knowledge: i) a reliable ball detection strategy that accurately detects the location of the ball in low-resolution sequences; ii) a novel framework for ball tracking using a multi-view system, and iii) a new state-machine based evaluation system for analysing table tennis rallies.
In a series of ten different test scenarios, the system achieved an average of 94% system detection rate and 100% accurate decisions. A test sequence of duration 1 s can be processed in 8 s, leading to a delay of only 7 s, which is considered acceptable for practical purposes. This solution has the potential to reform the way matches are umpired, providing objectivity in resolving disputed decisions. It affords an economic technology for amateur players, while the multi-view facility is extendible to other relevant ball-based sports. Finally, the ball flight path analysis mechanism can be a valuable training tool for skills development
Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning
We present a reinforcement learning (RL) framework that enables quadrupedal
robots to perform soccer goalkeeping tasks in the real world. Soccer
goalkeeping using quadrupeds is a challenging problem, that combines highly
dynamic locomotion with precise and fast non-prehensile object (ball)
manipulation. The robot needs to react to and intercept a potentially flying
ball using dynamic locomotion maneuvers in a very short amount of time, usually
less than one second. In this paper, we propose to address this problem using a
hierarchical model-free RL framework. The first component of the framework
contains multiple control policies for distinct locomotion skills, which can be
used to cover different regions of the goal. Each control policy enables the
robot to track random parametric end-effector trajectories while performing one
specific locomotion skill, such as jump, dive, and sidestep. These skills are
then utilized by the second part of the framework which is a high-level planner
to determine a desired skill and end-effector trajectory in order to intercept
a ball flying to different regions of the goal. We deploy the proposed
framework on a Mini Cheetah quadrupedal robot and demonstrate the effectiveness
of our framework for various agile interceptions of a fast-moving ball in the
real world.Comment: First two authors contributed equally. Accompanying video is at
https://youtu.be/iX6OgG67-Z
- …