Using a 3DOF Parallel Robot and a Spherical Bat to hit a Ping-Pong Ball

Playing the game of Ping-Pong is a challenge to human abilities since it requires developing skills, such as fast reaction capabilities, precision of movement and high speed mental responses. These processes include the utilization of seven DOF of the human arm, and translational movements through the legs, torso, and other extremities of the body, which are used for developing different game strategies or simply imposing movements that affect the ball such as spinning movements. Computationally, Ping-Pong requires a huge quantity of joints and visual information to be processed and analysed, something which really represents a challenge for a robot. In addition, in order for a robot to develop the task mechanically, it requires a large and dexterous workspace, and good dynamic capacities. Although there are commercial robots that are able to play Ping-Pong, the game is still an open task, where there are problems to be solved and simplified. All robotic Ping-Pong players cited in the bibliography used at least four DOF to hit the ball. In this paper, a spherical bat mounted on a 3-DOF parallel robot is proposed. The spherical bat is used to drive the trajectory of a Ping-Pong ball.Fil: Trasloheros, Alberto. Universidad Aeronáutica de Querétaro; MéxicoFil: Sebastián, José María. Universidad Politécnica de Madrid; España. Consejo Superior de Investigaciones Científicas; EspañaFil: Torrijos, Jesús. Consejo Superior de Investigaciones Científicas; España. Universidad Politécnica de Madrid; EspañaFil: Carelli Albarracin, Ricardo Oscar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan. Instituto de Automática. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Automática; ArgentinaFil: Roberti, Flavio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan. Instituto de Automática. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Automática; Argentin

Spin observation and trajectory prediction of a ping-pong ball

© 2014 IEEE. For ping-pong playing robots, observing a ball and predicting a ball's trajectory accurately in real-time is essential. However, most existing vision systems can only provide ball's position observation, and do not take into consideration the spin of the ball, which is very important in competitions. This paper proposes a way to observe and estimate ball's spin in real-time, and achieve an accurate prediction. Based on the fact that a spinning ball's motion can be separated into global movement and spinning respect to its center, we construct an integrated vision system to observe the two motions separately. With a pan-tilt vision system, the spinning motion is observed through recognizing the position of the brand on the ball and restoring the 3D pose of the ball. Then the spin state is estimated with the method of plane fitting on current and historical observations. With both position and spin information, accurate state estimation and trajectory prediction are realized via Extended Kalman Filter(EKF). Experimental results show the effectiveness and accuracy of the proposed method

Probabilistic movement modeling for intention inference in human-robot interaction.

Intention inference can be an essential step toward efficient humanrobot interaction. For this purpose, we propose the Intention-Driven Dynamics Model (IDDM) to probabilistically model the generative process of movements that are directed by the intention. The IDDM allows to infer the intention from observed movements using Bayes ’ theorem. The IDDM simultaneously finds a latent state representation of noisy and highdimensional observations, and models the intention-driven dynamics in the latent states. As most robotics applications are subject to real-time constraints, we develop an efficient online algorithm that allows for real-time intention inference. Two human-robot interaction scenarios, i.e., target prediction for robot table tennis and action recognition for interactive humanoid robots, are used to evaluate the performance of our inference algorithm. In both intention inference tasks, the proposed algorithm achieves substantial improvements over support vector machines and Gaussian processes.

Vision application of human robot interaction: Development of a ping pong playing robotic arm

Robotics is a science that is implemented parallel to human behavior. This work describes and implements techniques to mathematically model the game of ping pong played by the humans, and utilization of these methods in the design and development of a ping pong playing robotic arm as an application of robotic vision. Displaced frame difference (DFD) is used to segment the ball motion from background motion and parametric calibration of single CCD camera is utilized to track the ball in three dimensions. This visual information is temporally updated and further applied to guide a robot arm to hit the ball at a specified location in time. The results signify the system development based on single camera tracking and also demonstrate its working with self-sufficiency for the color of the ball. System latency is measured as a function of the camera interface, processor architecture, and robot motion. Various hardware and software parameters that influence the real time system performance are also discussed

Nonprehensile Dynamic Manipulation: A Survey

Nonprehensile dynamic manipulation can be reason- ably considered as the most complex manipulation task. It might be argued that such a task is still rather far from being fully solved and applied in robotics. This survey tries to collect the results reached so far by the research community about planning and control in the nonprehensile dynamic manipulation domain. A discussion about current open issues is addressed as well

Adaptive Robot Systems in Highly Dynamic Environments: A Table Tennis Robot

Hintergrund: Tischtennis bietet ideale Bedingungen, um Kamera-basierte Roboterarme am Limit zu testen. Die besondere Herausforderung liegt in der hohen Geschwindigkeit des Spiels und in der großen Varianz von Spin und Tempo jedes einzelnen Schlages. Die bisherige Forschung mit Tischtennisrobotern beschränkt sich jedoch auf einfache Szenarien, d.h. auf langsame Bälle mit einer geringen Rotation. Forschungsziel: Es soll ein lernfähiger Tischtennisroboter entwickelt werden, der mit dem Spin menschlicher Gegner umgehen kann. Methoden: Das vorgestellte Robotersystem besteht aus sechs Komponenten: Ballpositionserkennung, Ballspinerkennung, Balltrajektorienvorhersage, Schlagparameterbestimmung, Robotertrajektorienplanung und Robotersteuerung. Zuerst wird der Ball mit traditioneller Bildverarbeitung in den Kamerabildern lokalisiert. Mit iterativer Triangulation wird dann seine 3D-Position berechnet. Aus der Kurve der Ballpositionen wird die aktuelle Position und Geschwindigkeit des Balles ermittelt. Für die Spinerkennung werden drei Methoden präsentiert: Die ersten beiden verfolgen die Bewegung des aufgedruckten Ball-Logos auf hochauflösenden Bildern durch Computer Vision bzw. Convolutional Neural Networks. Im dritten Ansatz wird die Flugbahn des Balls unter Berücksichtigung der Magnus-Kraft analysiert. Anhand der Position, der Geschwindigkeit und des Spins des Balls wird die zukünftige Flugbahn berechnet. Dafür wird die physikalische Diffenzialgleichung mit Gravitationskraft, Luftwiderstandskraft und Magnus-Kraft schrittweise gelöst. Mit dem berechneten Zustand des Balls am Schlagpunkt haben wir einen Reinforcement-Learning-Algorithmus trainiert, der bestimmt, mit welchen Schlagparametern der Ball zu treffen ist. Eine passende Robotertrajektorie wird von der Reflexxes-Bibliothek generiert. %Der Roboter wird dann mit einer Frequenz von 250 Hz angesteuert. Ergebnisse: In der quantitativen Auswertung erzielen die einzelnen Komponenten mindestens so gute Ergebnisse wie vergleichbare Tischtennisroboter. Im Hinblick auf das Forschungsziel konnte der Roboter - ein Konterspiel mit einem Menschen führen, mit bis zu 60 Rückschlägen, - unterschiedlichen Spin (Über- und Unterschnitt) retournieren - und mehrere Tischtennisübungen innerhalb von 200 Schlägen erlernen. Schlußfolgerung: Bedeutende algorithmische Neuerungen führen wir in der Spinerkennung und beim Reinforcement Learning von Schlagparametern ein. Dadurch meistert der Roboter anspruchsvollere Spin- und Übungsszenarien als in vergleichbaren Arbeiten.Background: Robotic table tennis systems offer an ideal platform for pushing camera-based robotic manipulation systems to the limit. The unique challenge arises from the fast-paced play and the wide variation in spin and speed between strokes. The range of scenarios under which existing table tennis robots are able to operate is, however, limited, requiring slow play with low rotational velocity of the ball (spin). Research Goal: We aim to develop a table tennis robot system with learning capabilities able to handle spin against a human opponent. Methods: The robot system presented in this thesis consists of six components: ball position detection, ball spin detection, ball trajectory prediction, stroke parameter suggestion, robot trajectory generation, and robot control. For ball detection, the camera images pass through a conventional image processing pipeline. The ball’s 3D positions are determined using iterative triangulation and these are then used to estimate the current ball state (position and velocity). We propose three methods for estimating the spin. The first two methods estimate spin by analyzing the movement of the logo printed on the ball on high-resolution images using either conventional computer vision or convolutional neural networks. The final approach involves analyzing the trajectory of the ball using Magnus force fitting. Once the ball’s position, velocity, and spin are known, the future trajectory is predicted by forward-solving a physical ball model involving gravitational, drag, and Magnus forces. With the predicted ball state at hitting time as state input, we train a reinforcement learning algorithm to suggest the racket state at hitting time (stroke parameter). We use the Reflexxes library to generate a robot trajectory to achieve the suggested racket state. Results: Quantitative evaluation showed that all system components achieve results as good as or better than comparable robots. Regarding the research goal of this thesis, the robot was able to - maintain stable counter-hitting rallies of up to 60 balls with a human player, - return balls with different spin types (topspin and backspin) in the same rally, - learn multiple table tennis drills in just 200 strokes or fewer. Conclusion: Our spin detection system and reinforcement learning-based stroke parameter suggestion introduce significant algorithmic novelties. In contrast to previous work, our robot succeeds in more difficult spin scenarios and drills

Design et développement d’un quadrirotor joueur de tennis de table avec des hélices inclinables

RÉSUMÉ Les bras manipulateurs joueurs de tennis de table les plus avancés sont chers et leur configuration requière beaucoup d’espace. On pourrait considérer l’utilisation de robots aériens pour exécuter cette tâche, mais la plupart des avions à décollage et atterrissage verticales (ADAV) ne sont pas suffisamment rapides pour reproduire des mouvements de frappes. L’objectif de la recherche présentée dans ce mémoire est de développer un nouveau type de robot aérien joueur de tennis de table. Un prototype de quadrirotor ayant des hélices inclinables est d’abord considéré pour permettre de suivre des trajectoires agressives. Le robot a besoin d’atteindre des vitesses de 3.5 m/s au point d’impact et de rester suffisamment léger pour être agile. Ensuite, pour obtenir de hautes performances sur ce requis, un contrôleur Itérative Linéaire Quadratique Régulateur (iLQR) qui suit des trajectoires ayant un snap minimum est implémenté. Le contrôle de la boucle interne est délégué à un microcontrôleur PX4 pour le tangage, le lacet et le roulis pour assurer une bonne robustesse et une haute fréquence. Cette approche est testée dans une simulation réaliste et ensuite le framework complet pour cette application est développer sur un ordinateur embarqué. Des résultats expérimentaux ont été obtenus avec des caméras de capture de mouvements donnant la position et le temps d’impact. Cette information est envoyée au quadrirotor par communication sans-fil et la trajectoire est exécutée immédiatement. Au meilleur de nos connaissances, il s’agit du premier robot aérien étant capable de retourner des balles de tennis de table lancées par un humain. Un taux de succès de 40% est obtenu sur les frappes avec le modèle réel du quadrirotor, significativement supérieur à ce qui était possible d’atteindre auparavant avec un quadrirotor.----------ABSTRACT State-of-art table tennis robot manipulators are expensive and their setup require a lot of space. One could consider using aerial robots for this task, but most vertical takeoff and landing (VTOL) vehicles are not fast enough to reproduce hitting motions. The objective of the research presented in this thesis is to develop a novel type of aerial robot tennis table player. A prototype of a quadrotor that uses tilting propellers is first considered to enable the possibility of aggressive trajectory tracking. The system needs to reach speeds up to 3.5 m/s at the position of impact and to remain light enough to be agile. Next, to obtain high performances for this requirement, an Iterative Quadratic Linear Controller (iLQR) method that follows minimum snap planned trajectories is implemented. Inner-loop control is delegated to a PX4 microcontroller for roll, pitch and yaw to ensure good robustness and high frequency. This approach is tested in a realistic simulation and then the complete software for this task is developed on an onboard computer. Experimental results have been conducted with a motion capture system to have the full state estimate of the system. The trajectory of the ball is also estimated by the motion capture system, giving the position and time of impact. This information is then sent to quadrotor wirelessly and the trajectory is executed immediately. To the best of our knowledge, this is the first aerial robot capable to return tennis table balls thrown by a human. Hitting rates of 40% are achieved with the real quadrotor, significantly better than what was possible before for a quadrotor

Robotic Table Tennis: A Case Study into a High Speed Learning System

We present a deep-dive into a real-world robotic learning system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets. This system puts together a highly optimized perception subsystem, a high-speed low-latency robot controller, a simulation paradigm that can prevent damage in the real world and also train policies for zero-shot transfer, and automated real world environment resets that enable autonomous training and evaluation on physical robots. We complement a complete system description, including numerous design decisions that are typically not widely disseminated, with a collection of studies that clarify the importance of mitigating various sources of latency, accounting for training and deployment distribution shifts, robustness of the perception system, sensitivity to policy hyper-parameters, and choice of action space. A video demonstrating the components of the system and details of experimental results can be found at https://youtu.be/uFcnWjB42I0.Comment: Published and presented at Robotics: Science and Systems (RSS2023