1,617 research outputs found
Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images
In this paper, we study the challenging problem of predicting the dynamics of
objects in static images. Given a query object in an image, our goal is to
provide a physical understanding of the object in terms of the forces acting
upon it and its long term motion as response to those forces. Direct and
explicit estimation of the forces and the motion of objects from a single image
is extremely challenging. We define intermediate physical abstractions called
Newtonian scenarios and introduce Newtonian Neural Network () that learns
to map a single image to a state in a Newtonian scenario. Our experimental
evaluations show that our method can reliably predict dynamics of a query
object from a single image. In addition, our approach can provide physical
reasoning that supports the predicted dynamics in terms of velocity and force
vectors. To spur research in this direction we compiled Visual Newtonian
Dynamics (VIND) dataset that includes 6806 videos aligned with Newtonian
scenarios represented using game engines, and 4516 still images with their
ground truth dynamics
MonoTrack: Shuttle trajectory reconstruction from monocular badminton video
Trajectory estimation is a fundamental component of racket sport analytics,
as the trajectory contains information not only about the winning and losing of
each point, but also how it was won or lost. In sports such as badminton,
players benefit from knowing the full 3D trajectory, as the height of
shuttlecock or ball provides valuable tactical information. Unfortunately, 3D
reconstruction is a notoriously hard problem, and standard trajectory
estimators can only track 2D pixel coordinates. In this work, we present the
first complete end-to-end system for the extraction and segmentation of 3D
shuttle trajectories from monocular badminton videos. Our system integrates
badminton domain knowledge such as court dimension, shot placement, physical
laws of motion, along with vision-based features such as player poses and
shuttle tracking. We find that significant engineering efforts and model
improvements are needed to make the overall system robust, and as a by-product
of our work, improve state-of-the-art results on court recognition, 2D
trajectory estimation, and hit recognition.Comment: To appear in CVSports@CVPR 202
A technology platform for automatic high-level tennis game analysis
Sports video research is a popular topic that has been applied to many prominent sports for a large spectrum of applications. In this paper we introduce a technology platform which has been developed for the tennis context, able to extract action sequences and provide support to coaches for players performance analysis during training and official matches. The system consists of an hardware architecture, devised to acquire data in the tennis context and for the specific domain requirements, and a number of processing modules which are able to track both the ball and the players, to extract semantic information from their interactions and automatically annotate video sequences. The aim of this paper is to demonstrate that the proposed combination of hardware and software modules is able to extract 3D ball trajectories robust enough to evaluate ball changes of direction recognizing serves, strokes and bounces. Starting from these information, a finite state machine based decision process can be employed to evaluate the score of each action of the game. The entire platform has been tested in real experiments during both training sessions and matches, and results show that automatic annotation of key events along with 3D positions and scores can be used to support coaches in the extraction of valuable information about players intentions and behaviours
Boccia court analysis for promoting elderly physical activity
Physical inactivity is one of the leading risk factors for global mortality. Older adults, in particular, are more probable to suffer the consequences of physical inactivity, since it is one of the most sedentary age groups. On the other hand, engaging physical activity can have various benefits for the prevention of several diseases and functional loss prevention, therefore, it is critical to encourage its regular practice amongst the elderly. Boccia is a simple precision ball sport that is easily adaptable for individuals with physical limitations, which makes it a perfectly good game for this circumstance. The present paper proposes a ball-detection based system for monitoring the Boccia court, compute the current game score and display it on a user interface. The future goal of such system will be to motivate the elders to participate more frequently in the Boccia game and make the overall game experience more enjoyable. The proposed system was tested with twenty video recordings of different simulated game situations. Overall, the obtained results were encouraging, having only one incorrect game score being computed by the developed algorithm.This article is a result of the project Deus ex Machina: NORTE01-0145-FEDER-000026, supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through
the European Regional Development Fund (ERDF)
Towards Learning Monocular 3D Object Localization From 2D Labels using the Physical Laws of Motion
We present a novel method for precise 3D object localization in single images
from a single calibrated camera using only 2D labels. No expensive 3D labels
are needed. Thus, instead of using 3D labels, our model is trained with
easy-to-annotate 2D labels along with the physical knowledge of the object's
motion. Given this information, the model can infer the latent third dimension,
even though it has never seen this information during training. Our method is
evaluated on both synthetic and real-world datasets, and we are able to achieve
a mean distance error of just 6 cm in our experiments on real data. The results
indicate the method's potential as a step towards learning 3D object location
estimation, where collecting 3D data for training is not feasible
A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Deep learning has the potential to revolutionize sports performance, with
applications ranging from perception and comprehension to decision. This paper
presents a comprehensive survey of deep learning in sports performance,
focusing on three main aspects: algorithms, datasets and virtual environments,
and challenges. Firstly, we discuss the hierarchical structure of deep learning
algorithms in sports performance which includes perception, comprehension and
decision while comparing their strengths and weaknesses. Secondly, we list
widely used existing datasets in sports and highlight their characteristics and
limitations. Finally, we summarize current challenges and point out future
trends of deep learning in sports. Our survey provides valuable reference
material for researchers interested in deep learning in sports applications
PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction
Humans interact with objects all the time. Enabling a humanoid to learn
human-object interaction (HOI) is a key step for future smart animation and
intelligent robotics systems. However, recent progress in physics-based HOI
requires carefully designed task-specific rewards, making the system unscalable
and labor-intensive. This work focuses on dynamic HOI imitation: teaching
humanoid dynamic interaction skills through imitating kinematic HOI
demonstrations. It is quite challenging because of the complexity of the
interaction between body parts and objects and the lack of dynamic HOI data. To
handle the above issues, we present PhysHOI, the first physics-based whole-body
HOI imitation approach without task-specific reward designs. Except for the
kinematic HOI representations of humans and objects, we introduce the contact
graph to model the contact relations between body parts and objects explicitly.
A contact graph reward is also designed, which proved to be critical for
precise HOI imitation. Based on the key designs, PhysHOI can imitate diverse
HOI tasks simply yet effectively without prior knowledge. To make up for the
lack of dynamic HOI scenarios in this area, we introduce the BallPlay dataset
that contains eight whole-body basketball skills. We validate PhysHOI on
diverse HOI tasks, including whole-body grasping and basketball skills
- …