Long-Term Anticipation of Activities with Cycle Consistency

With the success of deep learning methods in analyzing activities in videos, more attention has recently been focused towards anticipating future activities. However, most of the work on anticipation either analyzes a partially observed activity or predicts the next action class. Recently, new approaches have been proposed to extend the prediction horizon up to several minutes in the future and that anticipate a sequence of future activities including their durations. While these works decouple the semantic interpretation of the observed sequence from the anticipation task, we propose a framework for anticipating future activities directly from the features of the observed frames and train it in an end-to-end fashion. Furthermore, we introduce a cycle consistency loss over time by predicting the past activities given the predicted future. Our framework achieves state-of-the-art results on two datasets: the Breakfast dataset and 50Salads.Comment: GCPR 202

Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports

Reasoning over sports videos for question answering is an important task with numerous applications, such as player training and information retrieval. However, this task has not been explored due to the lack of relevant datasets and the challenging nature it presents. Most datasets for video question answering (VideoQA) focus mainly on general and coarse-grained understanding of daily-life videos, which is not applicable to sports scenarios requiring professional action understanding and fine-grained motion analysis. In this paper, we introduce the first dataset, named Sports-QA, specifically designed for the sports VideoQA task. The Sports-QA dataset includes various types of questions, such as descriptions, chronologies, causalities, and counterfactual conditions, covering multiple sports. Furthermore, to address the characteristics of the sports VideoQA task, we propose a new Auto-Focus Transformer (AFT) capable of automatically focusing on particular scales of temporal information for question answering. We conduct extensive experiments on Sports-QA, including baseline studies and the evaluation of different methods. The results demonstrate that our AFT achieves state-of-the-art performance

Meta-analysis of genome-wide association studies for cattle stature identifies common genes that regulate body size in mammals

peer-reviewedH.D.D., A.J.C., P.J.B. and B.J.H. would like to acknowledge the Dairy Futures Cooperative Research Centre for funding. H.P. and R.F. acknowledge funding from the German Federal Ministry of Education and Research (BMBF) within the AgroClustEr ‘Synbreed—Synergistic Plant and Animal Breeding’ (grant 0315527B). H.P., R.F., R.E. and K.-U.G. acknowledge the Arbeitsgemeinschaft Süddeutscher Rinderzüchter, the Arbeitsgemeinschaft Österreichischer Fleckviehzüchter and ZuchtData EDV Dienstleistungen for providing genotype data. A. Bagnato acknowledges the European Union (EU) Collaborative Project LowInputBreeds (grant agreement 222623) for providing Brown Swiss genotypes. Braunvieh Schweiz is acknowledged for providing Brown Swiss phenotypes. H.P. and R.F. acknowledge the German Holstein Association (DHV) and the Confederación de Asociaciones de Frisona Española (CONCAFE) for sharing genotype data. H.P. was financially supported by a postdoctoral fellowship from the Deutsche Forschungsgemeinschaft (DFG) (grant PA 2789/1-1). D.B. and D.C.P. acknowledge funding from the Research Stimulus Fund (11/S/112) and Science Foundation Ireland (14/IA/2576). M.S. and F.S.S. acknowledge the Canadian Dairy Network (CDN) for providing the Holstein genotypes. P.S. acknowledges funding from the Genome Canada project entitled ‘Whole Genome Selection through Genome Wide Imputation in Beef Cattle’ and acknowledges WestGrid and Compute/Calcul Canada for providing computing resources. J.F.T. was supported by the National Institute of Food and Agriculture, US Department of Agriculture, under awards 2013-68004-20364 and 2015-67015-23183. A. Bagnato, F.P., M.D. and J.W. acknowledge EU Collaborative Project Quantomics (grant 516 agreement 222664) for providing Brown Swiss and Finnish Ayrshire sequences and genotypes. A.C.B. and R.F.V. acknowledge funding from the public–private partnership ‘Breed4Food’ (code BO-22.04-011- 001-ASG-LR) and EU FP7 IRSES SEQSEL (grant 317697). A.C.B. and R.F.V. acknowledge CRV (Arnhem, the Netherlands) for providing data on Dutch and New Zealand Holstein and Jersey bulls.Stature is affected by many polymorphisms of small effect in humans1. In contrast, variation in dogs, even within breeds, has been suggested to be largely due to variants in a small number of genes2,3. Here we use data from cattle to compare the genetic architecture of stature to those in humans and dogs. We conducted a meta-analysis for stature using 58,265 cattle from 17 populations with 25.4 million imputed whole-genome sequence variants. Results showed that the genetic architecture of stature in cattle is similar to that in humans, as the lead variants in 163 significantly associated genomic regions (P < 5 × 10−8) explained at most 13.8% of the phenotypic variance. Most of these variants were noncoding, including variants that were also expression quantitative trait loci (eQTLs) and in ChIP–seq peaks. There was significant overlap in loci for stature with humans and dogs, suggesting that a set of common genes regulates body size in mammals

From Sea to Sea: Canada's Three Oceans of Biodiversity

Evaluating and understanding biodiversity in marine ecosystems are both necessary and challenging for conservation. This paper compiles and summarizes current knowledge of the diversity of marine taxa in Canada's three oceans while recognizing that this compilation is incomplete and will change in the future. That Canada has the longest coastline in the world and incorporates distinctly different biogeographic provinces and ecoregions (e.g., temperate through ice-covered areas) constrains this analysis. The taxonomic groups presented here include microbes, phytoplankton, macroalgae, zooplankton, benthic infauna, fishes, and marine mammals. The minimum number of species or taxa compiled here is 15,988 for the three Canadian oceans. However, this number clearly underestimates in several ways the total number of taxa present. First, there are significant gaps in the published literature. Second, the diversity of many habitats has not been compiled for all taxonomic groups (e.g., intertidal rocky shores, deep sea), and data compilations are based on short-term, directed research programs or longer-term monitoring activities with limited spatial resolution. Third, the biodiversity of large organisms is well known, but this is not true of smaller organisms. Finally, the greatest constraint on this summary is the willingness and capacity of those who collected the data to make it available to those interested in biodiversity meta-analyses. Confirmation of identities and intercomparison of studies are also constrained by the disturbing rate of decline in the number of taxonomists and systematists specializing on marine taxa in Canada. This decline is mostly the result of retirements of current specialists and to a lack of training and employment opportunities for new ones. Considering the difficulties encountered in compiling an overview of biogeographic data and the diversity of species or taxa in Canada's three oceans, this synthesis is intended to serve as a biodiversity baseline for a new program on marine biodiversity, the Canadian Healthy Ocean Network. A major effort needs to be undertaken to establish a complete baseline of Canadian marine biodiversity of all taxonomic groups, especially if we are to understand and conserve this part of Canada's natural heritage

Non-Standard Errors

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants

Future Moment Assessment for Action Query

In this paper, we aim to tackle the task of Assessing Future Moment of an Action of Interest (AFM-AI). The goal of this task is to assess if an action of interest will happen or not as well as the starting moment of the action. We aim to assess starting moments at any time-horizon of the future. To this end, we tackle the regression task of the starting moments as a generation task using a Deterministic Residual Guided Variational Regression Module (DR-VRM), which is built on a Variational Regression Module (VRM) and a deterministic residual network. The VRM takes the uncertainty into account and is capable of generating diverse predictions for the starting moment. The deterministic network encourages the VRM to learn from deterministic residual information in order to generate more precise predictions for moment assessment. Experimental results on three datasets clearly show that the proposed method is capable of generating both diverse and precise predictions of starting moments for query actions

Time-Conditioned Action Anticipation in One Shot

The goal of human action anticipation is to predict future actions. Ideally, in real-world applications such as video surveillance and self-driving systems, future actions should not only be predicted with high accuracy but also at arbitrary and variable time-horizons ranging from shortto long-term predictions. Current work mostly focuses on predicting the next action and thus long-term prediction is achieved by recursive prediction of each next action, which is both inefficient and accumulates errors. In this paper, we propose a novel time-conditioned method for efficient and effective long-term action anticipation. There are two key ingredients to our approach. First, by explicitly conditioning our anticipation network on time allows to efficiently anticipate also long-term actions. And second, we propose an attended temporal feature and a time-conditioned skip connection to extract relevant and useful information from observations for effective anticipation. We conduct extensive experiments on the large-scale Epic-Kitchen and the 50Salads Datasets. Experimental results show that the proposed method is capable of anticipating future actions at both short-term and long-term, and achieves state-of-theart performance

Sports-QA : A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports

Reasoning over sports videos for question answering is an important task with numerous applications, such as player training and information retrieval. However, this task has not been explored due to the lack of relevant datasets and the challenging nature it presents. Most datasets for video question answering (VideoQA) focus mainly on general and coarse-grained understanding of daily-life videos, which is not applicable to sports scenarios requiring professional action understanding and fine-grained motion analysis. In this paper, we introduce the first dataset, named Sports-QA, specifically designed for the sports VideoQA task. The Sports-QA dataset includes various types of questions, such as descriptions, chronologies, causalities, and counterfactual conditions, covering multiple sports. Furthermore, to address the characteristics of the sports VideoQA task, we propose a new Auto-Focus Transformer (AFT) capable of automatically focusing on particular scales of temporal information for question answering. We conduct extensive experiments on Sports-QA, including baseline studies and the evaluation of different methods. The results demonstrate that our AFT achieves state-of-the-art performance