Video Object Detection with an Aligned Spatial-Temporal Memory

We introduce Spatial-Temporal Memory Networks for video object detection. At its core, a novel Spatial-Temporal Memory module (STMM) serves as the recurrent computation unit to model long-term temporal appearance and motion dynamics. The STMM's design enables full integration of pretrained backbone CNN weights, which we find to be critical for accurate detection. Furthermore, in order to tackle object motion in videos, we propose a novel MatchTrans module to align the spatial-temporal memory from frame to frame. Our method produces state-of-the-art results on the benchmark ImageNet VID dataset, and our ablative studies clearly demonstrate the contribution of our different design choices. We release our code and models at http://fanyix.cs.ucdavis.edu/project/stmn/project.html

A framework for automatic semantic video annotation

The rapidly increasing quantity of publicly available videos has driven research into developing automatic tools for indexing, rating, searching and retrieval. Textual semantic representations, such as tagging, labelling and annotation, are often important factors in the process of indexing any video, because of their user-friendly way of representing the semantics appropriate for search and retrieval. Ideally, this annotation should be inspired by the human cognitive way of perceiving and of describing videos. The difference between the low-level visual contents and the corresponding human perception is referred to as the ‘semantic gap’. Tackling this gap is even harder in the case of unconstrained videos, mainly due to the lack of any previous information about the analyzed video on the one hand, and the huge amount of generic knowledge required on the other. This paper introduces a framework for the Automatic Semantic Annotation of unconstrained videos. The proposed framework utilizes two non-domain-specific layers: low-level visual similarity matching, and an annotation analysis that employs commonsense knowledgebases. Commonsense ontology is created by incorporating multiple-structured semantic relationships. Experiments and black-box tests are carried out on standard video databases for action recognition and video information retrieval. White-box tests examine the performance of the individual intermediate layers of the framework, and the evaluation of the results and the statistical analysis show that integrating visual similarity matching with commonsense semantic relationships provides an effective approach to automated video annotation

Occlusion and Motion Reasoning for Long-Term Tracking

International audienceObject tracking is a reoccurring problem in computer vision. Tracking-by-detection approaches, in particular Struck (Hare et al., 2011), have shown to be competitive in recent evaluations. However, such approaches fail in the presence of long-term occlusions as well as severe viewpoint changes of the object. In this paper we propose a principled way to combine occlusion and motion reasoning with a tracking-by-detection approach. Occlusion and motion reasoning is based on state-of-the-art long-term trajectories which are labeled as object or background tracks with an energy-based formulation. The overlap between labeled tracks and detected regions allows to identify occlusions. The motion changes of the object between consecutive frames can be estimated robustly from the geometric relation between object trajectories. If this geometric change is significant, an additional detector is trained. Experimental results show that our tracker obtains state-of-the-art results and handles occlusion and viewpoints changes better than competing tracking methods

The maximum of the local time of a diffusion process in a drifted Brownian potential

We consider a one-dimensional diffusion process $X$ in a $(-\kappa/2)$-drifted Brownian potential for $\kappa\neq 0$. We are interested in the maximum of its local time, and study its almost sure asymptotic behaviour, which is proved to be different from the behaviour of the maximum local time of the transient random walk in random environment. We also obtain the convergence in law of the maximum local time of $X$ under the annealed law after suitable renormalization when $\kappa \geq 1$. Moreover, we characterize all the upper and lower classes for the hitting times of $X$, in the sense of Paul L\'evy, and provide laws of the iterated logarithm for the diffusion $X$ itself. To this aim, we use annealed technics.Comment: 38 pages, new version, merged with hal-00013040 (arXiv:math/0511053), with some additional result

Alternative proof for the localization of Sinai's walk

We give an alternative proof of the localization of Sinai's random walk in random environment under weaker hypothesis than the ones used by Sinai. Moreover we give estimates that are stronger than the one of Sinai on the localization neighborhood and on the probability for the random walk to stay inside this neighborhood

Efficacy of labral repair, biceps tenodesis, and diagnostic arthroscopy for SLAP Lesions of the shoulder: a randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>Surgery for type II SLAP (superior labral anterior posterior) lesions of the shoulder is a promising but unproven treatment. The procedures include labral repair or biceps tenodesis. Retrospective cohort studies have suggested that the benefits of tenodesis include pain relief and improved function, and higher patient satisfaction, which was reported in a prospective non-randomised study. There have been no completed randomised controlled trials of surgery for type II SLAP lesions. The aims of this participant and observer blinded randomised placebo-controlled trial are to compare the short-term (6 months) and long-term (2 years) efficacy of labral repair, biceps tenodesis, and placebo (diagnostic arthroscopy) for alleviating pain and improving function for type II SLAP lesions.</p> <p>Methods/Design</p> <p>A double-blind randomised controlled trial are performed using 120 patients, aged 18 to 60 years, with a history for type II SLAP lesions and clinical signs suggesting type II SLAP lesion, which were documented by MR arthrography and arthroscopy. Exclusion criteria include patients who have previously undergone operations for SLAP lesions or recurrent shoulder dislocations, and ruptures of the rotator cuff or biceps tendon. Outcomes will be assessed at baseline, three, six, 12, and 24 months. Primary outcome measures will be the clinical Rowe Score (1988-version) and the Western Ontario Instability Index (WOSI) at six and 24 months. Secondary outcome measures will include the Shoulder Instability Questionnaire (SIQ), the generic EuroQol (EQ-5 D and EQ-VAS), return to work and previous sports activity, complications, and the number of reoperations.</p> <p>Discussion</p> <p>The results of this trial will be of international importance and the results will be translatable into clinical practice.</p> <p>Trial Registration</p> <p><b>[ClinicalTrials.gov NCT00586742]</b></p

The development and general morphology of the telencephalon of actinopterygian fishes: synopsis, documentation and commentary

The Actinopterygii or ray-finned fishes comprise, in addition to the large superorder of teleosts, four other superorders, namely the cladistians, the chondrosteans, the ginglymodes, and the halecomorphs, each with a limited number of species. The telencephalon of actinopterygian fishes differs from that in all other vertebrates in that it consists of a pair of solid lobes. Lateral ventricles surrounded by nervous tissue are entirely lacking. At the end of the nineteenth century, the theory was advanced that the unusual configuration of the forebrain in actinopterygians results from an outward bending or eversion of its lateral walls. This theory was accepted by some authors, rejected or neglected by others, and modified by some other authors. The present paper is based on the data derived from the literature, complemented by new observations on a large collection of histological material comprising specimens of all five actinopterygian superorders. The paper consists of three parts. In the first, a survey of the development of the telencephalon in actinopterygian fishes is presented. The data collected show clearly that an outward bending or eversion of the pallial parts of the solid hemispheres is the principal morphogenetic event in all five actinopterygian superorders. In all of these superorders, except for the cladistians, eversion is coupled with a marked thickening of the pallial walls. In the second part, some aspects of the general morphology of the telencephalon in mature actinopterygians are highlighted. It is pointed out that (1) the degree of eversion varies considerably among the various actinopterygian groups; (2) eversion leads to the transformation of the telencephalic roof plate into a wide membrane or tela choroidea, which is bilaterally attached to the lateral or ventrolateral aspect of the solid hemispheres; (3) the lines of attachment or taeniae of the tela choroidea form the most important landmarks in the telencephalon of actinopterygians, indicating the sites where the greatly enlarged ventricular surface of the hemispheres ends and its reduced meningeal surface begins; (4) the meningeal surface of the telencephalon shows in most actinopterygians bilaterally a longitudinally oriented sulcus externus, the depth of which is generally positively correlated with the degree of eversion; (5) a distinct lateral olfactory tract, occupying a constant topological position close to the taenia, is present in all actinopterygians studied; and (6) this tract is not homologous to the tract of the same name in the evaginated and inverted forebrains of other groups of vertebrates. In the third and final section, the concept that the structural organization of the pallium in actinopterygians can be fully explained by a simple eversion of its walls, and the various theories, according to which the eversion is complicated by extensive shifts of its constituent cell groups, are discussed and evaluated. It is concluded that there are no reasons to doubt that the pallium of actinopterygian fishes is the product of a simple and complete eversion

Video Segmentation with Superpixels

Due to its importance, video segmentation has regained interest recently. However, there is no common agreement about the necessary ingredients for best performance. This work contributes a thorough analysis of various within- and between-frame affinities suitable for video segmentation. Our results show that a frame-based superpixel segmentation combined with a few motion and appearance-based affinities are sufficient to obtain good video segmentation performance. A second contribution of the paper is the extension of [1] to include motion-cues, which makes the algorithm globally aware of motion, thus improving its performance for video sequences. Finally, we contribute an extension of an established image segmentation benchmark [1] to videos, allowing coarse-to-fine video segmentations and multiple human annotations. Our results are tested on BMDS [2], and compared to existing methods

PDEs for tensor image processing

Methods based on partial differential equations (PDEs) belong to those image processing techniques that can be extended in a particularly elegant way to tensor fields. In this survey paper the most important PDEs for discontinuity-preserving denoising of tensor fields are reviewed such that the underlying design principles becomes evident. We consider isotropic and anisotropic diffusion filters and their corresponding variational methods, mean curvature motion, and selfsnakes. These filters preserve positive semidefiniteness of any positive semidefinite initial tensor field. Finally we discuss geodesic active contours for segmenting tensor fields. Experiments are presented that illustrate the behaviour of all these methods