Move Forward and Tell: A Progressive Generator of Video Descriptions

We present an efficient framework that can generate a coherent paragraph to describe a given video. Previous works on video captioning usually focus on video clips. They typically treat an entire video as a whole and generate the caption conditioned on a single embedding. On the contrary, we consider videos with rich temporal structures and aim to generate paragraph descriptions that can preserve the story flow while being coherent and concise. Towards this goal, we propose a new approach, which produces a descriptive paragraph by assembling temporally localized descriptions. Given a video, it selects a sequence of distinctive clips and generates sentences thereon in a coherent manner. Particularly, the selection of clips and the production of sentences are done jointly and progressively driven by a recurrent network -- what to describe next depends on what have been said before. Here, the recurrent network is learned via self-critical sequence training with both sentence-level and paragraph-level rewards. On the ActivityNet Captions dataset, our method demonstrated the capability of generating high-quality paragraph descriptions for videos. Compared to those by other methods, the descriptions produced by our method are often more relevant, more coherent, and more concise.Comment: Accepted by ECCV 201

Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation

Joint segmentation and classification of fine-grained actions is important for applications of human-robot interaction, video surveillance, and human skill evaluation. However, despite substantial recent progress in large-scale action classification, the performance of state-of-the-art fine-grained action recognition approaches remains low. We propose a model for action segmentation which combines low-level spatiotemporal features with a high-level segmental classifier. Our spatiotemporal CNN is comprised of a spatial component that uses convolutional filters to capture information about objects and their relationships, and a temporal component that uses large 1D convolutional filters to capture information about how object relationships change across time. These features are used in tandem with a semi-Markov model that models transitions from one action to another. We introduce an efficient constrained segmental inference algorithm for this model that is orders of magnitude faster than the current approach. We highlight the effectiveness of our Segmental Spatiotemporal CNN on cooking and surgical action datasets for which we observe substantially improved performance relative to recent baseline methods.Comment: Updated from the ECCV 2016 version. We fixed an important mathematical error and made the section on segmental inference cleare

Conditional Image-Text Embedding Networks

This paper presents an approach for grounding phrases in images which jointly learns multiple text-conditioned embeddings in a single end-to-end model. In order to differentiate text phrases into semantically distinct subspaces, we propose a concept weight branch that automatically assigns phrases to embeddings, whereas prior works predefine such assignments. Our proposed solution simplifies the representation requirements for individual embeddings and allows the underrepresented concepts to take advantage of the shared representations before feeding them into concept-specific layers. Comprehensive experiments verify the effectiveness of our approach across three phrase grounding datasets, Flickr30K Entities, ReferIt Game, and Visual Genome, where we obtain a (resp.) 4%, 3%, and 4% improvement in grounding performance over a strong region-phrase embedding baseline.Comment: ECCV 2018 accepted pape

Activation of the receptor protein tyrosine kinase EphB4 in endometrial hyperplasia and endometrial carcinoma

Background: Members of the Eph family of tyrosine kinases have been implicated in embryonic pattern formation and vascular development; however, little is known about their role in the adult organism. We have observed estrogen-dependent EphB4 expression in the normal breast suggesting its implication in the hormone-controlled homeostasis of this organ. Since the endometrium is a similarly hormone dependent organ and endometrial carcinoma is thought to result from estrogenic stimulation, we have investigated EphB4 expression in normal human endometrium and during its carcinogenesis. Patients and methods: EphB4 expression was analyzed immunohistochemically in 26 normal endometrium specimens, 15 hyperplasias and 102 endometrioid adenocarcinomas and correlated with clinical and prognostic tumor characteristics. Results: In normal endometrial tissue no EphB4 protein was detected. Strikingly, we observed a drastic increase (P <0.0001) in the number of EphB4 protein-expressing glandular epithelial cells in the majority of hyperplasias and carcinomas. Moreover, we found a statistically highly significant positive correlation between EphB4 expression and post-menopausal stage of the patient (P = 0.007). Conclusions: These findings indicate that in the endometrium, EphB4 is an early indicator of malignant development and, thus, EphB4 may represent a potent tool for diagnosis and therapeutic interventio

Urinary pyridinoline cross-links as biomarkers of osteogenesis imperfecta.

Osteogenesis imperfecta (OI) is a group of genetic heterogeneous connective tissue disorders characterized by increased bone fragility and susceptibility to fractures. Laboratory diagnosis relies on time-consuming and cost-intensive biochemical and molecular genetics analyses. Therefore, it is desirable to identify and establish new diagnostic markers for OI that are reliable, cost-effective and easily accessible. In our study we have identified the ratio of the urinary pyridinoline cross-links lysyl-pyridinoline and hydroxylysyl-pyridinoline as a promising, time- and cost-effective biomarker for osteogenesis imperfecta, that could be used furthermore to investigate cases of suspected non-accidental injury in infants

Correction of coarse-graining errors by a two-level method: Application to the Asakura-Oosawa model.

We present a method that exploits self-consistent simulation of coarse-grained and fine-grained models in order to analyze properties of physical systems. The method uses the coarse-grained model to obtain a first estimate of the quantity of interest, before computing a correction by analyzing properties of the fine system. We illustrate the method by applying it to the Asakura-Oosawa model of colloid-polymer mixtures. We show that the liquid-vapor critical point in that system is affected by three-body interactions which are neglected in the corresponding coarse-grained model. We analyze the size of this effect and the nature of the three-body interactions. We also analyze the accuracy of the method as a function of the associated computational effort.Leverhulme Trus

Thin layer composite unimorph ferroelectric driver and sensor

A method for forming ferroelectric wafers is provided. A prestress layer is placed on the desired mold. A ferroelectric wafer is placed on top of the prestress layer. The layers are heated and then cooled, causing the ferroelectric wafer to become prestressed. The prestress layer may include reinforcing material and the ferroelectric wafer may include electrodes or electrode layers may be placed on either side of the ferroelectric layer. Wafers produced using this method have greatly improved output motion