Going Deeper with Semantics: Video Activity Interpretation using Semantic Contextualization

A deeper understanding of video activities extends beyond recognition of underlying concepts such as actions and objects: constructing deep semantic representations requires reasoning about the semantic relationships among these concepts, often beyond what is directly observed in the data. To this end, we propose an energy minimization framework that leverages large-scale commonsense knowledge bases, such as ConceptNet, to provide contextual cues to establish semantic relationships among entities directly hypothesized from video signal. We mathematically express this using the language of Grenander's canonical pattern generator theory. We show that the use of prior encoded commonsense knowledge alleviate the need for large annotated training datasets and help tackle imbalance in training through prior knowledge. Using three different publicly available datasets - Charades, Microsoft Visual Description Corpus and Breakfast Actions datasets, we show that the proposed model can generate video interpretations whose quality is better than those reported by state-of-the-art approaches, which have substantial training needs. Through extensive experiments, we show that the use of commonsense knowledge from ConceptNet allows the proposed approach to handle various challenges such as training data imbalance, weak features, and complex semantic relationships and visual scenes.Comment: Accepted to WACV 201

Unsupervised Discovery of Parts, Structure, and Dynamics

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose a novel formulation that simultaneously learns a hierarchical, disentangled object representation and a dynamics model for object parts from unlabeled videos. Our Parts, Structure, and Dynamics (PSD) model learns to, first, recognize the object parts via a layered image representation; second, predict hierarchy via a structural descriptor that composes low-level concepts into a hierarchical structure; and third, model the system dynamics by predicting the future. Experiments on multiple real and synthetic datasets demonstrate that our PSD model works well on all three tasks: segmenting object parts, building their hierarchical structure, and capturing their motion distributions.Comment: ICLR 2019. The first two authors contributed equally to this wor

Doing evolution in economic geography

Evolutionary approaches in economic geography face questions about the relationships between their concepts, theories, methods, politics, and policy implications. Amidst the growing but unsettled consensus that evolutionary approaches should employ plural methodologies, the aims here are, first, to identify some of the difficult issues confronting those working with different frameworks. The concerns comprise specifying and connecting research objects, subjects, and levels; handling agency and context; engaging and integrating the quantitative and the qualitative; comparing cases; and, considering politics, policy, and praxis. Second, the purpose is to articulate a distinctive geographical political economy approach, methods, and illustrative examples in addressing these issues. Bringing different views of evolution in economic geography into dialogue and disagreement renders methodological pluralism a means toward improved understanding and explanation rather than an end in itself. Confronting such thorny matters needs to be embedded in our research practices and supported by greater openness; more and better substantiation of our conceptual, theoretical, and empirical claims; enhanced critical reflection; and deeper engagement with politics, policy, and praxis