Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop

The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systematic manipulation of input to neural networks and investigating the impact on their performance, testing whether interpretable knowledge can be decoded from intermediate representations acquired by neural networks, proposing modifications to neural network architectures to make their knowledge state or generated output more explainable, and examining the performance of networks on simplified or formal languages. Here we review a number of representative studies in each category

SEGMENTATION, RECOGNITION, AND ALIGNMENT OF COLLABORATIVE GROUP MOTION

Modeling and recognition of human motion in videos has broad applications in behavioral biometrics, content-based visual data analysis, security and surveillance, as well as designing interactive environments. Significant progress has been made in the past two decades by way of new models, methods, and implementations. In this dissertation, we focus our attention on a relatively less investigated sub-area called collaborative group motion analysis. Collaborative group motions are those that typically involve multiple objects, wherein the motion patterns of individual objects may vary significantly in both space and time, but the collective motion pattern of the ensemble allows characterization in terms of geometry and statistics. Therefore, the motions or activities of an individual object constitute local information. A framework to synthesize all local information into a holistic view, and to explicitly characterize interactions among objects, involves large scale global reasoning, and is of significant complexity. In this dissertation, we first review relevant previous contributions on human motion/activity modeling and recognition, and then propose several approaches to answer a sequence of traditional vision questions including 1) which of the motion elements among all are the ones relevant to a group motion pattern of interest (Segmentation); 2) what is the underlying motion pattern (Recognition); and 3) how two motion ensembles are similar and how we can 'optimally' transform one to match the other (Alignment). Our primary practical scenario is American football play, where the corresponding problems are 1) who are offensive players; 2) what are the offensive strategy they are using; and 3) whether two plays are using the same strategy and how we can remove the spatio-temporal misalignment between them due to internal or external factors. The proposed approaches discard traditional modeling paradigm but explore either concise descriptors, hierarchies, stochastic mechanism, or compact generative model to achieve both effectiveness and efficiency. In particular, the intrinsic geometry of the spaces of the involved features/descriptors/quantities is exploited and statistical tools are established on these nonlinear manifolds. These initial attempts have identified new challenging problems in complex motion analysis, as well as in more general tasks in video dynamics. The insights gained from nonlinear geometric modeling and analysis in this dissertation may hopefully be useful toward a broader class of computer vision applications

Unnatural Selection: A new formal approach to punctuated equilibrium in economic systems

Generalized Darwinian evolutionary theory has emerged as central to the description of economic process (e.g., Aldrich et. al., 2008). Here we demonstrate that, just as Darwinian principles provide necessary, but not sufficient, conditions for understanding the dynamics of social entities, in a similar manner the asymptotic limit theorems of information theory provide another set of necessary conditions that constrain the evolution of socioeconomic process. These latter constraints can, however, easily be formulated as a statistics-like analytic toolbox for the study of empirical data that is consistent with a generalized Darwinism, and this is no small thing

The Meaning of Action:a review on action recognition and mapping

In this paper, we analyze the different approaches taken to date within the computer vision, robotics and artificial intelligence communities for the representation, recognition, synthesis and understanding of action. We deal with action at different levels of complexity and provide the reader with the necessary related literature references. We put the literature references further into context and outline a possible interpretation of action by taking into account the different aspects of action recognition, action synthesis and task-level planning

Constructing a concept of number

Numbers are concepts whose content, structure, and organization are influenced by the material forms used to represent and manipulate them. Indeed, as argued here, it is the inclusion of multiple forms (distributed objects, fingers, single- and two-dimensional forms like pebbles and abaci, and written notations) that is the mechanism of numerical elaboration. Further, variety in employed forms explains at least part of the synchronic and diachronic variability that exists between and within cultural number systems. Material forms also impart characteristics like linearity that may persist in the form of knowledge and behaviors, ultimately yielding numerical concepts that are irreducible to and functionally independent of any particular form. Material devices used to represent and manipulate numbers also interact with language in ways that reinforce or contrast different aspects of numerical cognition. Not only does this interaction potentially explain some of the unique aspects of numerical language, it suggests that the two are complementary but ultimately distinct means of accessing numerical intuitions and insights. The potential inclusion of materiality in contemporary research in numerical cognition is advocated, both for its explanatory power, as well as its influence on psychological, behavioral, and linguistic aspects of numerical cognition