Network Sketching: Exploiting Binary Structure in Deep CNNs

Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks. However, deep networks are typically resource-intensive and thus difficult to be deployed on mobile devices. Recently, CNNs with binary weights have shown compelling efficiency to the community, whereas the accuracy of such models is usually unsatisfactory in practice. In this paper, we introduce network sketching as a novel technique of pursuing binary-weight CNNs, targeting at more faithful inference and better trade-off for practical applications. Our basic idea is to exploit binary structure directly in pre-trained filter banks and produce binary-weight models via tensor expansion. The whole process can be treated as a coarse-to-fine model approximation, akin to the pencil drawing steps of outlining and shading. To further speedup the generated models, namely the sketches, we also propose an associative implementation of binary tensor convolutions. Experimental results demonstrate that a proper sketch of AlexNet (or ResNet) outperforms the existing binary-weight models by large margins on the ImageNet large scale classification task, while the committed memory for network parameters only exceeds a little.Comment: To appear in CVPR201

Beyond capitalism and liberal democracy: on the relevance of GDH Cole’s sociological critique and alternative

This article argues for a return to the social thought of the often ignored early 20th-century English thinker GDH Cole. The authors contend that Cole combined a sociological critique of capitalism and liberal democracy with a well-developed alternative in his work on guild socialism bearing particular relevance to advanced capitalist societies. Both of these, with their focus on the limitations on ‘free communal service’ in associations and the inability of capitalism to yield emancipation in either production or consumption, are relevant to social theorists looking to understand, critique and contribute to the subversion of neoliberalism. Therefore, the authors suggest that Cole’s associational sociology, and the invitation it provides to think of formations beyond capitalism and liberal democracy, is a timely and valuable resource which should be returned to

Dynamic Adaptation on Non-Stationary Visual Domains

Domain adaptation aims to learn models on a supervised source domain that perform well on an unsupervised target. Prior work has examined domain adaptation in the context of stationary domain shifts, i.e. static data sets. However, with large-scale or dynamic data sources, data from a defined domain is not usually available all at once. For instance, in a streaming data scenario, dataset statistics effectively become a function of time. We introduce a framework for adaptation over non-stationary distribution shifts applicable to large-scale and streaming data scenarios. The model is adapted sequentially over incoming unsupervised streaming data batches. This enables improvements over several batches without the need for any additionally annotated data. To demonstrate the effectiveness of our proposed framework, we modify associative domain adaptation to work well on source and target data batches with unequal class distributions. We apply our method to several adaptation benchmark datasets for classification and show improved classifier accuracy not only for the currently adapted batch, but also when applied on future stream batches. Furthermore, we show the applicability of our associative learning modifications to semantic segmentation, where we achieve competitive results

Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during “Action Words

Studies demonstrating the involvement of motor brain structures in language processing typically focus on \ud time windows beyond the latencies of lexical-semantic access. Consequently, such studies remain inconclusive regarding whether motor brain structures are recruited directly in language processing or through post-linguistic conceptual imagery. In the present study, we introduce a grip-force sensor that allows online measurements of language-induced motor activity during sentence listening. We use this tool to investigate whether language-induced motor activity remains constant or is modulated in negative, as opposed to affirmative, linguistic contexts. Our findings demonstrate that this simple experimental paradigm can be used to study the online crosstalk between language and the motor systems in an ecological and economical manner. Our data further confirm that the motor brain structures that can be called upon during action word processing are not mandatorily involved; the crosstalk is asymmetrically\ud governed by the linguistic context and not vice versa

NASA JSC neural network survey results

A survey of Artificial Neural Systems in support of NASA's (Johnson Space Center) Automatic Perception for Mission Planning and Flight Control Research Program was conducted. Several of the world's leading researchers contributed papers containing their most recent results on artificial neural systems. These papers were broken into categories and descriptive accounts of the results make up a large part of this report. Also included is material on sources of information on artificial neural systems such as books, technical reports, software tools, etc

Thought/ visual processing: ctrl + x, y, z, v

With this text I wish to revisit a long-standing preoccupation of mine which addresses the extent to which the digital medium may have brought about a paradigm shift in creative process; one which has been effectuated through a conversion of the creative medium itself - from atoms to bits. As a visual practitioner, what is particularly significant to my inquiry is that this change in medium is deemed to have brought about a change in language that affects the very nature of autographic work which comes into being as visual output: Following Goodman’s definitions of allographic and autographic output (1976), McCullough (1996) asserts that visual artworks may now be considered to be allographic productions since they presently share the same attributes of notationally based allographic work, which has traditionally manifested only as music or as literary output. The result is a work environment which wide opens the doors to unprecedented levels of non-linear process and experimentation whilst engaged in the visually creative act