Learning Word Representations with Hierarchical Sparse Coding

We propose a new method for learning word representations using hierarchical regularization in sparse coding inspired by the linguistic study of word meanings. We show an efficient learning algorithm based on stochastic proximal methods that is significantly faster than previous approaches, making it possible to perform hierarchical sparse coding on a corpus of billions of word tokens. Experiments on various benchmark tasks---word similarity ranking, analogies, sentence completion, and sentiment analysis---demonstrate that the method outperforms or is competitive with state-of-the-art methods. Our word representations are available at \url{http://www.ark.cs.cmu.edu/dyogatam/wordvecs/}

From “Oh, OK” to “Ah, yes” to “Aha!”: Hyper-systemizing and the rewards of insight\ud

Hyper-systemizers are individuals displaying an unusually strong bias toward systemizing, i.e. toward explaining events and solving problems by appeal to mechanisms that do not involve intentions or agency. Hyper-systemizing in combination with deficit mentalizing ability typically presents clinically as an autistic spectrum disorder; however, the development of hyper-systemizing in combination with normal-range mentalizing ability is not well characterized. Based on a review and synthesis of clinical, observational, experimental, and neurofunctional studies, it is hypothesized that repeated episodes of insightful problem solving by systemizing result in attentional and motivational sensitization toward further systemizing via progressive and chronic deactivation of the default network. This hypothesis is distinguished from alternatives, and its correlational and causal implications are discussed. Predictions of the default-deactivation model accessible to survey-based instruments, standard cognitive measures and neurofunctional methods are outlined, and evidence pertaining to them considered

Motion as manipulation: Implementation of motion and force analogies by event-file binding and action planning\ud

Tool improvisation analogies are a special case of motion and force analogies that appear to be implemented pre-conceptually, in many species, by event-file binding and action planning. A detailed reconstruction of the analogical reasoning steps involved in Rutherford's and Bohr's development of the first quantized-orbit model of atomic structure is used to show that human motion and force analogies generally can be implemented by the event-file binding and action planning mechanism. Predictions that distinguish this model from competing concept-level models of analogy are discussed, available data pertaining to them are reviewed, and further experimental tests are proposed

Cognitive networks: brains, internet, and civilizations

In this short essay, we discuss some basic features of cognitive activity at several different space-time scales: from neural networks in the brain to civilizations. One motivation for such comparative study is its heuristic value. Attempts to better understand the functioning of "wetware" involved in cognitive activities of central nervous system by comparing it with a computing device have a long tradition. We suggest that comparison with Internet might be more adequate. We briefly touch upon such subjects as encoding, compression, and Saussurean trichotomy langue/langage/parole in various environments.Comment: 16 page

Specialization of the rostral prefrontal cortex for distinct analogy processes

Analogical reasoning is central to learning and abstract thinking. It involves using a more familiar situation (source) to make inferences about a less familiar situation (target). According to the predominant cognitive models, analogical reasoning includes 1) generation of structured mental representations and 2) mapping based on structural similarities between them. This study used functional magnetic resonance imaging to specify the role of rostral prefrontal cortex (PFC) in these distinct processes. An experimental paradigm was designed that enabled differentiation between these processes, by temporal separation of the presentation of the source and the target. Within rostral PFC, a lateral subregion was activated by analogy task both during study of the source (before the source could be compared with a target) and when the target appeared. This may suggest that this subregion supports fundamental analogy processes such as generating structured representations of stimuli but is not specific to one particular processing stage. By contrast, a dorsomedial subregion of rostral PFC showed an interaction between task (analogy vs. control) and period (more activated when the target appeared). We propose that this region is involved in comparison or mapping processes. These results add to the growing evidence for functional differentiation between rostral PFC subregions

Processing of analogy in the thalamocortical circuit

The corticothalamic feedback and the thalamic reticular nucleus have gained much attention lately because of their integrative and modulatory functions. A previous study by the author suggested that this circuitry can process analogies (i.e., the {\em analogy hypothesis}). In this paper, the proposed model was implemented as a network of leaky integrate-and-fire neurons to test the {\em analogy hypothesis}. The previous proposal required specific delay and temporal dynamics, and the implemented network tuned accordingly functioned as predicted. Furthermore, these specific conditions turn out to be consistent with experimental data, suggesting that a further investigation of the thalamocortical circuit within the {\em analogical framework} may be worthwhile

Cultural Neuroeconomics of Intertemporal Choice

According to theories of cultural neuroscience, Westerners and Easterners may have distinct styles of cognition (e.g., different allocation of attention). Previous research has shown that Westerners and Easterners tend to utilize analytical and holistic cognitive styles, respectively. On the other hand, little is known regarding the cultural differences in neuroeconomic behavior. For instance, economic decisions may be affected by cultural differences in neurocomputational processing underlying attention; however, this area of neuroeconomics has been largely understudied. In the present paper, we attempt to bridge this gap by considering the links between the theory of cultural neuroscience and neuroeconomic theory\ud of the role of attention in intertemporal choice. We predict that (i) Westerners are more impulsive and inconsistent in intertemporal choice in comparison to Easterners, and (ii) Westerners more steeply discount delayed monetary losses than Easterners. We examine these predictions by utilizing a novel temporal discounting model based on Tsallis' statistics (i.e. a q-exponential model). Our preliminary analysis of temporal discounting of gains and losses by Americans and Japanese confirmed the predictions from the cultural neuroeconomic theory. Future study directions, employing computational modeling via neural networks, are briefly outlined and discussed