The Origin of Clusters in Recurrent Neural Network State Space

Cluster analysis has been successfully applied to the problem understanding hidden unit representations in both feed-forward and recurrent neural networks. While the topological properties of feed-forward networks may support the use of cluster analysis, the results described within this paper suggest that applications to recurrent networks are not justified. This paper illustrates how clustering fails to provide useful insights into the underlying task-dependent information processing mechanism of recurrent networks. In this paper, I first demonstrate that randomly generated networks display a surprising amount of clustering before training. Then I explain that the clustering structure emerges, not in response to the task training, but because of the volume-reducing iterated mappings that comprise the commonly used recurrent neural networks models

Scenes from Exclusive-Or: Back Propagation is Sensitive to Initial Conditions

This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the badt-propagation technique. W e first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of bade propagation to initial weight configuration

Meta-analysis of genome-wide association studies for extraversion:Findings from the Genetics of Personality Consortium

Extraversion is a relatively stable and heritable personality trait associated with numerous psychosocial, lifestyle and health outcomes. Despite its substantial heritability, no genetic variants have been detected in previous genome-wide association (GWA) studies, which may be due to relatively small sample sizes of those studies. Here, we report on a large meta-analysis of GWA studies for extraversion in 63,030 subjects in 29 cohorts. Extraversion item data from multiple personality inventories were harmonized across inventories and cohorts. No genome-wide significant associations were found at the single nucleotide polymorphism (SNP) level but there was one significant hit at the gene level for a long non-coding RNA site (LOC101928162). Genome-wide complex trait analysis in two large cohorts showed that the additive variance explained by common SNPs was not significantly different from zero, but polygenic risk scores, weighted using linkage information, significantly predicted extraversion scores in an independent cohort. These results show that extraversion is a highly polygenic personality trait, with an architecture possibly different from other complex human traits, including other personality traits. Future studies are required to further determine which genetic variants, by what modes of gene action, constitute the heritable nature of extraversion

Tears evoke the intention to offer social support: A systematic investigation of the interpersonal effects of emotional crying across 41 countries

Tearful crying is a ubiquitous and likely uniquely human phenomenon. Scholars have argued that emotional tears serve an attachment function: Tears are thought to act as a social glue by evoking social support intentions. Initial experimental studies supported this proposition across several methodologies, but these were conducted almost exclusively on participants from North America and Europe, resulting in limited generalizability. This project examined the tears-social support intentions effect and possible mediating and moderating variables in a fully pre-registered study across 7007 participants (24,886 ratings) and 41 countries spanning all populated continents. Participants were presented with four pictures out of 100 possible targets with or without digitally-added tears. We confirmed the main prediction that seeing a tearful individual elicits the intention to support, d = 0.49 [0.43, 0.55]. Our data suggest that this effect could be mediated by perceiving the crying target as warmer and more helpless, feeling more connected, as well as feeling more empathic concern for the crier, but not by an increase in personal distress of the observer. The effect was moderated by the situational valence, identifying the target as part of one's group, and trait empathic concern. A neutral situation, high trait empathic concern, and low identification increased the effect. We observed high heterogeneity across countries that was, via split-half validation, best explained by country-level GDP per capita and subjective well-being with stronger effects for higher-scoring countries. These findings suggest that tears can function as social glue, providing one possible explanation why emotional crying persists into adulthood.</p

A multi-country test of brief reappraisal interventions on emotions during the COVID-19 pandemic.

The COVID-19 pandemic has increased negative emotions and decreased positive emotions globally. Left unchecked, these emotional changes might have a wide array of adverse impacts. To reduce negative emotions and increase positive emotions, we tested the effectiveness of reappraisal, an emotion-regulation strategy that modifies how one thinks about a situation. Participants from 87 countries and regions (n = 21,644) were randomly assigned to one of two brief reappraisal interventions (reconstrual or repurposing) or one of two control conditions (active or passive). Results revealed that both reappraisal interventions (vesus both control conditions) consistently reduced negative emotions and increased positive emotions across different measures. Reconstrual and repurposing interventions had similar effects. Importantly, planned exploratory analyses indicated that reappraisal interventions did not reduce intentions to practice preventive health behaviours. The findings demonstrate the viability of creating scalable, low-cost interventions for use around the world

Recurrent Networks: State Machines Or Iterated Function Systems?

this paper, clustering of hidden unit activations, or recurrent network state space, provides incomplete information regarding the IP state of the network. IP states determine future behavior as well as encapsulate input history. The network&apos;s state transformations can exhibit sensitivity to initial conditions and generate disparate futures for state clusters of all sizes. The second part of the paper presents IFS theory and shows how it can explain recurrent network state dynamics. By linking IFSs and recurrent networks, existing constraints on network dynamics independent of network models are now evident. By assuming a finite set of inputs, which is often the case in symbolic domains, one can describe recurrent network models as a finite collection of nonlinear state transformations.The interaction of several transforms produces complex state spaces with recursive structure. The limit behavior of the collection of transformations, and recurrent networks in symbolic applications, is more complex than the union of the individual transformations. An input driven recurrent network behaves like the random iteration algorithm. Infinite input sequence generates sequences of points dense in the state space attractor when the transformations are contractive. While the demonstration in this paper used the SCN, other models produce similar IFS-like behaviors as long as the network&apos;s input selects transformations [19]. The IFS approach also explains the phenomena of state clustering in recurrent networks. In [20], ServenSchreiber et al report significant clustering in simple recurrent networks [21] both before and after training from the Reber grammar prediction task. A set of random transformations will normally reduce the volume of the recurrent networks state space, and plac..

Back propagation is sensitive to initial conditions

This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the back-propagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of back propagation to initial weigh