Deep learning with asymmetric connections and Hebbian updates

We show that deep networks can be trained using Hebbian updates yielding similar performance to ordinary back-propagation on challenging image datasets. To overcome the unrealistic symmetry in connections between layers, implicit in back-propagation, the feedback weights are separate from the feedforward weights. The feedback weights are also updated with a local rule, the same as the feedforward weights - a weight is updated solely based on the product of activity of the units it connects. With fixed feedback weights as proposed in Lillicrap et. al (2016) performance degrades quickly as the depth of the network increases. If the feedforward and feedback weights are initialized with the same values, as proposed in Zipser and Rumelhart (1990), they remain the same throughout training thus precisely implementing back-propagation. We show that even when the weights are initialized differently and at random, and the algorithm is no longer performing back-propagation, performance is comparable on challenging datasets. We also propose a cost function whose derivative can be represented as a local Hebbian update on the last layer. Convolutional layers are updated with tied weights across space, which is not biologically plausible. We show that similar performance is achieved with untied layers, also known as locally connected layers, corresponding to the connectivity implied by the convolutional layers, but where weights are untied and updated separately. In the linear case we show theoretically that the convergence of the error to zero is accelerated by the update of the feedback weights

Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties

The pairwise maximum entropy model, also known as the Ising model, has been widely used to analyze the collective activity of neurons. However, controversy persists in the literature about seemingly inconsistent findings, whose significance is unclear due to lack of reliable error estimates. We therefore develop a method for accurately estimating parameter uncertainty based on random walks in parameter space using adaptive Markov Chain Monte Carlo after the convergence of the main optimization algorithm. We apply our method to the spiking patterns of excitatory and inhibitory neurons recorded with multielectrode arrays in the human temporal cortex during the wake-sleep cycle. Our analysis shows that the Ising model captures neuronal collective behavior much better than the independent model during wakefulness, light sleep, and deep sleep when both excitatory (E) and inhibitory (I) neurons are modeled; ignoring the inhibitory effects of I-neurons dramatically overestimates synchrony among E-neurons. Furthermore, information-theoretic measures reveal that the Ising model explains about 80%-95% of the correlations, depending on sleep state and neuron type. Thermodynamic measures show signatures of criticality, although we take this with a grain of salt as it may be merely a reflection of long-range neural correlations.Comment: 17 pages, 8 figure

Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future

Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)

Neurophysiological correlates underlying social behavioural adjustment of conformity

[eng] Conformity is the act of changing one’s behaviour to adjust to other human beings. It is a crucial social adaptation that happens when people cooperate, where one sacrifices their own perception, expectations, or beliefs to reach convergence with another person. The aim of the present study was to increase the understanding of the neurophysiological underpinnings regarding the social behavioural adjustment of conformity. We start by introducing cooperation and how it is ingrained in human behaviour. Then we explore the different processes that the brain requires for the social behavioural adjustment of conformity. To engage in this social adaptation, a person needs a self-referenced learning mechanism based on a predictive model that helps them track the prediction errors from unexpected events. Also, the brain uses its monitoring and control systems to encode different value functions used in action selection. The use of different learning models in neuroscience, such as reinforcement learning (RL) algorithms, has been a success story identifying learning systems by means of the mapped activity of different regions in the brain. Importantly, experimental paradigms which has been used to study conformity have not been based in a social interaction setting and, hence, the results, cannot be used to explain an inherently social phenomenon. The main goal of the present thesis is to study the neurophysiological mechanisms underlying the social behavioural adjustment of conformity and its modulation with repeated interaction. To reach this goal, we have first designed a new experimental task where conformity appears spontaneously between two persons and in a reiterative way. This design exposes learning acquisition processes, which require iterative loops, as well as other cognitive control mechanisms such as feedback processing, value-based decision making and attention. The first study shows that people who previously cooperate increase their level of convergence and report a significantly more satisfying overall experience. In addition, participants learning on their counterparts’ behaviour can be explained using a RL algorithm as opposed to when they do not have previously cooperated. In the second study, we have studied the event-related potentials (ERP) and oscillatory power underlying conformity. ERP results show different levels of cognitive engagement that are associated to distinct levels of conformity. Also, time-frequency analysis shows evidence in theta, alpha and beta related to different functions such as cognitive control, attention and, also, reward processing, supporting the idea that convergence between dyads acts as a social reward. Finally, in the third study, we explored the intra- and inter- oscillatory connectivity between electrodes related to behavioural convergence. In intra-brain oscillatory connectivity coherence, we have found two different dynamics related to attention and executive functions in alpha. Also, we have found that the learning about peer’s behaviour as computed using a RL is mediated by theta oscillatory connectivity. Consequently, combined evidence from Study 2 and Study 3 suggests that both cognitive control and learning computations happening in the social behavioural adaptation of conformity are signalled in theta frequency band. The present work is one of the first studies describing, with credible evidence, that conformity, when this occurs willingly and spontaneously rather than induced, engages different brain activity underlying reward-guided learning, cognitive control, and attention.[spa] La conformidad es el acto de cambiar el comportamiento de uno a favor de ajustarnos a otros seres humanos. Se trata de una adaptación crucial que ocurre cuando la gente coopera, donde uno sacrifica su propia percepción, expectativas o creencias en aras de conseguir una convergencia con la otra persona. El objetivo del presente estudio ha sido tratar de aportar a la comprensión de las estructuras neurofisiológicas que soportan un ajuste social como el de la conformidad. En la primera parte de esta tesis comenzamos hablando de la cooperación y lo profundamente arraigada que está en nuestro comportamiento. Más tarde exploramos diferentes procesos que el cerebro requiere en el ajuste social de la conformidad. Así pues, para involucrarse en esta adaptación social, una persona requiere de un mecanismo de aprendizaje auto-referenciado basado en un modelo predictivo que le ayude a seguir el rastro de los errores de predicción que acompañan a los eventos inesperados. Además, el cerebro usa sus sistemas de control y predicción para codificar diferentes funciones de valor usadas en la selección de acción. El uso de diferentes modelos de aprendizaje en neurociencia, como los algoritmos de aprendizaje por refuerzo (RL), han sido una historia de éxito a la hora de identificar los sistemas de aprendizaje a través del mapeo de la actividad de diferentes regiones del cerebro. Es importante destacar que los paradigmas experimentales que se han usado para estudiar la conformidad no se han basado en entornos de interacción social y que, por lo tanto, sus resultados no pueden usarse para explicar un fenómeno inherentemente social. El objetivo principal de la presente tesis es el estudio de los mecanismos neurofisiológicos que fundamentan el comportamiento de ajuste social de la conformidad y su modulación con la interacción repetida. Para alcanzar este objetivo, primero hemos diseñado una nueva tarea experimental en la que la conformidad aparece de forma espontánea entre dos personas y, además, de forma reiterativa. Este diseño permite exponer tanto los procesos de adquisición del aprendizaje, que requieren de ciclos iterativos, así como otros mecanismos de control cognitivo tales como el procesamiento de la retroalimentación, las tomas de decisiones basadas en procesos valorativos y la atención. El primer estudio nos muestra que la gente que coopera previamente incrementa sus niveles de convergencia y reportan significativamente una experiencia generalmente más satisfactoria en el experimento. Adicionalmente, un modelo de RL nos explica que los participantes tratan de aprender del comportamiento de sus parejas en mayor medida si estos han cooperado previamente. En el segundo estudio, hemos estudiado los potenciales relacionados con eventos (ERP) y el poder de las oscilaciones que sustentan la conformidad. Los estudios de ERP muestran diferentes niveles de implicación cognitiva asociados con diferentes niveles de conformidad. Además, los análisis de tiempo-frecuencia muestran evidencia en theta, alfa y beta relacionados con diferentes funciones como el control cognitivo, la atención, y, también, el procesamiento de la recompensa, apoyando la idea de que la convergencia entre díadas actúa como una recompensa social. Finalmente, en el tercer estudio, exploramos la conectividad oscilatoria intra e inter entre electrodos que se pudieran relacionar con la conducta de convergencia. A propósito de la conectividad oscilatoria coherente intra, hemos hallado dos dinámicas relacionadas con la atención y las funciones ejecutivas en alfa. Asimismo, hemos encontrado que el aprendizaje de la conducta de la pareja computada a través de RL está mediada a través de la conectividad oscilatoria de theta. Consecuentemente, la evidencia combinada entre el estudio 2 y el estudio 3 sugiere que conjuntamente el control cognitivo y las computaciones de aprendizaje que ocurren en la conducta de adaptación social de la conformidad están relacionadas con la actividad de la banda de frecuencia theta. Este trabajo constituye uno de los primeros estudios que describen, con evidencia creíble, que la conformidad, cuando ocurre voluntaria y espontáneamente a diferencia cuando esta es inducida, involucra actividad del cerebro que se fundamenta en el aprendizaje guiado por reforzamiento, el control cognitivo y la atención