The relational processing limits of classic and contemporary neural network models of language processing

Whether neural networks can capture relational knowledge is a matter of long-standing controversy. Recently, some researchers have argued that (1) classic connectionist models can handle relational structure and (2) the success of deep learning approaches to natural language processing suggests that structured representations are unnecessary to model human language. We tested the Story Gestalt model, a classic connectionist model of text comprehension, and a Sequence-to-Sequence with Attention model, a modern deep learning architecture for natural language processing. Both models were trained to answer questions about stories based on abstract thematic roles. Two simulations varied the statistical structure of new stories while keeping their relational structure intact. The performance of each model fell below chance at least under one manipulation. We argue that both models fail our tests because they can't perform dynamic binding. These results cast doubts on the suitability of traditional neural networks for explaining relational reasoning and language processing phenomena

Relation learning and reasoning on computational models of high level cognition

Relational reasoning is central to many cognitive processes, ranging from “lower” processes like object recognition to “higher” processes such as analogy-making and sequential decision-making. The first chapter of this thesis gives an overview of relational reasoning and the computational demands that it imposes on a system that performs relational reasoning. These demands are characterized in terms of the binding problem in neural networks. There has been a longstanding debate in the literature regarding whether neural network models of cognition are, in principle, capable of relation-base processing. In the second chapter I investigated the relational reasoning capabilities of the Story Gestalt model (St. John, 1992), a classic connectionist model of text comprehension, and a Seq-to-Seq model, a deep neural network of text processing (Bahdanau, Cho, & Bengio, 2015). In both cases I found that the purportedly relational behavior of the models was explainable by the statistics of their training datasets. We propose that both models fail at relational processing because of the binding problem in neural networks. In the third chapter of this thesis, I present an updated version of the DORA architecture (Doumas, Hummel, & Sandhofer, 2008), a symbolic-connectionist model of relation learning and inference that uses temporal synchrony to solve the binding problem. We use this model to perform relational policy transfer between two Atari games. Finally, in the fourth chapter I present a model of relational reinforcement that is able to select relevant relations, from a potentially large pool of applicable relations, to characterize a problem and learn simple rules from the reward signal, helping to bridge the gap between reinforcement learning and relational reasoning

A theory of relation learning and cross-domain generalization

People readily generalize knowledge to novel domains and stimuli. We present a theory, instantiated in a computational model, based on the idea that cross-domain generalization in humans is a case of analogical inference over structured (i.e., symbolic) relational representations. The model is an extension of the LISA and DORA models of relational inference and learning. The resulting model learns both the content and format (i.e., structure) of relational representations from non-relational inputs without supervision, when augmented with the capacity for reinforcement learning, leverages these representations to learn individual domains, and then generalizes to new domains on the first exposure (i.e., zero-shot learning) via analogical inference. We demonstrate the capacity of the model to learn structured relational representations from a variety of simple visual stimuli, and to perform cross-domain generalization between video games (Breakout and Pong) and between several psychological tasks. We demonstrate that the model's trajectory closely mirrors the trajectory of children as they learn about relations, accounting for phenomena from the literature on the development of children's reasoning and analogy making. The model's ability to generalize between domains demonstrates the flexibility afforded by representing domains in terms of their underlying relational structure, rather than simply in terms of the statistical relations between their inputs and outputs.Comment: Includes supplemental materia

Present and future of surface-enhanced Raman scattering

The discovery of the enhancement of Raman scattering by molecules adsorbed on nanostructured metal surfaces is a landmark in the history of spectroscopic and analytical techniques. Significant experimental and theoretical effort has been directed toward understanding the surface-enhanced Raman scattering (SERS) effect and demonstrating its potential in various types of ultrasensitive sensing applications in a wide variety of fields. In the 45 years since its discovery, SERS has blossomed into a rich area of research and technology, but additional efforts are still needed before it can be routinely used analytically and in commercial products. In this Review, prominent authors from around the world joined together to summarize the state of the art in understanding and using SERS and to predict what can be expected in the near future in terms of research, applications, and technological development. This Review is dedicated to SERS pioneer and our coauthor, the late Prof. Richard Van Duyne, whom we lost during the preparation of this article

Association of rs712 polymorphism in a let-7 microRNA-binding site of KRAS gene with colorectal cancer in a Mexican population

Objective(s): The rs712 polymorphism in a let-7 microRNA-binding site at KRAS gene has been associated with cancer. To examine its association with rs712 polymorphism, we analyzed Mexican individuals with colorectal cancer (CRC) and healthy subjects. Materials and Methods: Genotyping of the rs712 polymorphism was performed by polymerase chain reaction in 281 controls and 336 CRC patients. Results: The observed frequencies of rs712 polymorphism indicated an associated protective factor for CRC (P=0.032). An association between genotype and the disease was evident in: colon localization (allele T, odds ratio (OR) 3.82, 95% confidence Intervals (CI) 2.77-5.28, P=0.0001), node metastasis (genotype TT, OR 2.49, 95% CI 1.45-4.28, P=0.0009), poor differentiation (genotype GT, OR 2.35, 95% CI 1.35-4.1, P=0.0033), and poor chemotherapy response (genotype GT, OR 2.6, 95% CI 1.7-4.24, P=0.0001). Conclusion: Comparison of the data from patients with control group showed that polymorphism of rs712 in KRAS gene was protective factor, which was associated with susceptibility for CRC. However, the genotypes TT and GT of rs712 polymorphism in KRAS could contribute significantly to colon localization, node metastasis, poor differentiation and poor chemotherapy response in CRC patients in this sample population

Clarifying status of DNNs as models of human vision

On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response

Present and Future of Surface-Enhanced Raman Scattering.

The discovery of the enhancement of Raman scattering by molecules adsorbed on nanostructured metal surfaces is a landmark in the history of spectroscopic and analytical techniques. Significant experimental and theoretical effort has been directed toward understanding the surface-enhanced Raman scattering (SERS) effect and demonstrating its potential in various types of ultrasensitive sensing applications in a wide variety of fields. In the 45 years since its discovery, SERS has blossomed into a rich area of research and technology, but additional efforts are still needed before it can be routinely used analytically and in commercial products. In this Review, prominent authors from around the world joined together to summarize the state of the art in understanding and using SERS and to predict what can be expected in the near future in terms of research, applications, and technological development. This Review is dedicated to SERS pioneer and our coauthor, the late Prof. Richard Van Duyne, whom we lost during the preparation of this article