79 research outputs found
The relational processing limits of classic and contemporary neural network models of language processing
Whether neural networks can capture relational knowledge is a matter of long-standing controversy. Recently, some researchers have argued that (1) classic connectionist models can handle relational structure and (2) the success of deep learning approaches to natural language processing suggests that structured representations are unnecessary to model human language. We tested the Story Gestalt model, a classic connectionist model of text comprehension, and a Sequence-to-Sequence with Attention model, a modern deep learning architecture for natural language processing. Both models were trained to answer questions about stories based on abstract thematic roles. Two simulations varied the statistical structure of new stories while keeping their relational structure intact. The performance of each model fell below chance at least under one manipulation. We argue that both models fail our tests because they can't perform dynamic binding. These results cast doubts on the suitability of traditional neural networks for explaining relational reasoning and language processing phenomena
Relation learning and reasoning on computational models of high level cognition
Relational reasoning is central to many cognitive processes, ranging from âlowerâ processes
like object recognition to âhigherâ processes such as analogy-making and sequential decision-making. The first chapter of this thesis gives an overview of relational reasoning and the
computational demands that it imposes on a system that performs relational reasoning. These
demands are characterized in terms of the binding problem in neural networks. There has
been a longstanding debate in the literature regarding whether neural network models of
cognition are, in principle, capable of relation-base processing. In the second chapter I investigated the relational reasoning capabilities of the Story Gestalt model (St. John, 1992), a
classic connectionist model of text comprehension, and a Seq-to-Seq model, a deep neural
network of text processing (Bahdanau, Cho, & Bengio, 2015). In both cases I found that the
purportedly relational behavior of the models was explainable by the statistics of their training
datasets. We propose that both models fail at relational processing because of the binding
problem in neural networks. In the third chapter of this thesis, I present an updated version
of the DORA architecture (Doumas, Hummel, & Sandhofer, 2008), a symbolic-connectionist
model of relation learning and inference that uses temporal synchrony to solve the binding
problem. We use this model to perform relational policy transfer between two Atari games.
Finally, in the fourth chapter I present a model of relational reinforcement that is able to
select relevant relations, from a potentially large pool of applicable relations, to characterize
a problem and learn simple rules from the reward signal, helping to bridge the gap between
reinforcement learning and relational reasoning
A theory of relation learning and cross-domain generalization
People readily generalize knowledge to novel domains and stimuli. We present
a theory, instantiated in a computational model, based on the idea that
cross-domain generalization in humans is a case of analogical inference over
structured (i.e., symbolic) relational representations. The model is an
extension of the LISA and DORA models of relational inference and learning. The
resulting model learns both the content and format (i.e., structure) of
relational representations from non-relational inputs without supervision, when
augmented with the capacity for reinforcement learning, leverages these
representations to learn individual domains, and then generalizes to new
domains on the first exposure (i.e., zero-shot learning) via analogical
inference. We demonstrate the capacity of the model to learn structured
relational representations from a variety of simple visual stimuli, and to
perform cross-domain generalization between video games (Breakout and Pong) and
between several psychological tasks. We demonstrate that the model's trajectory
closely mirrors the trajectory of children as they learn about relations,
accounting for phenomena from the literature on the development of children's
reasoning and analogy making. The model's ability to generalize between domains
demonstrates the flexibility afforded by representing domains in terms of their
underlying relational structure, rather than simply in terms of the statistical
relations between their inputs and outputs.Comment: Includes supplemental materia
Present and future of surface-enhanced Raman scattering
The discovery of the enhancement of Raman scattering by molecules adsorbed on nanostructured metal surfaces is a landmark in the history of spectroscopic and analytical techniques. Significant experimental and theoretical effort has been directed toward understanding the surface-enhanced Raman scattering (SERS) effect and demonstrating its potential in various types of ultrasensitive sensing applications in a wide variety of fields. In the 45 years since its discovery, SERS has blossomed into a rich area of research and technology, but additional efforts are still needed before it can be routinely used analytically and in commercial products. In this Review, prominent authors from around the world joined together to summarize the state of the art in understanding and using SERS and to predict what can be expected in the near future in terms of research, applications, and technological development. This Review is dedicated to SERS pioneer and our coauthor, the late Prof. Richard Van Duyne, whom we lost during the preparation of this article
Association of rs712 polymorphism in a let-7 microRNA-binding site of KRAS gene with colorectal cancer in a Mexican population
Objective(s): The rs712 polymorphism in a let-7 microRNA-binding site at KRAS gene has been associated with cancer. To examine its association with rs712 polymorphism, we analyzed Mexican individuals with colorectal cancer (CRC) and healthy subjects. Materials and Methods: Genotyping of the rs712 polymorphism was performed by polymerase chain reaction in 281 controls and 336 CRC patients. Results: The observed frequencies of rs712 polymorphism indicated an associated protective factor for CRC (P=0.032). An association between genotype and the disease was evident in: colon localization (allele T, odds ratio (OR) 3.82, 95% confidence Intervals (CI) 2.77-5.28, P=0.0001), node metastasis (genotype TT, OR 2.49, 95% CI 1.45-4.28, P=0.0009), poor differentiation (genotype GT, OR 2.35, 95% CI 1.35-4.1, P=0.0033), and poor chemotherapy response (genotype GT, OR 2.6, 95% CI 1.7-4.24, P=0.0001). Conclusion: Comparison of the data from patients with control group showed that polymorphism of rs712 in KRAS gene was protective factor, which was associated with susceptibility for CRC. However, the genotypes TT and GT of rs712 polymorphism in KRAS could contribute significantly to colon localization, node metastasis, poor differentiation and poor chemotherapy response in CRC patients in this sample population
Clarifying status of DNNs as models of human vision
On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response
Present and Future of Surface-Enhanced Raman Scattering.
The discovery of the enhancement of Raman scattering by molecules adsorbed on nanostructured metal surfaces is a landmark in the history of spectroscopic and analytical techniques. Significant experimental and theoretical effort has been directed toward understanding the surface-enhanced Raman scattering (SERS) effect and demonstrating its potential in various types of ultrasensitive sensing applications in a wide variety of fields. In the 45 years since its discovery, SERS has blossomed into a rich area of research and technology, but additional efforts are still needed before it can be routinely used analytically and in commercial products. In this Review, prominent authors from around the world joined together to summarize the state of the art in understanding and using SERS and to predict what can be expected in the near future in terms of research, applications, and technological development. This Review is dedicated to SERS pioneer and our coauthor, the late Prof. Richard Van Duyne, whom we lost during the preparation of this article
- âŠ