A Neural Attention Model for Categorizing Patient Safety Events

Medical errors are leading causes of death in the US and as such, prevention of these errors is paramount to promoting health care. Patient Safety Event reports are narratives describing potential adverse events to the patients and are important in identifying and preventing medical errors. We present a neural network architecture for identifying the type of safety events which is the first step in understanding these narratives. Our proposed model is based on a soft neural attention model to improve the effectiveness of encoding long sequences. Empirical results on two large-scale real-world datasets of patient safety reports demonstrate the effectiveness of our method with significant improvements over existing methods.Comment: ECIR 201

Label-Dependencies Aware Recurrent Neural Networks

In the last few years, Recurrent Neural Networks (RNNs) have proved effective on several NLP tasks. Despite such great success, their ability to model \emph{sequence labeling} is still limited. This lead research toward solutions where RNNs are combined with models which already proved effective in this domain, such as CRFs. In this work we propose a solution far simpler but very effective: an evolution of the simple Jordan RNN, where labels are re-injected as input into the network, and converted into embeddings, in the same way as words. We compare this RNN variant to all the other RNN models, Elman and Jordan RNN, LSTM and GRU, on two well-known tasks of Spoken Language Understanding (SLU). Thanks to label embeddings and their combination at the hidden layer, the proposed variant, which uses more parameters than Elman and Jordan RNNs, but far fewer than LSTM and GRU, is more effective than other RNNs, but also outperforms sophisticated CRF models.Comment: 22 pages, 3 figures. Accepted at CICling 2017 conference. Best Verifiability, Reproducibility, and Working Description awar

Using Regular Languages to Explore the Representational Capacity of Recurrent Neural Architectures

The presence of Long Distance Dependencies (LDDs) in sequential data poses significant challenges for computational models. Various recurrent neural architectures have been designed to mitigate this issue. In order to test these state-of-the-art architectures, there is growing need for rich benchmarking datasets. However, one of the drawbacks of existing datasets is the lack of experimental control with regards to the presence and/or degree of LDDs. This lack of control limits the analysis of model performance in relation to the specific challenge posed by LDDs. One way to address this is to use synthetic data having the properties of subregular languages. The degree of LDDs within the generated data can be controlled through the k parameter, length of the generated strings, and by choosing appropriate forbidden strings. In this paper, we explore the capacity of different RNN extensions to model LDDs, by evaluating these models on a sequence of SPk synthesized datasets, where each subsequent dataset exhibits a longer degree of LDD. Even though SPk are simple languages, the presence of LDDs does have significant impact on the performance of recurrent neural architectures, thus making them prime candidate in benchmarking tasks.Comment: International Conference of Artificial Neural Networks (ICANN) 201

Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system

A number of representation schemes have been presented for use within learning classifier systems, ranging from binary encodings to neural networks. This paper presents results from an investigation into using discrete and fuzzy dynamical system representations within the XCSF learning classifier system. In particular, asynchronous random Boolean networks are used to represent the traditional condition-action production system rules in the discrete case and asynchronous fuzzy logic networks in the continuous-valued case. It is shown possible to use self-adaptive, open-ended evolution to design an ensemble of such dynamical systems within XCSF to solve a number of well-known test problems

Words, rules, and mechanisms of language acquisition

We review recent artificial language learning studies, especially those following Endress and Bonatti (2007), suggesting that humans can deploy a variety of learning mechanisms to acquire artificial languages. Several experiments provide evidence for multiple learning mechanisms that can be deployed in fluent speech: one mechanism encodes the positions of syllables within words and can be used to extract generalization, while the other registers co-occurrence statistics of syllables and can be used to break a continuum into its components. We review dissociations between these mechanisms and their potential role in language acquisition. We then turn to recent criticisms of the multiple mechanisms hypothesis and show that they are inconsistent with the available data. Our results suggest that artificial and natural language learning is best understood by dissecting the underlying specialized learning abilities, and that these data provide a rare opportunity to link important language phenomena to basic psychological mechanisms

Neurodevelopmental disorders

Recent technological advances allow us to measure how the infant brain functions in ways that were not possible just a decade ago. Although methodological advances are exciting, we must also consider how theories guide research: what we look for and how we explain what we find. Indeed, the ways in which research findings are interpreted affects the design of policies, educational practices, and interventions. Thus, the theoretical approaches adopted by scientists have a real impact on the lives of children with neurodevelopmental disorders (NDDs) and their families, as well as on the wider community. Here, we introduce and compare two theoretical approaches that are used to understand NDDs: the neuropsychological account and neuroconstructivism. We show how the former, adult account, is inadequate for explaining NDDs and illustrate this using the examples of Williams syndrome and specific language impairment. Neuroconstructivism, by contrast, focuses on the developing organism and is helping to change the way in which NDDs are investigated. Whereas neuropsychological static approaches assume that one or more ‘modules’ (e.g., visuospatial ability in Williams syndrome) are impaired while the rest of the system is spared (e.g., language in Williams syndrome), neuroconstructivism proposes that basic‐level deficits have subtle cascading effects on numerous domains over development. Neuroconstructivism leads researchers to embrace complexity by establishing large research consortia to integrate findings at multiple levels (e.g., genetic, neural, cognitive, environmental) across developmental time

A framework for Distributional Formal Semantics

Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at such a unified framework has, however, proven extremely challenging. One reason for this is that formal and distributional semantics operate on a fundamentally different `representational currency': formal semantics defines meaning in terms of models of the world, whereas distributional semantics defines meaning in terms of linguistic co-occurrence. Here, we pursue an alternative approach by deriving a vector space model that defines meaning in a distributed manner relative to formal models of the world. We will show that the resulting Distributional Formal Semantics offers probabilistic distributed representations that are also inherently compositional, and that naturally capture quantification and entailment. We moreover show that, when used as part of a neural network model, these representations allow for capturing incremental meaning construction and probabilistic inferencing. This framework thus lays the groundwork for an integrated distributional and formal approach to meaning

Look and Feel What and How Recurrent Self-Organizing Maps Learn

International audienceThis paper introduces representations and measurements for revealing the inner self-organization that occurs in a 1D recurrent self-organizing map. Experiments show the incredible richness and robustness of an extremely simple architecture when it extracts hidden states of the HMM that feeds it with ambiguous and noisy inputs

Ant-based Neural Topology Search (ANTS) for Optimizing Recurrent Networks

Hand-crafting effective and efficient structures for recurrent neural networks (RNNs) is a difficult, expensive, and time-consuming process. To address this challenge, we propose a novel neuro-evolution algorithm based on ant colony optimization (ACO), called Ant-based Neural Topology Search (ANTS), for directly optimizing RNN topologies. The procedure selects from multiple modern recurrent cell types such as ∆-RNN, GRU, LSTM, MGU and UGRNN cells, as well as recurrent connections which may span multiple layers and/or steps of time. In order to introduce an inductive bias that encourages the formation of sparser synaptic connectivity patterns, we investigate several variations of the core algorithm. We do so primarily by formulating different functions that drive the underlying pheromone simulation process (which mimic L1 and L2 regularization in standard machine learning) as well as by introducing ant agents with specialized roles (inspired by how real ant colonies operate), i.e., explorer ants that construct the initial feed forward structure and social ants which select nodes from the feed forward connections to subsequently craft recurrent memory structures. We also incorporate communal intelligence, where best weights are shared by the ant colony for weight initialization, reducing the number of backpropagation epochs required to locally train candidate RNNs, speeding up the neuro-evolution process. Our results demonstrate that the sparser RNNs evolved by ANTS significantly outperform traditional one and two layer architectures consisting of modern memory cells, as well as the well-known NEAT algorithm. Furthermore, we improve upon prior state-of-the-art results on the time series dataset utilized in our experiments

Hybrid of swarm intelligent algorithms in medical applications

In this paper, we designed a hybrid of swarm intelligence algorithms to diagnose hepatitis, breast tissue, and dermatology conditions in patients with such infection. The effectiveness of hybrid swarm intelligent algorithms was studied since no single algorithm is effective in solving all types of problems. In this study, feed forward and Elman recurrent neural network (ERN) with swarm intelligent algorithms is used for the classification of the mentioned diseases. The capabilities of six (6) global optimization learning algorithms were studied and their performances in training as well as testing were compared. These algorithms include: hybrid of Cuckoo Search algorithm and Levenberg-Marquardt (LM) (CSLM), Cuckoo Search algorithm (CS) and backpropagation (BP) (CSBP), CS and ERN (CSERN), Artificial Bee Colony (ABC) and LM (ABCLM), ABC and BP (ABCBP), Genetic Algorithm (GA) and BP (GANN). Simulation comparative results indicated that the classification accuracy and run time of the CSLM outperform the CSERN, GANN, ABCBP, ABCLM, and CSBP in the breast tissue dataset. On the other hand, the CSERN performs better than the CSLM, GANN, ABCBP, ABCLM, and CSBP in both th