Recurrent Memory Networks for Language Modeling

Recurrent Neural Networks (RNN) have obtained excellent result in many natural language processing (NLP) tasks. However, understanding and interpreting the source of this success remains a challenge. In this paper, we propose Recurrent Memory Network (RMN), a novel RNN architecture, that not only amplifies the power of RNN but also facilitates our understanding of its internal functioning and allows us to discover underlying patterns in data. We demonstrate the power of RMN on language modeling and sentence completion tasks. On language modeling, RMN outperforms Long Short-Term Memory (LSTM) network on three large German, Italian, and English dataset. Additionally we perform in-depth analysis of various linguistic dimensions that RMN captures. On Sentence Completion Challenge, for which it is essential to capture sentence coherence, our RMN obtains 69.2% accuracy, surpassing the previous state-of-the-art by a large margin.Comment: 8 pages, 6 figures. Accepted at NAACL 201

Weaving Rules into [email protected] for Embedded Smart Systems

Smart systems are characterised by their ability to analyse measured data in live and to react to changes according to expert rules. Therefore, such systems exploit appropriate data models together with actions, triggered by domain-related conditions. The challenge at hand is that smart systems usually need to process thousands of updates to detect which rules need to be triggered, often even on restricted hardware like a Raspberry Pi. Despite various approaches have been investigated to efficiently check conditions on data models, they either assume to fit into main memory or rely on high latency persistence storage systems that severely damage the reactivity of smart systems. To tackle this challenge, we propose a novel composition process, which weaves executable rules into a data model with lazy loading abilities. We quantitatively show, on a smart building case study, that our approach can handle, at low latency, big sets of rules on top of large-scale data models on restricted hardware.Comment: pre-print version, published in the proceedings of MOMO-17 Worksho

Overcoming the Newtonian Paradigm: The Unfinished Project of Theoretical Biology from a Schellingian Perspective

Defending Robert Rosen’s claim that in every confrontation between physics and biology it is physics that has always had to give ground, it is shown that many of the most important advances in mathematics and physics over the last two centuries have followed from Schelling’s demand for a new physics that could make the emergence of life intelligible. Consequently, while reductionism prevails in biology, many biophysicists are resolutely anti-reductionist. This history is used to identify and defend a fragmented but progressive tradition of anti-reductionist biomathematics. It is shown that the mathematicoephysico echemical morphology research program, the biosemiotics movement, and the relational biology of Rosen, although they have developed independently of each other, are built on and advance this antireductionist tradition of thought. It is suggested that understanding this history and its relationship to the broader history of post-Newtonian science could provide guidance for and justify both the integration of these strands and radically new work in post-reductionist biomathematics