Using Regular Languages to Explore the Representational Capacity of Recurrent Neural Architectures

The presence of Long Distance Dependencies (LDDs) in sequential data poses significant challenges for computational models. Various recurrent neural architectures have been designed to mitigate this issue. In order to test these state-of-the-art architectures, there is growing need for rich benchmarking datasets. However, one of the drawbacks of existing datasets is the lack of experimental control with regards to the presence and/or degree of LDDs. This lack of control limits the analysis of model performance in relation to the specific challenge posed by LDDs. One way to address this is to use synthetic data having the properties of subregular languages. The degree of LDDs within the generated data can be controlled through the k parameter, length of the generated strings, and by choosing appropriate forbidden strings. In this paper, we explore the capacity of different RNN extensions to model LDDs, by evaluating these models on a sequence of SPk synthesized datasets, where each subsequent dataset exhibits a longer degree of LDD. Even though SPk are simple languages, the presence of LDDs does have significant impact on the performance of recurrent neural architectures, thus making them prime candidate in benchmarking tasks.Comment: International Conference of Artificial Neural Networks (ICANN) 201

ON MONITORING LANGUAGE CHANGE WITH THE SUPPORT OF CORPUS PROCESSING

One of the fundamental characteristics of language is that it can change over time. One method to monitor the change is by observing its corpora: a structured language documentation. Recent development in technology, especially in the field of Natural Language Processing allows robust linguistic processing, which support the description of diverse historical changes of the corpora. The interference of human linguist is inevitable as it determines the gold standard, but computer assistance provides considerable support by incorporating computational approach in exploring the corpora, especially historical corpora. This paper proposes a model for corpus development, where corpus are annotated to support further computational operations such as lexicogrammatical pattern matching, automatic retrieval and extraction. The corpus processing operations are performed by local grammar based corpus processing software on a contemporary Indonesian corpus. This paper concludes that data collection and data processing in a corpus are equally crucial importance to monitor language change, and none can be set aside

Reenactment: An embodied cognition approach to meaning and linguistic content

A central finding in experimental research identified with Embodied Cognition (EC) is that understanding actions involves their embodied simulation, i.e. executing some processes involved in performing these actions. Extending these findings, I argue that reenactment – the overt embodied simulation of actions and practices, including especially communicative actions and practices, within utterances – makes it possible to forge an integrated EC-based account of linguistic meaning. In particular, I argue: (a) that remote entities can be referred to by reenacting actions performed with them; (b) that the use of grammatical constructions can be conceived of as the reenactment of linguistic action routines; (c) that complex enunciational structures (reported speech, irony, etc.) involve a separate level of reenactment, on which characters are presented as interacting with one another within the utterance; (d) that the segmentation of long utterances into shorter units involves the reenactment of brief audience interventions between units; and (e) that the overall meaning of an utterance can be stated in reenactment terms. The notion of reenactment provides a conceptual framework for accounting for aspects of language that are usually thought to be outside the reach of EC in an EC framework, thus supporting a view of meaning and linguistic content as thoroughly grounded in action and interaction