Natural language generation as neural sequence learning and beyond

Natural Language Generation (NLG) is the task of generating natural language (e.g., English sentences) from machine readable input. In the past few years, deep neural networks have received great attention from the natural language processing community due to impressive performance across different tasks. This thesis addresses NLG problems with deep neural networks from two different modeling views. Under the first view, natural language sentences are modelled as sequences of words, which greatly simplifies their representation and allows us to apply classic sequence modelling neural networks (i.e., recurrent neural networks) to various NLG tasks. Under the second view, natural language sentences are modelled as dependency trees, which are more expressive and allow to capture linguistic generalisations leading to neural models which operate on tree structures. Specifically, this thesis develops several novel neural models for natural language generation. Contrary to many existing models which aim to generate a single sentence, we propose a novel hierarchical recurrent neural network architecture to represent and generate multiple sentences. Beyond the hierarchical recurrent structure, we also propose a means to model context dynamically during generation. We apply this model to the task of Chinese poetry generation and show that it outperforms competitive poetry generation systems. Neural based natural language generation models usually work well when there is a lot of training data. When the training data is not sufficient, prior knowledge for the task at hand becomes very important. To this end, we propose a deep reinforcement learning framework to inject prior knowledge into neural based NLG models and apply it to sentence simplification. Experimental results show promising performance using our reinforcement learning framework. Both poetry generation and sentence simplification are tackled with models following the sequence learning view, where sentences are treated as word sequences. In this thesis, we also explore how to generate natural language sentences as tree structures. We propose a neural model, which combines the advantages of syntactic structure and recurrent neural networks. More concretely, our model defines the probability of a sentence by estimating the generation probability of its dependency tree. At each time step, a node is generated based on the representation of the generated subtree. We show experimentally that this model achieves good performance in language modeling and can also generate dependency trees

Bertsobot: gizaki-robot arteko komunikazio eta elkarrekintzarako portaerak

216 p.Bertsobot: Robot-Portaerak Gizaki-Robot Arteko Komunikazio eta ElkarrekintzanBertsotan aritzeko gaitasuna erakutsiko duen robot autonomoa garatzeada gure ikerketa-lanaren helburu behinena. Bere egitekoa, bertsoa osatzekoinstrukzioak ahoz jaso, hauek prozesatu eta ahalik eta bertsorik egokienaosatu eta kantatzea litzateke, bertsolarien oholtza gaineko adierazkortasunmaila erakutsiz gorputzarekin. Robot-bertsolariak, gizaki eta roboten artekoelkarrekintza eta komunikazioan aurrera egiteko modua jarri nahi luke, lengoaianaturala erabiliz robot-gizaki arteko bi noranzkoko komunikazioan

Winning, Losing, and Changing the Rules: The Rhetoric of Poetry Contests and Competition

This dissertation attempts to trace the shifting relationship between the fields of Rhetoric and Poetry in Western culture by focusing on poetry contests and competitions during several different historical eras. In order to examine how the distinction between the two fields is contingent on a variety of local factors, this study makes use of research in contemporary cognitive neuroscience, particularly work in categorization and cognitive linguistics, to emphasize the provisional nature of conceptual thought; that is, on the type of mental activity that gives rise to conceptualizations such as “Rhetoric” and “Poetry.” The final portions of the research attempt to use some modeling techniques derived from cognitive linguistics as invention strategies for producing stylistically idiosyncratic academic knowledge, and for examining the relationship between the stylistic markers we associate with each of the two aforementioned fields

Raising awareness of frontotemporal dementia among Nigerian immigrant communities in the UK through storytelling : an autoethnography thesis using an art-based research approach

Even though medical research on dementia is wide and has long roots internationally, the awareness of the condition varies among different populations. People in ethnic minority communities, for example, may view dementia issues through a traditional or cultural lens. In these communities, diagnosis is more likely to occur at an advanced stage of the disease, and there is a low take-up of mainstream dementia services. This study explores new ways of raising awareness of dementia in such groups, in this case, among Nigerian immigrants in the UK. This group is understudied, even though they represent the largest number of people of African origin in the UK. The research questions set for the research are: (1) How can the awareness of frontotemporal dementia (FTD) be raised using an art-based approach? (2) What autoethnographic process preceded the development of the play ‘My Name is Beatrice’? My research approach is art-based, and the tool I used for my data interpretation is ethnodrama, which is a written transformation and adaptation of research data into a dramatic play script. I aim to present an aesthetically sound, intellectually rich, and emotionally evocative play that can capture my audience’s attention and leave them with enduring memories. The analysis focused on both the process that preceded the writing of a play about someone with dementia in a Nigerian immigrant community and the play itself. The data comprised two sets: my previous works and desktop research. These were analysed for their contribution to the process preceding the playwriting. The art-based part of this thesis included the play ‘My Name is Beatrice’ and its critical commentary. This research explores and discusses the efficacy of using drama as an educational tool to raise awareness of a disease. Art has an instantaneous effect on an audience because it can capture their attention and leave enduring memories. In addition, my research shows evidence of the complex needs of people living with dementia in Black Minority Ethnic (BME) communities that can be highlighted through art-based research and methods in a meaningful way. This art-based research has shown how ethnodrama can facilitate engagement and action from the researcher, participant, and audience. The aim is that this research would enlighten BME communities about FTD, the importance of early diagnosis and holistic approaches to care. The research will be a microcosm for further work that will enable educators and healthcare workers to share similar information within larger BME communities in the United Kingdom, other developed countries, and Africa. It will also enable educators and medical practitioners to understand the needs of BME communities and other similar groups worldwide

Reading and Rereading Shakespeare’s Sonnets: Combining Quantitative Narrative Analysis and Predictive Modeling

Natural reading is rather like a juggling feat, as our eyes and minds are kept on several things at the same time. Instead, reading texts developed by researchers (so-called “textoids”; Graesser, Millis, & Zwaan, 1997) may be fairly simple, since this facilitates an experimental investigation. It thus provides the chance for clear statements regarding the effect of predefined variables. Likewise, most empirical studies focused only a few selected features while ignoring the great diversity of possibly important others (e.g., Rayner et al., 2001; Reichle, Rayner, & Pollatsek, 2003; Rayner & Pollatsek, 2006; Engbert et al., 2005; Rayner, 2009). However, it is not possible to directly transfer the results generated from textoids to natural reading due to the identification of more than 100 features on different hierarchical levels, which may influence processing a natural text (Graf, Nagler, & Jacobs, 2005; Jacobs, 2015a, b; Jacobs et al., 2017). The present dissertation differed from past research in that it used a literary text, i.e., Shakespeare’s sonnets, instead of texts constructed by the experimenter. The goal of the present dissertation was to investigate how psycholinguistic features may influence the reading behavior during poem perception. To this end, two problems need to be handled: Firstly, complex natural texts need to be broken up into measurable and testable features by “turning words into numbers” (Franzosi, 2010) for the sake of statistical analysis. Secondly, statistical ways were sought to deal with the non-linear webs of correlations among different features, which has long been a concern of Jacob’s working group (e.g., Willems, 2015; Willems & Jacobs, 2016; Jacobs & Willems, 2018). A quantitative narrative analysis (QNA) based predictive modeling approach was suggested to solve the above problems (e.g., Jacobs et al., 2017; Jacobs, 2017, 2018a, b). Since it is impossible to identify all relevant features of a natural text [e.g., over 50 features mentioned for single word recognition (Graf et al., 2005) or over 100 features computed for the corpus of Shakespeare sonnets (Jacobs et al., 2017)] and including more inter/supra-lexical features also requires extending sample sizes (i.e., more/longer texts and more participants), my dissertation focuses on lexical features. Seven of these are surface features (word length, word frequency, orthographic neighborhood density, higher frequency neighbors, orthographic dissimilarity index, consonant vowel quotient, and the sonority score) and two are affective-semantic features (valence and arousal). By applying the QNA-based predictive modeling approach, I conducted three eye tracking studies: study 1 (Chapter 5) asked English native speakers to read three of Shakespeare’s sonnets (sonnet 27, 60, and 66), aiming to investigate the role of seven surface psycholinguistic features in sonnets reading. Study 2 (Chapter 6) used a rereading paradigm and let another group of English natives read two of the three sonnets (sonnet 27 and 66), to find out whether the roles of the surface psycholinguistic features may be changed in rereading. In study 3 (Chapter 7), I reanalyzed the data of study 2, in which beyond the surface features I started to pay attention to the affective-semantic features, hoping to examine whether the roles of surface and affective-semantic features may be different throughout reading sessions. The three studies show highly reliable data for high feature importance of surface variables, and in rereading an increasing impact of affective-semantic features in reading Shakespeare’s sonnets. From a methodological viewpoint, all three studies show a much better sufficiency of neural net approach than the classical general linear model approach in psycholinguistic eye tracking research. For the rereading studies, in general, compared to the first reading, rereading improved the fluency of reading on poem level (shorter total reading times, shorter regression times, and lower fixation probability) and the depth of comprehension (e.g., Hakemulder, 2004; Kuijpers & Hakemulder, 2018). Contrary to the other rereading studies using literary texts (e.g., Dixon et al., 1993; Millis, 1995; Kuijpers & Hakemulder, 2018), no increase in appreciation was apparent. In summary, this dissertation can show that the application of predictive modeling to investigate poetry might be far more suitable to capture the highly interactive, non-linear composition of linguistic features in natural texts that guide reading behavior and reception. Besides, surface features seem to influence reading during all reading sessions, while affective-semantic features seem to increase their importance in line with processing depth as indicated by higher influence during rereading. The results seem to be stable and valid as I could replicate these novel findings using machine learning algorithms within my dissertation project. My dissertation project is a first step towards a more differentiated picture of the guiding factors of poetry reception and a poetry specific reading model

Chinese whispers Chinese rooms: the poetry of John Ashbery and cognitive studies

This thesis examines the relationship of John Ashbery’s poetry to developments in cognitive studies over the course of the last sixty years, particularly the science of linguistics as viewed from a Chomskyan perspective. The thesis is divided into four chapters which position particular topics in cognitive studies as organising principles for examining Ashbery’s poetry. The first chapter concentrates on developments in syntactic theory in relation to Ashbery’s experiments with poetic syntax. The second chapter examines the notion of “intention” and “intentionality” in Ashbery’s writing from the perspective of cognitive “theory of context” writing, particularly the work of Deirdre Wilson and Daniel Sperber. The final two chapters consider cognitive questions using Ashbery’s poetry as a means of entry into controversial areas in formal cognitive studies. The third chapter examines his poetry in relation to temporality, suggesting that Ashbery’s experiments with time form “theories of consciousness” as they consciously manipulate readerly consciousness and attention. The final chapter explores perception in relation to Ashbery’s writing. The thesis argues that poetry can be conceived of as a less formalised method of cognitive study, and that poetic experiment can lead to significant reconceptualisations of cognitive notions which may play a role in framing critical questions for more formal experiments in cognitive science-philosophy going forward. The thesis concludes with reflections on the wider implications for literary cognitive studies in general

An Inquiry into the Creative Process of Butoh: With Reference to the Implications of Eastern and Western Significances

This thesis investigates the creative processes of Butoh choreography. A phenomenological perspective is used in this thesis to explore the implications for choreographers of the choreographic options employed in Butoh creative processes. Phenomenology corresponds to the Japanese philosophical context which gave birth to Butoh, elucidating the worldview merging with the universe which underpins Butoh. In terms of phenomenology, merging with the universe is firstly understood as a state of inter-subjectivity or transcendental consciousness, and secondly as the interweaving through actions of individual woridviews and a greater world of shared socio-cultural significances contributed by different people. The inter-subjective relationship between self, other self and objects is used to examine and elucidate the juxtaposition of various kinds of imagery in Butoh. The phenomenological notion of actions is used to examine bodily movement with respect to a greater world in Butoh. Phenomenology particularly corresponds to some of the fundamental processes used by Butoh choreographers. The choreographers' initial options for treating materials, namely visceral sensations as media for merging with the universe, texts as media for perception, paintings as media for presenting images, and actions as building blocks of an inter-subjective world, are inclined to guide the creative processes to develop the manifold of a spiritual imagery and bodily actions. These options and treatments are elucidated in this thesis through the network of perception and the phenomenological notion of graded fulfillment. The choreographer's treatment of the materials requires that the network of perception operates differently for different materials. The results of the operations are then integrated by the choreographer, through a process of graded fulfillment, into a holistic perception of the imagery or into a greater world, of which every image on stage is a part. In contrast the dialectical choreographic options, namely texts as tools for reasoning, paintings as representative structures of the subjects, and actions as representative units of social structure and cultural patterns, are inclined to guide the choreographers towards a focus on the development of formalised postures and gestures. The dialectical options are underpinned by rationalist, sociological or anthropological perspectives. It is argued that both the initial and dialectical options have value. The initial and dialectical options have co-existed and merged over the course of Butoh's development. Through their use different significances are incorporated into dance through the creative processes. Those significances can be identified as mainly rooted in Eastern philosophy, but later expanded to include Western philosophy when Butoh began to develop in a global context. Accordingly, Butoh creative processes are enriched by the use of a variety of choreographic options and by incorporating viewpoints from different people and perspectives

An evolutionary algorithm approach to poetry generation

Institute for Communicating and Collaborative SystemsPoetry is a unique artifact of the human language faculty, with its defining feature being a strong unity between content and form. Contrary to the opinion that the automatic generation of poetry is a relatively easy task, we argue that it is in fact an extremely difficult task that requires intelligence, world and linguistic knowledge, and creativity. We propose a model of poetry generation as a state space search problem, where a goal state is a text that satisfies the three properties of meaningfulness, grammaticality, and poeticness. We argue that almost all existing work on poetry generation only properly addresses a subset of these properties. In designing a computational approach for solving this problem, we draw upon the wealth of work in natural language generation (NLG). Although the emphasis of NLG research is on the generation of informative texts, recent work has highlighted the need for more flexible models which can be cast as one end of a spectrum of search sophistication, where the opposing end is the deterministically goal-directed planning of traditional NLG. We propose satisfying the properties of poetry through the application to NLG of evolutionary algorithms (EAs), a wellstudied heuristic search method. MCGONAGALL is our implemented instance of this approach. We use a linguistic representation based on Lexicalized Tree Adjoining Grammar (LTAG) that we argue is appropriate for EA-based NLG. Several genetic operators are implemented, ranging from baseline operators based on LTAG syntactic operations to heuristic semantic goal-directed operators. Two evaluation functions are implemented: one that measures the isomorphism between a solution’s stress pattern and a target metre using the edit distance algorithm, and one that measures the isomorphism between a solution’s propositional semantics and a target semantics using structural similarity metrics. We conducted an empirical study using MCGONAGALL to test the validity of employing EAs in solving the search problem, and to test whether our evaluation functions adequately capture the notions of semantic and metrical faithfulness. We conclude that our use of EAs offers an innovative approach to flexible NLG, as demonstrated by its successful application to the poetry generation task

Conceptual Representations for Computational Concept Creation

Computational creativity seeks to understand computational mechanisms that can be characterized as creative. The creation of new concepts is a central challenge for any creative system. In this article, we outline different approaches to computational concept creation and then review conceptual representations relevant to concept creation, and therefore to computational creativity. The conceptual representations are organized in accordance with two important perspectives on the distinctions between them. One distinction is between symbolic, spatial and connectionist representations. The other is between descriptive and procedural representations. Additionally, conceptual representations used in particular creative domains, such as language, music, image and emotion, are reviewed separately. For every representation reviewed, we cover the inference it affords, the computational means of building it, and its application in concept creation.Peer reviewe

Automatic Image Captioning with Style

This thesis connects two core topics in machine learning, vision and language. The problem of choice is image caption generation: automatically constructing natural language descriptions of image content. Previous research into image caption generation has focused on generating purely descriptive captions; I focus on generating visually relevant captions with a distinct linguistic style. Captions with style have the potential to ease communication and add a new layer of personalisation. First, I consider naming variations in image captions, and propose a method for predicting context-dependent names that takes into account visual and linguistic information. This method makes use of a large-scale image caption dataset, which I also use to explore naming conventions and report naming conventions for hundreds of animal classes. Next I propose the SentiCap model, which relies on recent advances in artificial neural networks to generate visually relevant image captions with positive or negative sentiment. To balance descriptiveness and sentiment, the SentiCap model dynamically switches between two recurrent neural networks, one tuned for descriptive words and one for sentiment words. As the first published model for generating captions with sentiment, SentiCap has influenced a number of subsequent works. I then investigate the sub-task of modelling styled sentences without images. The specific task chosen is sentence simplification: rewriting news article sentences to make them easier to understand. For this task I design a neural sequence-to-sequence model that can work with limited training data, using novel adaptations for word copying and sharing word embeddings. Finally, I present SemStyle, a system for generating visually relevant image captions in the style of an arbitrary text corpus. A shared term space allows a neural network for vision and content planning to communicate with a network for styled language generation. SemStyle achieves competitive results in human and automatic evaluations of descriptiveness and style. As a whole, this thesis presents two complete systems for styled caption generation that are first of their kind and demonstrate, for the first time, that automatic style transfer for image captions is achievable. Contributions also include novel ideas for object naming and sentence simplification. This thesis opens up inquiries into highly personalised image captions; large scale visually grounded concept naming; and more generally, styled text generation with content control