Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: N

Introspection as a method of identifying and describing competence in reading skills

Reading comprehension in English as a second language in the context of Iranian education system is not unproblematic. Hardly any studies have been attempted to investigate reading strategies and processes employed by novice and skilled readers through an on-line method of reading skills research in this context. The present study was thus undertaken to address the present need by employing think-aloud methodology to compare novice and skilled reading strategies. Therefore, a qualitative approach was taken to elicit as much information as possible for the purpose of identifying and describing competence in reading skills. The main research question addressed in this study deals with comparing strategy use of a group of novice second language EST readers studying academic English in Iran with another group of skilled second language EST readers from the same ethnic population but studying at the highest academic levels outside their mother land, viz. in Scotland. Several hypotheses were formed following a preliminary pilot study which included the following: a) there was a positive relationship between the number of strategies used by readers of each group and their performance on the TOEFL test; b) there are common areas in the readers' use of comprehension strategies which make the individual difference hypothesis in reading comprehension a debatable issue; c) the readers tend to follow an interactive approach to reading comprehension. Using an interactive model of reading seven categories of strategies were identified and classified. Non-significant correlation was obtained between number of strategies and language proficiency scores. Using a human information processing system, each reader's protocol was subjected to a detailed stage by stage analysis which supported the notion of the individual difference in reading comprehension. The readers also applied an interactive reading process to text comprehension

Going Beyond the Text: The Inferencing Processes of Skilled Readers in L1 and L2 across Reading Tasks

This small exploratory study investigated the inferencing processes of skilled first language (L1) and second language (L2) readers for two academic tasks. The goal was to examine possible effects of language and task, or reading purpose, on the frequency and distribution of inferences. Participants (n = 10) were native speakers of German enrolled at a large university in Hessen, Germany in a B.Ed. program. Participants read two expository texts (one written in German and the other written in English) in two task conditions: summary and position-paper. Think-aloud protocols while reading and stimulated recall immediately after reading were recorded, transcribed, coded, and the results were compared quantitatively and qualitatively across tasks and languages. The statistical analyses indicated that there were task effects on inferencing processes, and that they were stronger in L2. When reading for a summary purpose, inferencing processes differed across languages which was not the case for the position-paper task. Readers’ inferencing processes differed significantly across tasks in L2, but not in L1. The results suggest that skilled readers strategically inference based on academic task demands, but that transfer of strategic inferencing skills from L1 to L2 is not complete even with advanced L2 readers. Findings raise questions about the explicit instruction of strategic inferencing for academic tasks in L2 reading classrooms

Going Beyond the Text: The Inferencing Processes of Skilled Readers in L1 and L2 Across Reading Tasks

This small exploratory study investigated the inferencing processes of skilled first language (L1) and second language (L2) readers for two academic tasks. The goal was to examine possible effects of language and task, or reading purpose, on the frequency and distribution of inferences. Participants (n = 10) were native speakers of German enrolled at a large university in Hessen, Germany in a B.Ed. program. Participants read two expository texts (one written in German and the other written in English) in two task conditions: summary and position-paper. Think-aloud protocols while reading and stimulated recall immediately after reading were recorded, transcribed, coded, and the results were compared quantitatively and qualitatively across tasks and languages. The statistical analyses indicated that there were task effects on inferencing processes, and that they were stronger in L2. When reading for a summary purpose, inferencing processes differed across languages which was not the case for the position-paper task. Readers inferencing processes differed significantly across tasks in L2, but not in L1. The results suggest that skilled readers strategically inference based on academic task demands, but that transfer of strategic inferencing skills from L1 to L2 is not complete even with advanced L2 readers. Findings raise questions about the explicit instruction of strategic inferencing for academic tasks in L2 reading classrooms

Legal knowledge engineering: Computing, logic and law

The general problem approached in this thesis is that of building computer based legal advisory programs (otherwise known as expert systems or Intelligent Knowledge Based Systems). Such computer systems should be able to provide an individual with advice about either the general legal area being investigated, or advice about how the individual should proceed in a given case. In part the thesis describes a program (the ELl program) which attempts to confront some of the problems inherent in the building of these systems. The ELl system is seen as an experimental program (currently handling welfare rights legislation) and development vehicle. It is not presented as a final commercially implementable program. We present a detailed criticism of the type of legal knowledge contained within the system. The second, though in part intertwined, major subject of the thesis describes the jurisprudential aspects of the attempt to model the law by logic, a conjunction which is seen to be at the heart of the computer/law problem. We suggest that the conjunction offers very little to those who are interested in the real application of the real law, and that this is most forcefully seen when a working computer system models that conjunction. Our conclusion is that neither logic nor rule-based methods are sufficient for handling legal knowledge. The novelty and import of this thesis is not simply that it presents a negative conclusion; rather that it offers a sound theoretical and pragmatic framework for understanding why these methods are insufficient - the limits to the field are, in fact, defined

Sensitivity analysis in a scoping review on police accountability : assessing the feasibility of reporting criteria in mixed studies reviews

In this paper, we report on the findings of a sensitivity analysis that was carried out within a previously conducted scoping review, hoping to contribute to the ongoing debate about how to assess the quality of research in mixed methods reviews. Previous sensitivity analyses mainly concluded that the exclusion of inadequately reported or lower quality studies did not have a significant effect on the results of the synthesis. In this study, we conducted a sensitivity analysis on the basis of reporting criteria with the aims of analysing its impact on the synthesis results and assessing its feasibility. Contrary to some previous studies, our analysis showed that the exclusion of inadequately reported studies had an impact on the results of the thematic synthesis. Initially, we also sought to propose a refinement of reporting criteria based on the literature and our own experiences. In this way, we aimed to facilitate the assessment of reporting criteria and enhance its consistency. However, based on the results of our sensitivity analysis, we opted not to make such a refinement since many publications included in this analysis did not sufficiently report on the methodology. As such, a refinement would not be useful considering that researchers would be unable to assess these (sub-)criteria