Natural language processing

Beginning with the basic issues of NLP, this chapter aims to chart the major research activities in this area since the last ARIST Chapter in 1996 (Haas, 1996), including: (i) natural language text processing systems - text summarization, information extraction, information retrieval, etc., including domain-specific applications; (ii) natural language interfaces; (iii) NLP in the context of www and digital libraries ; and (iv) evaluation of NLP systems

Automatic summarising: factors and directions

This position paper suggests that progress with automatic summarising demands a better research methodology and a carefully focussed research strategy. In order to develop effective procedures it is necessary to identify and respond to the context factors, i.e. input, purpose, and output factors, that bear on summarising and its evaluation. The paper analyses and illustrates these factors and their implications for evaluation. It then argues that this analysis, together with the state of the art and the intrinsic difficulty of summarising, imply a nearer-term strategy concentrating on shallow, but not surface, text analysis and on indicative summarising. This is illustrated with current work, from which a potentially productive research programme can be developed

Steering textual reasoning with explanations

Recent breakthroughs in pretraining have significantly extended the boundaries of language models' (LMs') potential applications, partially because of their increased ability to do complex reasoning. However, LMs have well-documented reasoning failures, such as hallucinations and inability to systematically generalize. In this dissertation, we aim to steer LMs in reliably performing textual reasoning, with a particular focus on leveraging explanations. We describe our work on steering textual reasoning with explanations in two paradigms: 1) intervening on model predictions post-hoc by using explanations from LMs to verify their predictions, and 2) teaching LMs to reason by demonstrating the reasoning process of solving reasoning tasks to them. We first introduce how to leverage post-hoc explanations for intervening on model predictions. Past work has attempted to use post-hoc explanations for interpreting and debugging model behavior but often heavily relies on human effort. We focus instead on automating the process of using explanations to improve model predictions. Through a case study on QA models, we show that pairwise interaction-based explanation techniques align well with QA model behavior on counterfactuals, highlighting the connection between explanations and model behavior. This motivates us to introduce a framework for automatically assessing the robustness of black-box models using explanations. The framework first extracts features to describe the "reasoning process" disclosed by the explanations, and then uses a trained verifier to judge the reliability of predictions based on these features. Using our framework, we successfully improve two classes of models on diverse tasks spanning QA, NLI, and commonsense reasoning: BERT-based models, improved using attributions, and GPT-3-based in-context learning, using free-text explanations. We further study how to use explanations for teaching LMs to reason, especially free-text explanations for large language models (LLMs). We show that the performance of LLMs on downstream tasks is sensitive to the choice of explanations (among varied possible explanations) provided to them. We therefore propose methods for constructing effective explanations for LLMs. We introduce an approach that automatically optimizes explanations using unlabeled data, reducing the requirement of heavy manual prompt engineering. We also propose a framework that uses declarative formal specifications as explanations and employs an SMT solver to amend the limited planning capabilities of LLMs, which scales LLMs to handle problems requiring significantly deeper reasoning depth. Lastly, we outline future directions for further enhancing LLMs to better aid humans in challenging real-world applications demanding deep reasoning.Computer Science