Critiquing: Effective Decision Support in Time-Critical Domains (Dissertation Proposal)

The effective communication of information is an important concern in the design of an expert consultation system. Several researchers have chosen to adopt a critiquing mode, in which the system evaluates and reacts to a solution proposed by the user rather than presenting its own solution. In this proposal, I present an architecture for a critiquing system that functions in real-time, during the process of developing and executing a management plan in time-critical situations. The architecture is able to take account of and reason about multiple, interacting goals and to identify critical errors in the proposed management plan. This architecture is being implemented as part of the TraumAID system for the management of patients with severe injuries

A Review of Reinforcement Learning for Natural Language Processing, and Applications in Healthcare

Reinforcement learning (RL) has emerged as a powerful approach for tackling complex medical decision-making problems such as treatment planning, personalized medicine, and optimizing the scheduling of surgeries and appointments. It has gained significant attention in the field of Natural Language Processing (NLP) due to its ability to learn optimal strategies for tasks such as dialogue systems, machine translation, and question-answering. This paper presents a review of the RL techniques in NLP, highlighting key advancements, challenges, and applications in healthcare. The review begins by visualizing a roadmap of machine learning and its applications in healthcare. And then it explores the integration of RL with NLP tasks. We examined dialogue systems where RL enables the learning of conversational strategies, RL-based machine translation models, question-answering systems, text summarization, and information extraction. Additionally, ethical considerations and biases in RL-NLP systems are addressed

Personalized Memory Transfer for Conversational Recommendation Systems

Dialogue systems are becoming an increasingly common part of many users\u27 daily routines. Natural language serves as a convenient interface to express our preferences with the underlying systems. In this work, we implement a full-fledged Conversational Recommendation System, mainly focusing on learning user preferences through online conversations. Compared to the traditional collaborative filtering setting where feedback is provided quantitatively, conversational users may only indicate their preferences at a high level with inexact item mentions in the form of natural language chit-chat. This makes it harder for the system to correctly interpret user intent and in turn provide useful recommendations to the user. To tackle the ambiguities in natural language conversations, we propose Personalized Memory Transfer (PMT) which learns a personalized model in an online manner by leveraging a key-value memory structure to distill user feedback directly from conversations. This memory structure enables the integration of prior knowledge to transfer existing item representations/preferences and natural language representations. We also implement a retrieval based response generation module, where the system in addition to recommending items to the user, also responds to the user, either to elicit more information regarding the user intent or just for a casual chit-chat. The experiments were conducted on two public datasets and the results demonstrate the effectiveness of the proposed approach

Developing attribute acquisition strategies in spoken dialogue systems via user simulation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 159-169).A spoken dialogue system (SDS) is an application that supports conversational interaction with a human to perform some task. SDSs are emerging as an intuitive and efficient means for accessing information. A critical barrier to their widespread deployment remains in the form of communication breakdown at strategic points in the dialogue, often when the user tries to supply a named entity from a large or open vocabulary set. For example, a weather system might know several thousand cities, but there is no easy way to inform the user about what those cities are. The system will likely misrecognize any unknown city as some known city. The inability of a system to acquire an unknown value can lead to unpredictable behavior by the system, as well as by the user. This thesis presents a framework for developing attribute acquisition strategies with a simulated user. We specifically focus on the acquisition of unknown city names in a flight domain, through a spell-mode subdialogue. Collecting data from real users is costly in both time and resources. In addition, our goal is to focus on situations that tend to occur sporadically in real dialogues, depending on the domain and the user's experience in that domain.(cont.) Therefore, we chose to employ user simulation, which would allow us to generate a large number of dialogues, and to configure the input as desired in order to exercise specific strategies. We present a novel method of utterance generation for user simulation, that exploits an existing corpus of real user dialogues, but recombines the utterances using an example-based, template approach. Items of interest not in the corpus, such as foreign or unknown cities, can be included by splicing in synthesized speech. This method allows us to produce realistic utterances by retaining the structural variety of real user utterances, while introducing cities that can only be resolved via spelling. We also developed a model of generic dialogue management, allowing a developer to quickly specify interaction properties on a per-attribute basis. This model was used to assess the effectiveness of various combinations of dialogue strategies and simulated user behavior. Current approaches to user simulation typically model simulated utterances at the intention level, assuming perfect recognition and understanding. We employ speech to develop our strategies in the context of errors that occur naturally from recognition and understanding.(cont.) We use simulation to address two problems: the conflict problem requires the system to choose how to act when a new hypothesis for an attribute conflicts with its current belief, while the compliance problem requires the system to decide whether a user was compliant with a spelling request. Decision models were learned from simulated data, and were tested with real users, showing that the learned model significantly outperformed a heuristic model in choosing the "ideal" response to the conflict problem, with accuracies of 84.1% and 52.1%, respectively. The learned model to predict compliance achieved a respectable 96.3% accuracy. These results suggest that such models learned from simulated data can attain similar, if not better, performance in dialogues with real users.by Edward A. Filisko.Ph.D

Knowledge-Based Systems. Overview and Selected Examples

The Advanced Computer Applications (ACA) project builds on IIASA's traditional strength in the methodological foundations of operations research and applied systems analysis, and its rich experience in numerous application areas including the environment, technology and risk. The ACA group draws on this infrastructure and combines it with elements of AI and advanced information and computer technology to create expert systems that have practical applications. By emphasizing a directly understandable problem representation, based on symbolic simulation and dynamic color graphics, and the user interface as a key element of interactive decision support systems, models of complex processes are made understandable and available to non-technical users. Several completely externally-funded research and development projects in the field of model-based decision support and applied Artificial Intelligence (AI) are currently under way, e.g., "Expert Systems for Integrated Development: A Case Study of Shanxi Province, The People's Republic of China." This paper gives an overview of some of the expert systems that have been considered, compared or assessed during the course of our research, and a brief introduction to some of our related in-house research topics

Development of methodologies and procedures for identifying STS users and uses

A study was conducted to identify new uses and users of the new Space Transporation System (STS) within the domestic government sector. The study develops a series of analytical techniques and well-defined functions structured as an integrated planning process to assure efficient and meaningful use of the STS. The purpose of the study is to provide NASA with the following functions: (1) to realize efficient and economic use of the STS and other NASA capabilities, (2) to identify new users and uses of the STS, (3) to contribute to organized planning activities for both current and future programs, and (4) to air in analyzing uses of NASA's overall capabilities

Evaluating performance for procurement: A structured method for assessing the usability of future speech interfaces

Procurement is a process by which organizations acquire equipment to enhance the effectiveness of their operations. Equipment will only enhance effectiveness if it is usable for its purpose in the work environment, i.e. if it enables tasks to be performed to the desired quality with acceptable costs to those who operate it. Procurement presents a requirement, then, for evaluations of the performance of human-machine work systems. This thesis is concerned with the provision of information to support procurers in performing such evaluations. The Ministry of Defence (an equipment procurer) has presented a particular requirement for a means of assessing the usability of speech interfaces in the establishment of the feasibility of computerized battlefield work systems. A structured method was developed to meet this requirement, the scope, notation and process of which sought to be explicit and proceduralized. The scope was specified in terms of a conceptualization of human-computer interaction: the method supported the development of representations of the task, device and user, which could be implemented as simulations and used in empirical evaluations of system performance. Notations for representations were proposed, and procedures enabling the use of the notations. The specification and implementation of the four sub-methods is described, and subsequent enhancement in the context of evaluations of speech interfaces for battlefield observation tasks. The complete method is presented. An evaluation of the method was finally performed with respect to the quality of the assessment output and costs to the assessor. The results suggested that the method facilitated systematic assessment, although some inadequacies were identified in the expression of diagnostic information which was recruited by the procedures, and in some of the procedures themselves. The research offers support for the use of structured human factors evaluation methods in procurement. Qualifications relate to the appropriate expression of knowledge of device-user interaction, and to the conflict between requirements for flexibility and low-level proceduralization

CLiFF Notes: Research in the Language Information and Computation Laboratory of The University of Pennsylvania

This report takes its name from the Computational Linguistics Feedback Forum (CLIFF), an informal discussion group for students and faculty. However the scope of the research covered in this report is broader than the title might suggest; this is the yearly report of the LINC Lab, the Language, Information and Computation Laboratory of the University of Pennsylvania. It may at first be hard to see the threads that bind together the work presented here, work by faculty, graduate students and postdocs in the Computer Science, Psychology, and Linguistics Departments, and the Institute for Research in Cognitive Science. It includes prototypical Natural Language fields such as: Combinatorial Categorial Grammars, Tree Adjoining Grammars, syntactic parsing and the syntax-semantics interface; but it extends to statistical methods, plan inference, instruction understanding, intonation, causal reasoning, free word order languages, geometric reasoning, medical informatics, connectionism, and language acquisition. With 48 individual contributors and six projects represented, this is the largest LINC Lab collection to date, and the most diverse