CLiFF Notes: Research In Natural Language Processing at the University of Pennsylvania

CLIFF is the Computational Linguists\u27 Feedback Forum. We are a group of students and faculty who gather once a week to hear a presentation and discuss work currently in progress. The \u27feedback\u27 in the group\u27s name is important: we are interested in sharing ideas, in discussing ongoing research, and in bringing together work done by the students and faculty in Computer Science and other departments. However, there are only so many presentations which we can have in a year. We felt that it would be beneficial to have a report which would have, in one place, short descriptions of the work in Natural Language Processing at the University of Pennsylvania. This report then, is a collection of abstracts from both faculty and graduate students, in Computer Science, Psychology and Linguistics. We want to stress the close ties between these groups, as one of the things that we pride ourselves on here at Penn is the communication among different departments and the inter-departmental work. Rather than try to summarize the varied work currently underway at Penn, we suggest reading the abstracts to see how the students and faculty themselves describe their work. The report illustrates the diversity of interests among the researchers here, as well as explaining the areas of common interest. In addition, since it was our intent to put together a document that would be useful both inside and outside of the university, we hope that this report will explain to everyone some of what we are about

Hearing meanings: the revenge of context

According to the perceptual view of language comprehension, listeners typically recover high-level linguistic properties such as utterance meaning without inferential work. The perceptual view is subject to the Objection from Context: since utterance meaning is massively context-sensitive, and context-sensitivity requires cognitive inference, the perceptual view is false. In recent work, Berit Brogaard provides a challenging reply to this objection. She argues that in language comprehension context-sensitivity is typically exercised not through inferences, but rather through top-down perceptual modulations or perceptual learning. This paper provides a complete formulation of the Objection from Context and evaluates Brogaards reply to it. Drawing on conceptual considerations and empirical examples, we argue that the exercise of context-sensitivity in language comprehension does, in fact, typically involve inference

Acts of reference and the miscommunication of referents by first and second language speakers of English

Central to communication are acts of reference, in which speakers clarify to hearers the identity of specific individuals (referents) through the use of referring expressions. Consequently, pragmatic competence in referring is likely to be an important skill for second language learners (SLLs). One of the complexities that speakers face is that the most appropriate referring expression (RE) on one occasion may be entirely inappropriate for that same individual on a different occasion. To explore pragmatic appropriateness in the use of REs, two approaches appear particularly apt. Firstly, Accessibility Theory (Ariel, 1990) holds that speakers select an RE type based on their best estimate of how recoverable the referent is in the addressee’s memory. Secondly, acts of reference may be most helpfully viewed as occurring across discoursal units above the level of the noun phrase. However, it appears that neither of these frameworks has been adopted in previous studies of SLL reference. In addition, it appears that little has been reported in relation to the factors that trigger referential miscommunication. This study examined references by relatively advanced SLLs and by L1 participants. The focus was on pragmatic appropriateness in marking accessibility, and factors implicated in referential miscommunication. A film retelling task was used to elicit data from participants in twenty SLL-L1 and ten L1-L1 dyads; following each retelling, a two-part stimulated recall interview was conducted with the addressee to identify miscommunications. Individual acts of reference were analyzed according to an accessibility coding system. Miscommunications were analyzed and likely triggers identified in terms of, for example, accessibility marking and NP errors. Findings are reported in relation to SLL referential competence, triggers of referential miscommunication, and some more general issues relating to the nature of reference. When introducing hearer-known referents into discourse, the SLL participants tended to be under-explicit for entities with low-accessibility but over-explicit for central characters. The least accessible entities were frequently either avoided or were under-explicit and, for this reason, were often miscommunicated, suggesting that these learners had difficulties constructing references across larger discourse units. In subsequent references (referent tracking), the SLLs tended to be over-explicit, particularly in certain contexts. It appears that this resulted partly from a communicative strategy and partly from cognitive load. The most frequent trigger of miscommunication in referent tracking was pronoun errors. More generally, I argue that the present approach to miscommunication analysis complements error analysis. The theoretical framework for the study involves a core definition of reference (adopted from Bach, 2008), which is then relaxed in two stages, thereby distinguishing three levels of reference. I argue that this framework enables principled definitions of reference and referential miscommunication, which partially bridge contrasting views found in philosophy and linguistics. The findings challenge three widespread assumptions about reference: (a) that all references require resolution, (b) that interactants attempt to clarify all references, and (c) that only highly accessible referents are felicitously referred to with pronouns. I propose a theory of referentiality in which speakers are held to signal the extent to which references require resolution

Use of Discourse Cues During Garden-Path Resolution is Modulated by Verb Argument Structure

Studies on garden-path sentences such as While the man hunted the deer ran into the woods have shown that comprehenders face processing difficulties due to the locally ambiguous noun phrase “the deer”. This critical noun phrase tends to be initially interpreted as the object of the preceding verb, but it must ultimately be interpreted as the subject of the following clause. This grammatical role ambiguity is particularly of interest because in English (and other languages) discourse information tends to be packaged in such a way that objects are typically indefinite, new information, and subjects are most often previously-mentioned, definite information (e.g., Comrie 1989, Prince 1992). We hypothesized that, if discourse information is at play, noun phrases that are more subject-like might facilitate garden-path resolution relative to more object-like noun phrases. However, in order to better understand when the discourse level is engaged in processing these constructions, we examined garden-paths with two verb types: reflexive absolute verbs (RATs, e.g., “wash”) and optionally transitive verbs (OPTs, e.g., “hunt”). Due to their reflexive nature, RATs were expected to operate mainly through a structural route (syntax-only). On the other hand, OPT verbs can introduce implicit arguments and are more likely to engage in operations beyond the domain of syntax (e.g., syntax+discourse). In this study, we discuss data from a self-paced reading experiment (see Besserman and Kaiser 2016) to shed light on the syntax/discourse division of labor: we found effects of information status related to subject-/object-hood in the processing of garden-paths with OPT, but not RAT verbs. These findings suggest that engagement of discourse representations was modulated by the verb’s argument structure

CLiFF Notes: Research in the Language, Information and Computation Laboratory of the University of Pennsylvania

One concern of the Computer Graphics Research Lab is in simulating human task behavior and understanding why the visualization of the appearance, capabilities and performance of humans is so challenging. Our research has produced a system, called Jack, for the definition, manipulation, animation and human factors analysis of simulated human figures. Jack permits the envisionment of human motion by interactive specification and simultaneous execution of multiple constraints, and is sensitive to such issues as body shape and size, linkage, and plausible motions. Enhanced control is provided by natural behaviors such as looking, reaching, balancing, lifting, stepping, walking, grasping, and so on. Although intended for highly interactive applications, Jack is a foundation for other research. The very ubiquitousness of other people in our lives poses a tantalizing challenge to the computational modeler: people are at once the most common object around us, and yet the most structurally complex. Their everyday movements are amazingly fluid, yet demanding to reproduce, with actions driven not just mechanically by muscles and bones but also cognitively by beliefs and intentions. Our motor systems manage to learn how to make us move without leaving us the burden or pleasure of knowing how we did it. Likewise we learn how to describe the actions and behaviors of others without consciously struggling with the processes of perception, recognition, and language. Present technology lets us approach human appearance and motion through computer graphics modeling and three dimensional animation, but there is considerable distance to go before purely synthesized figures trick our senses. We seek to build computational models of human like figures which manifest animacy and convincing behavior. Towards this end, we: Create an interactive computer graphics human model; Endow it with reasonable biomechanical properties; Provide it with human like behaviors; Use this simulated figure as an agent to effect changes in its world; Describe and guide its tasks through natural language instructions. There are presently no perfect solutions to any of these problems; ultimately, however, we should be able to give our surrogate human directions that, in conjunction with suitable symbolic reasoning processes, make it appear to behave in a natural, appropriate, and intelligent fashion. Compromises will be essential, due to limits in computation, throughput of display hardware, and demands of real-time interaction, but our algorithms aim to balance the physical device constraints with carefully crafted models, general solutions, and thoughtful organization. The Jack software is built on Silicon Graphics Iris 4D workstations because those systems have 3-D graphics features that greatly aid the process of interacting with highly articulated figures such as the human body. Of course, graphics capabilities themselves do not make a usable system. Our research has therefore focused on software to make the manipulation of a simulated human figure easy for a rather specific user population: human factors design engineers or ergonomics analysts involved in visualizing and assessing human motor performance, fit, reach, view, and other physical tasks in a workplace environment. The software also happens to be quite usable by others, including graduate students and animators. The point, however, is that program design has tried to take into account a wide variety of physical problem oriented tasks, rather than just offer a computer graphics and animation tool for the already computer sophisticated or skilled animator. As an alternative to interactive specification, a simulation system allows a convenient temporal and spatial parallel programming language for behaviors. The Graphics Lab is working with the Natural Language Group to explore the possibility of using natural language instructions, such as those found in assembly or maintenance manuals, to drive the behavior of our animated human agents. (See the CLiFF note entry for the AnimNL group for details.) Even though Jack is under continual development, it has nonetheless already proved to be a substantial computational tool in analyzing human abilities in physical workplaces. It is being applied to actual problems involving space vehicle inhabitants, helicopter pilots, maintenance technicians, foot soldiers, and tractor drivers. This broad range of applications is precisely the target we intended to reach. The general capabilities embedded in Jack attempt to mirror certain aspects of human performance, rather than the specific requirements of the corresponding workplace. We view the Jack system as the basis of a virtual animated agent that can carry out tasks and instructions in a simulated 3D environment. While we have not yet fooled anyone into believing that the Jack figure is real , its behaviors are becoming more reasonable and its repertoire of actions more extensive. When interactive control becomes more labor intensive than natural language instructional control, we will have reached a significant milestone toward an intelligent agent