45 research outputs found
Exploiting Deep Semantics and Compositionality of Natural Language for Human-Robot-Interaction
We develop a natural language interface for human robot interaction that
implements reasoning about deep semantics in natural language. To realize the
required deep analysis, we employ methods from cognitive linguistics, namely
the modular and compositional framework of Embodied Construction Grammar (ECG)
[Feldman, 2009]. Using ECG, robots are able to solve fine-grained reference
resolution problems and other issues related to deep semantics and
compositionality of natural language. This also includes verbal interaction
with humans to clarify commands and queries that are too ambiguous to be
executed safely. We implement our NLU framework as a ROS package and present
proof-of-concept scenarios with different robots, as well as a survey on the
state of the art
Recommended from our members
Sub-morphemic form-meaning systematicity: the impact of onset phones on wordconcreteness
Do individual sounds carry meaning? The relationship between sound and meaning in human languages is typicallyassumed to be arbitrary, though recent research provides evidence for the existence of both iconicity and systematicitybetween word forms and their meaning. However, this research has not asked whether individual sounds in a languagecovary in systematic ways with aspects of meaning. In two analyses, we find evidence for more systematicity betweenthe initial phones of words and those words concreteness ratings than one would expect in a truly arbitrary lexicon. Thissuggests that initial phones may act as cues to aspects of word meaning, and raises questions about whether languagelearners detect and exploit these cues
Recommended from our members
Do Multimodal Large Language Models and Humans Ground Language Similarly?
Abstract:
Large Language Models (LLMs) have been criticized for failing to connect linguistic meaning to the worldâfor failing to solve the âsymbol grounding problem.â Multimodal Large Language Models (MLLMs) offer a potential solution to this challenge by combining linguistic representations and processing with other modalities. However, much is still unknown about exactly how and to what degree MLLMs integrate their distinct modalitiesâand whether the way they do so mirrors the mechanisms believed to underpin grounding in humans. In humans, it has been hypothesized that linguistic meaning is grounded through âembodied simulation,â the activation of sensorimotor and affective representations reflecting described experiences. Across four pre-registered studies, we adapt experimental techniques originally developed to investigate embodied simulation in human comprehenders to ask whether MLLMs are sensitive to sensorimotor features that are implied but not explicit in descriptions of an event. In Experiment 1, we find sensitivity to some features (color and shape) but not others (size, orientation, and volume). In Experiment 2, we identify likely bottlenecks to explain an MLLMâs lack of sensitivity. In Experiment 3, we find that despite sensitivity to implicit sensorimotor features, MLLMs cannot fully account for human behavior on the same task. Finally, in Experiment 4, we compare the psychometric predictive power of different MLLM architectures and find that ViLT, a single-stream architecture, is more predictive of human responses to one sensorimotor feature (shape) than CLIP, a dual-encoder architectureâdespite being trained on orders of magnitude less data. These results reveal strengths and limitations in the ability of current MLLMs to integrate language with other modalities, and also shed light on the likely mechanisms underlying human language comprehension
Do Large Language Models know what humans know?
Humans can attribute beliefs to others. However, it is unknown to what extent
this ability results from an innate biological endowment or from experience
accrued through child development, particularly exposure to language describing
others' mental states. We test the viability of the language exposure
hypothesis by assessing whether models exposed to large quantities of human
language display sensitivity to the implied knowledge states of characters in
written passages. In pre-registered analyses, we present a linguistic version
of the False Belief Task to both human participants and a Large Language Model,
GPT-3. Both are sensitive to others' beliefs, but while the language model
significantly exceeds chance behavior, it does not perform as well as the
humans, nor does it explain the full extent of their behavior -- despite being
exposed to more language than a human would in a lifetime. This suggests that
while statistical learning from language exposure may in part explain how
humans develop the ability to reason about the mental states of others, other
mechanisms are also responsible
A pre-registered, multi-lab non-replication of the Action-sentence Compatibility Effect (ACE)
The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g., toward) matches the direction of the action in the to-be-judged sentence (e.g., Art gave you the pen describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.Fil: Morey, Richard. Cardiff University; Reino UnidoFil: Kaschak, Michael. Florida State University; Estados UnidosFil: DĂez Ălamo, Antonio. Universidad de Salamanca; España. Arizona State University; Estados UnidosFil: Glenberg, Arthur. Arizona State University; Estados Unidos. Universidad de Salamanca; EspañaFil: Zwaan, Rolf A.. Erasmus University Rotterdam; PaĂses BajosFil: Lakens, DaniĂ«l. Eindhoven University of Technology; PaĂses BajosFil: Ibåñez, Santiago AgustĂn. Consejo Nacional de Investigaciones CientĂficas y TĂ©cnicas; Argentina. Universidad de San AndrĂ©s; Argentina. University of San Francisco; Estados Unidos. Universidad Adolfo Ibañez; Chile. Trinity College Dublin; IrlandaFil: GarcĂa, Adolfo MartĂn. Consejo Nacional de Investigaciones CientĂficas y TĂ©cnicas; Argentina. Universidad de San AndrĂ©s; Argentina. University of San Francisco; Estados Unidos. Universidad Nacional de Cuyo. Facultad de EducaciĂłn Elemental y Especial; Argentina. Universidad de Santiago de Chile; ChileFil: Gianelli, Claudia. Universitat Potsdam; Alemania. Scuola Universitaria Superiore; ItaliaFil: Jones, John L.. Florida State University; Estados UnidosFil: Madden, Julie. University of Tennessee; Estados UnidosFil: Alifano Ferrero, Florencia. Consejo Nacional de Investigaciones CientĂficas y TĂ©cnicas; ArgentinaFil: Bergen, Benjamin. University of California at San Diego; Estados UnidosFil: Bloxsom, Nicholas G.. Ashland University; Estados UnidosFil: Bub, Daniel N.. University of Victoria; CanadĂĄFil: Cai, Zhenguang G.. The Chinese University; Hong KongFil: Chartier, Christopher R.. Ashland University; Estados UnidosFil: Chatterjee, Anjan. University of Pennsylvania; Estados UnidosFil: Conwell, Erin. North Dakota State University; Estados UnidosFil: Wagner Cook, Susan. University of Iowa; Estados UnidosFil: Davis, Joshua D.. University of California at San Diego; Estados UnidosFil: Evers, Ellen R. K.. University of California at Berkeley; Estados UnidosFil: Girard, Sandrine. University of Carnegie Mellon; Estados UnidosFil: Harter, Derek. Texas A&m University Commerce; Estados UnidosFil: Hartung, Franziska. University of Pennsylvania; Estados UnidosFil: Herrera, Eduar. Universidad ICESI; ColombiaFil: Huettig, Falk. Max Planck Institute for Psycholinguistics; PaĂses BajosFil: Humphries, Stacey. University of Pennsylvania; Estados UnidosFil: Juanchich, Marie. University of Essex; Reino UnidoFil: KĂŒhne, Katharina. Universitat Potsdam; AlemaniaFil: Lu, Shulan. Texas A&m University Commerce; Estados UnidosFil: Lynes, Tom. University of East Anglia; Reino UnidoFil: Masson, Michael E. J.. University of Victoria; CanadĂĄFil: Ostarek, Markus. Max Planck Institute for Psycholinguistics; PaĂses BajosFil: Pessers, Sebastiaan. Katholikie Universiteit Leuven; BĂ©lgicaFil: Reglin, Rebecca. Universitat Potsdam; AlemaniaFil: Steegen, Sara. Katholikie Universiteit Leuven; BĂ©lgicaFil: Thiessen, Erik D.. University of Carnegie Mellon; Estados UnidosFil: Thomas, Laura E.. North Dakota State University; Estados UnidosFil: Trott, Sean. University of California at San Diego; Estados UnidosFil: Vandekerckhove, Joachim. University of California at Irvine; Estados UnidosFil: Vanpaeme, Wolf. Katholikie Universiteit Leuven; BĂ©lgicaFil: Vlachou, Maria. Katholikie Universiteit Leuven; BĂ©lgicaFil: Williams, Kristina. Texas A&m University Commerce; Estados UnidosFil: Ziv Crispel, Noam. BehavioralSight; Estados Unido
Evolution of the insecticide target Rdl in African Anopheles is driven by interspecific and interkaryotypic introgression.
The evolution of insecticide resistance mechanisms in natural populations of Anopheles malaria vectors is a major public health concern across Africa. Using genome sequence data, we study the evolution of resistance mutations in the resistance to dieldrin locus (Rdl), a GABA receptor targeted by several insecticides, but most notably by the long-discontinued cyclodiene, dieldrin. The two Rdl resistance mutations (296G and 296S) spread across West and Central African Anopheles via two independent hard selective sweeps that included likely compensatory nearby mutations, and were followed by a rare combination of introgression across species (from A. gambiae and A. arabiensis to A. coluzzii) and across non-concordant karyotypes of the 2La chromosomal inversion. Rdl resistance evolved in the 1950s as the first known adaptation to a large-scale insecticide-based intervention, but the evolutionary lessons from this system highlight contemporary and future dangers for management strategies designed to combat development of resistance in malaria vectors