Grounding for a computational model of place

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006.Text printed 2 columns per page.Includes bibliographical references (leaves 66-70).Places are spatial locations that have been given meaning by human experience. The sense of a place is it's support for experiences and the emotional responses associated with them. This sense provides direction and focus for our daily lives. Physical maps and their electronic decedents deconstruct places into discrete data and require user interpretation to reconstruct the original sense of place. Is it possible to create maps that preserve this sense of place and successfully communicate it to the user? This thesis presents a model, and an application upon that model, that captures sense of place for translation, rather then requires the user to recreate it from disparate data. By grounding a human place-sense for machine interpretation, new presentations of space can be presented that more accurately mirror human cognitive conceptions. By using measures of semantic distance a user can observe the proximity of place not only in distance but also by context or association. Applications built upon this model can then construct representations that show places that are similar in feeling or reasonable destinations given the user's current location.(cont.) To accomplish this, the model attempts to understand place in the context a human might by using commonsense reasoning to analyze textual descriptions of place, and implicit statements of support for the role of these places in natural activity. It produces a semantic description of a place in terms of human action and emotion. Representations built upon these descriptions can offer powerful changes in the cognitive processing of space.Matthew Curtis Hockenberry.S.M

GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents

Current gesture recognition systems primarily focus on identifying gestures within a predefined set, leaving a gap in connecting these gestures to interactive GUI elements or system functions (e.g., linking a 'thumb-up' gesture to a 'like' button). We introduce GestureGPT, a novel zero-shot gesture understanding and grounding framework leveraging large language models (LLMs). Gesture descriptions are formulated based on hand landmark coordinates from gesture videos and fed into our dual-agent dialogue system. A gesture agent deciphers these descriptions and queries about the interaction context (e.g., interface, history, gaze data), which a context agent organizes and provides. Following iterative exchanges, the gesture agent discerns user intent, grounding it to an interactive function. We validated the gesture description module using public first-view and third-view gesture datasets and tested the whole system in two real-world settings: video streaming and smart home IoT control. The highest zero-shot Top-5 grounding accuracies are 80.11% for video streaming and 90.78% for smart home tasks, showing potential of the new gesture understanding paradigm

Basic Human Values and Moral Foundations Theory in ValueNet Ontology

Values, as intended in ethics, determine the shape and validity of moral and social norms, grounding our everyday individual and community behavior on commonsense knowledge. The attempt to untangle human moral and social value-oriented structure of relations requires investigating both the dimension of subjective human perception of the world, and socio-cultural dynamics and multi-agent social interactions. Formalising latent moral content in human interaction is an appealing perspective that would enable a deeper understanding of both social dynamics and individual cognitive and behavioral dimension. To formalize this broad knowledge area, in the context of ValueNet, a modular ontology representing and operationalising moral and social values, we present two modules aiming at representing two main informal theories in literature: (i) the Basic Human Values theory by Shalom Schwartz and (ii) the Moral Foundations Theory by Graham and Haidt. ValueNet is based on reusable Ontology Design Patterns, is aligned to the DOLCE foundational ontology, and is a component of the Framester factual-linguistic knowledge graph

Contextual analysis: a multiperspective inquiry into emergence of complex socio-cultural systems

This paper explores the concept of organizations as complex human activity systems, through the perspectives of alternative systemic models. The impact of alternative models on perception of individual and organizational emergence is highlighted. Using information systems development as an example of management activity, individual and collective sense-making and learning processes are discussed. Their roles in relation to information systems concepts are examined. The main locus of the paper is on individual emergence in the context of organizational systems. A case is made for the importance of attending to individual uniqueness and contextual dependency when carrying out organizational analyses, e.g. information systems analysis. One particular method for contextual inquiry, the framework for Strategic Systemic Thinking, is then introduced, The framework supports stakeholders to own and control their own analyses. This approach provides a vehicle through which multiple levels of contextual dependencies can be explored and allows for individual emergence to develop

Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.Peer reviewe

Navigating Complex Search Tasks with AI Copilots

As many of us in the information retrieval (IR) research community know and appreciate, search is far from being a solved problem. Millions of people struggle with tasks on search engines every day. Often, their struggles relate to the intrinsic complexity of their task and the failure of search systems to fully understand the task and serve relevant results. The task motivates the search, creating the gap/problematic situation that searchers attempt to bridge/resolve and drives search behavior as they work through different task facets. Complex search tasks require more than support for rudimentary fact finding or re-finding. Research on methods to support complex tasks includes work on generating query and website suggestions, personalizing and contextualizing search, and developing new search experiences, including those that span time and space. The recent emergence of generative artificial intelligence (AI) and the arrival of assistive agents, or copilots, based on this technology, has the potential to offer further assistance to searchers, especially those engaged in complex tasks. There are profound implications from these advances for the design of intelligent systems and for the future of search itself. This article, based on a keynote by the author at the 2023 ACM SIGIR Conference, explores these issues and charts a course toward new horizons in information access guided by AI copilots.Comment: 10 pages, 6 figure