Visual recognition of multi-agent action

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.Includes bibliographical references (p. 167-184).Developing computer vision sensing systems that work robustly in everyday environments will require that the systems can recognize structured interaction between people and objects in the world. This document presents a new theory for the representation and recognition of coordinated multi-agent action from noisy perceptual data. The thesis of this work is as follows: highly structured, multi-agent action can be recognized from noisy perceptual data using visually grounded goal-based primitives and low-order temporal relationships that are integrated in a probabilistic framework. The theory is developed and evaluated by examining general characteristics of multi-agent action, analyzing tradeoffs involved when selecting a representation for multi-agent action recognition, and constructing a system to recognize multi-agent action for a real task from noisy data. The representation, which is motivated by work in model-based object recognition and probabilistic plan recognition, makes four principal assumptions: (1) the goals of individual agents are natural atomic representational units for specifying the temporal relationships between agents engaged in group activities, (2) a high-level description of temporal structure of the action using a small set of low-order temporal and logical constraints is adequate for representing the relationships between the agent goals for highly structured, multi-agent action recognition, (3) Bayesian networks provide a suitable mechanism for integrating multiple sources of uncertain visual perceptual feature evidence, and (4) an automatically generated Bayesian network can be used to combine uncertain temporal information and compute the likelihood that a set of object trajectory data is a particular multi-agent action. The recognition algorithm is tested using a database of American football play descriptions. A system is described that can recognize single-agent and multi-agent actions in this domain given noisy trajectories of object movements. The strengths and limitations of the recognition system are discussed and compared with other multi-agent recognition algorithms.by Stephen Sean Intille.Ph.D

Tracking using a local closed-world assumption : tracking in the football domain

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1994.Includes bibliographical references (leaves 81-85).by Stephen Sean Intille.M.S

Data, Data Everywhere, and Still Too Hard to Link: Insights from User Interactions with Diabetes Apps

For those with chronic conditions, such as Type 1 diabetes, smartphone apps offer the promise of an affordable, convenient, and personalized disease management tool. How- ever, despite significant academic research and commercial development in this area, diabetes apps still show low adoption rates and underwhelming clinical outcomes. Through user-interaction sessions with 16 people with Type 1 diabetes, we provide evidence that commonly used interfaces for diabetes self-management apps, while providing certain benefits, can fail to explicitly address the cognitive and emotional requirements of users. From analysis of these sessions with eight such user interface designs, we report on user requirements, as well as interface benefits, limitations, and then discuss the implications of these findings. Finally, with the goal of improving these apps, we identify 3 questions for designers, and review for each in turn: current shortcomings, relevant approaches, exposed challenges, and potential solutions

Acquiring in situ training data for context-aware ubiquitous computing applications

Ubiquitous, context-aware computer systems may ultimately enable computer applications that naturally and usefully respond to a user's everyday activity. Although new algorithms that can automatically detect context from wearable and environmental sensor systems show promise, many of the most flexible and robust systems use probabilistic detection algorithms that require extensive libraries of training data with labeled examples. In this paper, we describe the need for such training data and some challenges we have identified when trying to collect it while testing three contextdetection systems for ubiquitous computing and mobile applications. Author Keywords Context-aware, ubiquitous, computing, supervised learning, experience sampling, user interface design ACM Classification Keywords H5.m Information interfaces and presentation (e.g. HCI): Miscellaneous

Designing for Diabetes Decision Support Systems with Fluid Contextual Reasoning

Type 1 diabetes is a potentially life-threatening chronic condition that requires frequent interactions with diverse data to inform treatment decisions. While mobile technolo- gies such as blood glucose meters have long been an essen- tial part of this process, designing interfaces that explicitly support decision-making remains challenging. Dual-process models are a common approach to understanding such cog- nitive tasks. However, evidence from the first of two stud- ies we present suggests that in demanding and complex situations, some individuals approach disease management in distinctive ways that do not seem to fit well within existing models. This finding motivated, and helped frame our second study, a survey (n=192) to investigate these behaviors in more detail. On the basis of the resulting analysis, we posit Fluid Contextual Reasoning to explain how some people with diabetes respond to particular situations, and discuss how an extended framework might help inform the design of user interfaces for diabetes management

Technological Innovations Enabling Automatic, Context-Sensitive Ecological Momentary Assessment

Health-related behavior, subjective states, cognitions, and interpersonal experiences are inextricably linked to context. Context includes information about location, time, past activities, interaction with other people and objects, and mental, physiological, and emotional states. Most real-time data collection methodologies require that subjects self-report information about contextual influences, notwithstanding the difficulty they have identifying the contextual factors that are influencing their behavior and subjective states. Often these assessment methodologies ask subjects to report on their activities or thoughts long after the actual events, thereby relying on retrospective recall and introducing memory biases. The “gold standard” alternative to these self-report instruments is direct observation. Direct observation in a laboratory setting, however, artificially constrains behavior. Direct observation is also typically too costly and invasive for long-term, large-sample-size studies of people in their natural environments.National Science Foundation (U.S.) (NSF ITR grant #0112900)Massachusetts Institute of Technology (House_n Consortium

The goal: smart people, not smart homes

Abstract. At MIT, a multi-disciplinary team of researchers from the House_n Consortium is studying how to create pervasive computing environments for the home. We are developing technologies and design strategies that use contextaware sensing to empower people by providing information when and where decisions and actions can be made. Contrary to many visions of future home environments in the literature, we advocate an approach that uses technology to teach as opposed to using technology primarily for automated control. We have designed and constructed a live-in laboratory (or “living laboratory”) that provides a unique, flexible infrastructure for scientifically studying the power of pervasive computing for motivating learning and behavior change in the home

Framework for recognizing multi-agent action from visual evidence,”

Abstract A probabilistic framework for representing and visually recognizing complex multi-agent action is presented. Motivated by work in model-based object recognition and designed for the recognition of action from visual evidence, the representation has three components: (1) temporal structure descriptions representing the temporal relationships between agent goals, (2) belief networks for probabilistically representing and recognizing individual agent goals from visual evidence, and (3) belief networks automatically generated from the temporal structure descriptions that support the recognition of the complex action. We describe our current work on recognizing American football plays from noisy trajectory data