Tracking Hands Above Large Interactive Surfaces with a Low-Cost Scanning Laser Rangefinder

a set of receive electrodes placed about the display perimeter provided signals that corresponded to body distance. Although this system responded well enough to body dynamics and location for its original application, the limited disambiguation from four receive electrodes was unable to result in a repeatable hand tracker without excessively constraining the bodys posture and placement. We have developed an inexpensive scanning laser rangefinder to measure the real-time position of bare hands in a 2-D plane up to distances of several meters. We have used this device to build a precise, multipoint touch - screen interface for large video projection systems. In this paper, we describe the concepts and hardware, plus outline an application for an interactive multimedia environment. Other groups have implemented hand trackers using video cameras and computer vision techniques. Some [3] employ IR light sources and cameras behind a translucent rearprojected screen to see hands near the front, while others [4] use multiple cameras to observe a 2D gesture space. Like most vision approaches, the performance of these systems can suffer from background light (including light from the display itself in the latter case), image clutter, limited speed of response, and the need for multi-camera correspondence. Keywords Laser rangefinder, hand tracker, touchscreen, music interfac

The design of personal ambient displays

Thesis (S.M.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1999.Includes bibliographical references (leaves 58-59).The goal of this thesis is to investigate the design of personal ambient displays. These are small, physical devices worn to display information to a person in a subtle, persistent, and private manner. They can be small enough to be carried in a pocket, worn as a watch, or even adorned like jewelry. In my implementations, information is displayed solely through tactile modalities such as thermal change (heating and cooling), movement (shifting and vibration), and change of shape (expanding, contracting, and deformation). Using a tactile display allows information to be kept private and reduces the chance of overloading primary visual and auditory activities. The display can remain ambient, transmitting information in the background of a person's perception through simple, physical means. The specific focus of this thesis is to create a number of these tactile displays, to identify and implement applications they can serve, and to evaluate aspects of their effectiveness. I have created a group of small, wireless objects that can warm up and cool down or gently move or shift. Users can reconfigure each display so that information sources like stock data or the activity of people on the internet are mapped to these different tactile modalities. Furthermore, in this thesis I consider the implications that human perception have on the design of these displays and examine potential application areas for further implementations.Craig Alexander Wisneski.S.M

Direct interaction with large displays through monocular computer vision

Large displays are everywhere, and have been shown to provide higher productivity gain and user satisfaction compared to traditional desktop monitors. The computer mouse remains the most common input tool for users to interact with these larger displays. Much effort has been made on making this interaction more natural and more intuitive for the user. The use of computer vision for this purpose has been well researched as it provides freedom and mobility to the user and allows them to interact at a distance. Interaction that relies on monocular computer vision, however, has not been well researched, particularly when used for depth information recovery. This thesis aims to investigate the feasibility of using monocular computer vision to allow bare-hand interaction with large display systems from a distance. By taking into account the location of the user and the interaction area available, a dynamic virtual touchscreen can be estimated between the display and the user. In the process, theories and techniques that make interaction with computer display as easy as pointing to real world objects is explored. Studies were conducted to investigate the way human point at objects naturally with their hand and to examine the inadequacy in existing pointing systems. Models that underpin the pointing strategy used in many of the previous interactive systems were formalized. A proof-of-concept prototype is built and evaluated from various user studies. Results from this thesis suggested that it is possible to allow natural user interaction with large displays using low-cost monocular computer vision. Furthermore, models developed and lessons learnt in this research can assist designers to develop more accurate and natural interactive systems that make use of human’s natural pointing behaviours

Stretchable music : a graphically rich, interactive composition system

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1998.Includes bibliographical references (p. 120-122).by Peter W. Rice, Jr.S.M

Probabilistic characterization and synthesis of complex driven systems

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2000.Includes bibliographical references (leaves 194-204).Real-world systems that have characteristic input-output patterns but don't provide access to their internal states are as numerous as they are difficult to model. This dissertation introduces a modeling language for estimating and emulating the behavior of such systems given time series data. As a benchmark test, a digital violin is designed from observing the performance of an instrument. Cluster-weighted modeling (CWM), a mixture density estimator around local models, is presented as a framework for function approximation and for the prediction and characterization of nonlinear time series. The general model architecture and estimation algorithm are presented and extended to system characterization tools such as estimator uncertainty, predictor uncertainty and the correlation dimension of the data set. Furthermore a real-time implementation, a Hidden-Markov architecture, and function approximation under constraints are derived within the framework. CWM is then applied in the context of different problems and data sets, leading to architectures such as cluster-weighted classification, cluster-weighted estimation, and cluster-weighted sampling. Each application relies on a specific data representation, specific pre and post-processing algorithms, and a specific hybrid of CWM. The third part of this thesis introduces data-driven modeling of acoustic instruments, a novel technique for audio synthesis. CWM is applied along with new sensor technology and various audio representations to estimate models of violin-family instruments. The approach is demonstrated by synthesizing highly accurate violin sounds given off-line input data as well as cello sounds given real-time input data from a cello player.by Bernd Schoner.Ph.D

Supporting Situation Awareness and Workspace Awareness in Co-located Collaborative Systems Involving Dynamic Data

Co-located technologies can provide digital functionality to support collaborative work for multiple users in the same physical space. For example, digital tabletop computers — large interactive tables that allow users to directly interact with the content — can provide the most up-to-date map information while users can work together face-to-face. Combinations of interactive devices, large and small, can also be used together in a multi-device environment to support collaborative work of large groups. This environment allows individuals to utilize different networked devices. In some co-located group work, integrating automation into the available technologies can provide benefits such as automatically switching between different data views or updating map information based on underlying changes in deployed field agents’ locations. However, dynamic changes in the system state can create confusion for users and lead to low situation awareness. Furthermore, with the large size of a tabletop system or with multiple devices being used in the workspace, users may not be able to observe collaborators’ actions due to physical separations between users. Consequently, workspace awareness — knowledge of collaborators’ up-to-the-moment actions — can be difficult to maintain. As a result, users may be frustrated, and the collaboration may become inefficient or ineffective. The current tabletop applications involving dynamic data focus on interaction and information sharing techniques for collaboration rather than providing situation awareness support. Moreover, the situation awareness literature focuses primarily on single-user applications, whereas, the literature in workspace awareness primarily focuses on remote collaborative work. The aim of this dissertation was in supporting situation awareness of system-automated dynamic changes and workspace awareness of collaborators’ actions. The first study (Timeline Study) presented in this dissertation used tabletop systems to investigate supporting situation awareness of automated changes and workspace awareness, and the second study (Callout Bubble Study) followed up to further investigate workspace awareness support in the context of multi-device classrooms. Digital tabletop computers are increasingly being used for complex domains involving dynamic data, such as coastal surveillance and emergency response. Maintaining situation awareness of these changes driven by the system is crucial for quick and appropriate response when problems arise. However, distractors in the environment can make users miss the changes and negatively impact their situation awareness, e.g., the large size of the table and conversations with team members. As interactive event timelines have been shown to improve response time and decision accuracy after interruptions, in this dissertation they were adapted to the context of collaborative tabletop applications to address the lack of situation awareness due to dynamic changes. A user study was conducted to understand design factors related to the adaption and their impacts on situation awareness and workspace awareness. The Callout Bubble Study investigated workspace awareness support for multi-device classrooms, where students were co-located with their personal devices and were connected through a large shared virtual canvas. This context was chosen due to the environment’s ability to support work in large groups and the increasing prevalence of individual devices in co-located collaborative workspaces. By studying another co-located context, this research also sought to combine the lessons learned and provide a set of more generalized design recommendations for co-located technologies. Existing work on workspace awareness focuses on remote collaboration; however, the co-located users may not need all the information beneficial for remote work. This study aimed to balance awareness and distraction to improve students’ workspace awareness maintenance while minimizing distraction to their learning. A Callout Bubble was designed to augment students’ interactions in the shared online workspace, and a field study was conducted to understand how it impacted the students’ collaboration behaviour. Overall, the research presented in this dissertation aimed to investigate information visualizations for supporting situation awareness and workspace awareness in co-located collaborative environments. The contributions included the design of an interactive event timeline and an investigation of how the control placement (how many timelines and where they should be located) and feedback location (whether to display feedback to the group or to individuals when users interact with timelines) factors affected situation awareness. The empirical results revealed that individual timelines were more effective in facilitating situation awareness maintenance and the timelines were used mainly for perceiving new changes. Furthermore, this dissertation contributed in the design of a workspace awareness cue, Callout Bubble. The field study revealed that Callout Bubbles were effective in improving students’ coordination and self-monitoring behaviours, which in turn reduced teachers’ workloads. The dissertation provided overall design lessons learned for supporting awareness in co-located collaborative environments

Tangible interfaces for manipulating aggregates of digital information

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2002.Includes bibliographical references (p. 255-269).This thesis develops new approaches for people to physically represent and interact with aggregates of digital information. These support the concept of Tangible User Interfaces (TUIs), a genre of human-computer interaction that uses spatially reconfigurable physical objects as representations and controls for digital information. The thesis supports the manipulation of information aggregates through systems of physical tokens and constraints. In these interfaces, physical tokens act as containers and parameters for referencing digital information elements and aggregates. Physical constraints are then used to map structured compositions of tokens onto a variety of computational interpretations. This approach is supported through the design and implementation of several systems. The mediaBlocks system enables people to use physical blocks to "copy and paste" digital media between specialized devices and general-purpose computers, and to physically compose and edit this content (e.g., to build multimedia presentations). This system also contributes new tangible interface techniques for binding, aggregating, and disaggregating sequences of digital information into physical objects.(cont.) Tangible query interfaces allow people to physically express and manipulate database queries. This system demonstrates ways in which tangible interfaces can manipulate larger aggregates of information. One of these query approaches has been evaluated in a user study, which has compared favorably with a best-practice graphical interface alternative. These projects are used to support the claim that physically constrained tokens can provide an effective approach for interacting with aggregates of digital information.by Brygg Anders Ullmer.Ph.D