ImageSpirit: Verbal Guided Image Parsing

Humans describe images in terms of nouns and adjectives while algorithms operate on images represented as sets of pixels. Bridging this gap between how humans would like to access images versus their typical representation is the goal of image parsing, which involves assigning object and attribute labels to pixel. In this paper we propose treating nouns as object labels and adjectives as visual attribute labels. This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images. We propose an efficient (interactive time) solution. Using the extracted labels as handles, our system empowers a user to verbally refine the results. This enables hands-free parsing of an image into pixel-wise object/attribute labels that correspond to human semantics. Verbally selecting objects of interests enables a novel and natural interaction modality that can possibly be used to interact with new generation devices (e.g. smart phones, Google Glass, living room devices). We demonstrate our system on a large number of real-world images with varying complexity. To help understand the tradeoffs compared to traditional mouse based interactions, results are reported for both a large scale quantitative evaluation and a user study.Comment: http://mmcheng.net/imagespirit

Integrating 2D Mouse Emulation with 3D Manipulation for Visualizations on a Multi-Touch Table

We present the Rizzo, a multi-touch virtual mouse that has been designed to provide the fine grained interaction for information visualization on a multi-touch table. Our solution enables touch interaction for existing mouse-based visualizations. Previously, this transition to a multi-touch environment was difficult because the mouse emulation of touch surfaces is often insufficient to provide full information visualization functionality. We present a unified design, combining many Rizzos that have been designed not only to provide mouse capabilities but also to act as zoomable lenses that make precise information access feasible. The Rizzos and the information visualizations all exist within a touch-enabled 3D window management system. Our approach permits touch interaction with both the 3D windowing environment as well as with the contents of the individual windows contained therein. We describe an implementation of our technique that augments the VisLink 3D visualization environment to demonstrate how to enable multi-touch capabilities on all visualizations written with the popular prefuse visualization toolkit.

Rethinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface

Multi-touch tabletops have been much heralded as an innovative technology that can facilitate new ways of group working. However, there is little evidence of these materialising outside of research lab settings. We present the findings of a 5-week in-the-wild study examining how a shared planning application – designed to run on a walk-up- and-use tabletop – was used when placed in a tourist information centre. We describe how groups approached, congregated and interacted with it and the social interactions that took place – noting how they were quite different from research findings describing the ways groups work around a tabletop in lab settings. We discuss the implications of such situated group work for designing collaborative tabletop applications for use in public settings

Beta: Bioprinting engineering technology for academia

Higher STEM education is a field of growing potential, but too many middle school and high school students are not testing proficiently in STEM subjects. The BETA team worked to improve biology classroom engagement through the development of technologies for high school biology experiments. The BETA project team expanded functionality of an existing product line to allow for better student and teacher user experience and the execution of more interesting experiments. The BETA project’s first goal was to create a modular incubating Box for the high school classroom. This Box, called the BETA Box was designed with a variety of sensors to allow for custom temperature and lighting environments for each experiment. It was completed with a clear interface to control the settings and an automatic image capture system. The team also conducted a feasibility study on auto calibration and dual-extrusion for SE3D’s existing 3D bioprinter. The findings of this study led to the incorporation of a force sensor for auto calibration and the evidence to support the feasibility of dual extrusion, although further work is needed. These additions to the current SE3D educational product line will increase effectiveness in the classroom and allow the target audience, high school students, to better engage in STEM education activities

Human-display interaction technology: Emerging remote interfaces for pervasive display environments

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.We're living in a world where information processing isn't confined to desktop computers - it's being integrated into everyday objects and activities. Pervasive computation is human centered: it permeates our physical world, helping us achieve goals and fulfill our needs with minimum effort by exploiting natural interaction styles. Remote interaction with screen displays requires a sensor-based, multimodal, touchless approach. For example, by processing user hand gestures, this paradigm removes constraints requiring physical contact and permits natural interaction with tangible digital information. Such touchless interaction can be multimodal, exploiting the visual, auditory, and olfactory senses.Ministerio de Educación y Ciencia and Amper Sistemas, SA