IDeixis : image-based deixis for recognizing locations

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.Includes bibliographical references (p. 31-32).In this thesis, we describe an approach to recognizing location from camera-equipped mobile devices using image-based web search. This is an image-based deixis capable of pointing at a distant location away from the user's current location. We demonstrate our approach on an application allowing users to browse web pages matching the image of a nearby location. Common image search metrics can match images captured with a camera-equipped mobile device to images found on the World Wide Web. The users can recognize the location if those pages contain information about this location (e.g. name, facts, stories ... etc). Since the amount of information displayable on the device is limited, automatic keyword extraction methods can be applied to help efficiently identify relevant pieces of location information. Searching the entire web can be computationally overwhelming, so we devise a hybrid image-and-keyword searching technique. First, image-search is performed over images and links to their source web pages in a database that indexes only a small fraction of the web. Then, relevant keywords on these web pages are automatically identified and submitted to an existing text-based search engine (e.g. Google) that indexes a much larger portion of the web. Finally, the resulting image set is filtered to retain images close to the original query in terms of visual similarity. It is thus possible to efficiently search hundreds of millions of images that are not only textually related but also visually relevant.by Pei-Hsiu Yeh.S.M

Pervasive Personal Information Spaces

Each user’s electronic information-interaction uniquely matches their information behaviour, activities and work context. In the ubiquitous computing environment, this information-interaction and the underlying personal information is distributed across multiple personal devices. This thesis investigates the idea of Pervasive Personal Information Spaces for improving ubiquitous personal information-interaction. Pervasive Personal Information Spaces integrate information distributed across multiple personal devices to support anytime-anywhere access to an individual’s information. This information is then visualised through context-based, flexible views that are personalised through user activities, diverse annotations and spontaneous information associations. The Spaces model embodies the characteristics of Pervasive Personal Information Spaces, which emphasise integration of the user’s information space, automation and communication, and flexible views. The model forms the basis for InfoMesh, an example implementation developed for desktops, laptops and PDAs. The design of the system was supported by a tool developed during the research called activity snaps that captures realistic user activity information for aiding the design and evaluation of interactive systems. User evaluation of InfoMesh elicited a positive response from participants for the ideas underlying Pervasive Personal Information Spaces, especially for carrying out work naturally and visualising, interpreting and retrieving information according to personalised contexts, associations and annotations. The user studies supported the research hypothesis, revealing that context-based flexible views may indeed provide better contextual, ubiquitous access and visualisation of information than current-day systems

Creating cohesive video with the narrative-informed use of ubiquitous wearable and imaging sensor networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2010.Page 232 blank. Cataloged from PDF version of thesis.Includes bibliographical references (p. 222-231).In today's digital era, elements of anyone's life can be captured, by themselves or others, and be instantly broadcast. With little or no regulation on the proliferation of camera technology and the increasing use of video for social communication, entertainment, and education, we have undoubtedly entered the age of ubiquitous media. A world permeated by connected video devices promises a more democratized approach to mass-media culture, enabling anyone to create and distribute personalized content. While these advancements present a plethora of possibilities, they are not without potential negative effects, particularly with regard to privacy, ownership, and the general decrease in quality associated with minimal barriers to entry. This dissertation presents a first-of-its-kind research platform designed to investigate the world of ubiquitous video devices in order to confront inherent problems and create new media applications. This system takes a novel approach to the creation of user-generated, documentary video by augmenting a network of video cameras integrated into the environment with on-body sensing. The distributed video camera network can record the entire life of anyone within its coverage range and it will be shown that it, almost instantly, records more audio and video than can be viewed without prohibitive human resource cost.(cont.) This drives the need to develop a mechanism to automatically understand the raw audiovisual information in order to create a cohesive video output that is understandable, informative, and/or enjoyable to its human audience. We address this need with the SPINNER system. As humans, we are inherently able to transform disconnected occurrences and ideas into cohesive narratives as a method to understand, remember, and communicate meaning. The design of the SPINNER application and ubiquitous sensor platform is informed by research into narratology, in other words how stories are created from fragmented events. The SPINNER system maps low level sensor data from the wearable sensors to higher level social signal and body language information. This information is used to label the raw video data. The SPINNER system can then build a cohesive narrative by stitching together the appropriately labeled video segments. The results from three test runs are shown, each resulting in one or more automatically edited video piece. The creation of these videos is evaluated through review by their intended audience and by comparing the system to a human trying to perform similar actions. In addition, the mapping of the wearable sensor data to meaningful information is evaluated by comparing the calculated results to those from human observation of the actual video.by Mathew Laibowitz.Ph.D

Interaction Methods for Smart Glasses : A Survey

Since the launch of Google Glass in 2014, smart glasses have mainly been designed to support micro-interactions. The ultimate goal for them to become an augmented reality interface has not yet been attained due to an encumbrance of controls. Augmented reality involves superimposing interactive computer graphics images onto physical objects in the real world. This survey reviews current research issues in the area of human-computer interaction for smart glasses. The survey first studies the smart glasses available in the market and afterwards investigates the interaction methods proposed in the wide body of literature. The interaction methods can be classified into hand-held, touch, and touchless input. This paper mainly focuses on the touch and touchless input. Touch input can be further divided into on-device and on-body, while touchless input can be classified into hands-free and freehand. Next, we summarize the existing research efforts and trends, in which touch and touchless input are evaluated by a total of eight interaction goals. Finally, we discuss several key design challenges and the possibility of multi-modal input for smart glasses.Peer reviewe

Statistics of the Self: Shaping the Self Through Quantified Self-Tracking

Self-tracking practices are growing in popularity worldwide. From heart-rate monitoring to mood tracking, many believe that wearable technologies are making their users more mindful in exclusively positive ways. However, I will argue that consistent and deliberate self-tracking (with or without portable devices) necessitates a particular understanding of the self with consequences that have yet to be fully explored. Through an analysis of forum posts on a popular self-tracking discussion and informational site, QuantifiedSelf.com, I will claim that self-trackers approach the creation of self-knowledge in a manner that is particular to today’s society. I will discuss how the ubiquitous conflation of numerical identities with objective reasoning feeds into a mindset that supports quantification of the self, and how the views of self exhibited by these self-trackers can be considered a version of creating a “scientific self.” The notion of the scientific self supports both an individual and societal shift in the practice of “being”—a shift that carries with it many possible repercussions that have yet to be widely analyzed. This analysis, I will argue, is key to limiting the destructive potential of understanding people in terms of data, while simultaneously enabling new conceptualizations of self to be practiced in modern society

An analysis of smartphone commercials: multimodality and frames

The goal of this paper is to present the cultural model linked to smartphones (and its users) created in video advertisement. The first part of the paper consists of a theoretical introduction presenting major linguistic terms important for our research as well as a short introduction to advertising. For the research, we have chosen twelve smartphone commercials (from seven smartphone brands) featured on YouTube and launched between the years 2010 and 2016. We will be analyzing how the cultural model is constructed through video advertisement using different modes. We will be analyzing which techniques the commercials use in promoting their product and to what extent the viewer is informed about the product and other products advertised together with the targeted one. We wish to know who the mobile phone companies’ targeted audience is and what persuasion techniques they prefer in advertising. When promoting the product or the brand, the commercials emphasize one or more key features. We will be analyzing the following elements: accessibility, innovation, lifestyle and information. We will discuss product placement, reference and the power of endorsement. This paper shows how commercials make use of conceptual metaphor, metonymy, conceptual integration and multimodality