Activity detection in conversational sign language video for mobile telecommunication

The goal of the MobileASL project is to increase accessibility by making the mobile telecommunications network available to the signing Deaf community. Video cell phones enable Deaf users to communicate in their native language, American Sign Language (ASL). However, encoding and transmission of real-time video over cell phones is a powerintensive task that can quickly drain the battery. By recognizing activity in the conversational video, we can drop the frame rate during less important segments without significantly harming intelligibility, thus reducing the computational burden. This recognition must take place from video in real-time on a cell phone processor, on users that wear no special clothing. In this work, we quantify the power savings from droppin

Linguistics As Structure In Computer Animation: Toward A More Effective Synthesis Of Brow Motion In American Sign Language

Computer-generated three-dimensional animation holds great promise for synthesizing utterances in American Sign Language (ASL) that are not only grammatical, but well tolerated by members of the Deaf community. Unfortunately, animation poses several challenges stemming from the necessity of grappling with massive amounts of data. However, the linguistics of ASL can aid in surmounting the challenge by providing structure and rules for organizing animation data. An exploration of the linguistic and extra linguistic behavior of the brows from an animator’s viewpoint yields a new approach for synthesizing nonmanuals that differs from the conventional animation of anatomy and instead offers a different approach for animating the effects of interacting levels of linguistic function. Results of formal testing with Deaf users have indicated that this is a promising approach

Video quality requirements for South African Sign Language communications over mobile phones.

Includes abstract.Includes bibliographical references.This project aims to find the minimum video resolution and frame rate that supports intelligible cell phone based video communications in South African Sign Language

A Computational Model Of The Intelligibility Of American Sign Language Video And Video Coding Applications

Real-time, two-way transmission of American Sign Language (ASL) video over cellular networks provides natural communication among members of the Deaf community. Bandwidth restrictions on cellular networks and limited computational power on cellular devices necessitate the use of advanced video coding techniques designed explicitly for ASL video. As a communication tool, compressed ASL video must be evaluated according to the intelligibility of the conversation, not according to conventional definitions of video quality. The intelligibility evaluation can either be performed using human subjects participating in perceptual experiments or using computational models suitable for ASL video. This dissertation addresses each of these issues in turn, presenting a computational model of the intelligibility of ASL video, which is demonstrated to be accurate with respect to true intelligibility ratings as provided by human subjects. The computational model affords the development of video compression techniques that are optimized for ASL video. Guided by linguistic principles and human perception of ASL, this dissertation presents a full-reference computational model of intelligibility for ASL (CIM-ASL) that is suitable for evaluating compressed ASL video. The CIM-ASL measures distortions only in regions relevant for ASL communication, using spatial and temporal pooling mechanisms that vary the contribution of distortions according to their relative impact on the intelligibility of the compressed video. The model is trained and evaluated using ground truth experimental data, collected in three separate perceptual studies. The CIM-ASL provides accurate estimates of subjective intelligibility and demonstrates statistically significant improvements over computational models traditionally used to estimate video quality. The CIM-ASL is incorporated into an H.264/AVC compliant video coding framework, creating a closed-loop encoding system optimized explicitly for ASL intelligibility. This intelligibility optimized coder achieves bitrate reductions between 10% and 42% without reducing intelligibility, when compared to a general purpose H.264/AVC encoder. The intelligibility optimized encoder is refined by introducing reduced complexity encoding modes, which yield a 16% improvement in encoding speed. The purpose of the intelligibility optimized encoder is to generate video that is suitable for real-time ASL communication. Ultimately, the preferences of ASL users determine the success of the intelligibility optimized coder. User preferences are explicitly evaluated in a perceptual experiment in which ASL users select between the intelligibility optimized coder and a general purpose video coder. The results of this experiment demonstrate that the preferences vary depending on the demographics of the participants and that a significant proportion of users prefer the intelligibility optimized coder

Designing to support impression management

This work investigates impression management and in particular impression management using ubiquitous technology. Generally impression management is the process through which people try to influence the impressions that others have about them. In particular, impression management focuses on the flow of information between a performer and his/her audience, with control over what is presented to whom being of the utmost importance when trying to create the appropriate impression. Ubiquitous technology has provided opportunities for individuals to present themselves to others. However, the disconnection between presenter and audience over both time and space can result in individuals being misrepresented. This thesis outlines two important areas when trying to control the impression one gives namely, hiding and revealing, and accountability. By exploring these two themes the continuous evolution and dynamic nature of controlling the impression one gives is explored. While this ongoing adaptation is recognised by designers they do not always create technology that is sufficiently dynamic to support this process. As a result, this work attempts to answer three research questions: RQ1: How do users of ubicomp systems appropriate recorded data from their everyday activity and make it into a resource for expressing themselves to others in ways that are dynamically tailored to their ongoing social context and audience? RQ2: What technology can be built to support ubicomp system developers to design and develop systems to support appropriation as a central part of a useful or enjoyable user experience? RQ3: What software architectures best suit this type of appropriated interaction and developers’ designing to support such interaction? Through a thorough review of existing literature, and the extensive study of several large ubicomp systems, the issues when presenting oneself through technology are identified. The main issues identified are hiding and revealing, and accountability. These are built into a framework that acts as a reference for designers wishing to support impression management. An architecture for supporting impression management has also been developed that conforms to this framework and its evolution is documented later in the thesis. A demonstration of this architecture in a multi-player mobile experience is subsequently presented