Intelligent Interfaces to Empower People with Disabilities

Severe motion impairments can result from non-progressive disorders, such as cerebral palsy, or degenerative neurological diseases, such as Amyotrophic Lateral Sclerosis (ALS), Multiple Sclerosis (MS), or muscular dystrophy (MD). They can be due to traumatic brain injuries, for example, due to a traffic accident, or to brainste

Smartphone-Based Escalator Recognition for the Visually Impaired

It is difficult for visually impaired individuals to recognize escalators in everyday environments. If the individuals ride on escalators in the wrong direction, they will stumble on the steps. This paper proposes a novel method to assist visually impaired individuals in finding available escalators by the use of smartphone cameras. Escalators are recognized by analyzing optical flows in video frames captured by the cameras, and auditory feedback is provided to the individuals. The proposed method was implemented on an Android smartphone and applied to actual escalator scenes. The experimental results demonstrate that the proposed method is promising for helping visually impaired individuals use escalators

Hand Gesture Based Surveillance Robot

In this work, a hardware and software based integrated system is developed for hand gesture based surveillance robot. The proposed system is a non-invasive technique and software part of the system uses gesture based image processing technique. The hardware part is developed based on AVR microcontroller platform. The captured image of hand is segmented and its contour is determined. The convexity defects are computed to detect the number of fingers used by the subject. The number of fingers directs the path to robot that is to be followed. The camera placed on the robot capture the images of its surrounding, wherever it travels and send it back to the PC for monitoring. In this way, it can be used as a surveillance system. Experimental results show that the overall accuracy obtained above 90% for gesture recognition by which robot will be directed to follow the path. The system can be directly applied to defence grounds for detection of enemy, for spying purpose where the human reach is avoided or not recommended. This unit can be used for overcoming physical handicaps by helping in development of gesture-based wheel chairs, for control of home devices and appliances for persons with physical handicaps and/or elderly users with impaired mobility

Video interaction using pen-based technology

Dissertação para obtenção do Grau de Doutor em InformáticaVideo can be considered one of the most complete and complex media and its manipulating is still a difficult and tedious task. This research applies pen-based technology to video manipulation, with the goal to improve this interaction. Even though the human familiarity with pen-based devices, how they can be used on video interaction, in order to improve it, making it more natural and at the same time fostering the user’s creativity is an open question. Two types of interaction with video were considered in this work: video annotation and video editing. Each interaction type allows the study of one of the interaction modes of using pen-based technology: indirectly, through digital ink, or directly, trough pen gestures or pressure. This research contributes with two approaches for pen-based video interaction: pen-based video annotations and video as ink. The first uses pen-based annotations combined with motion tracking algorithms, in order to augment video content with sketches or handwritten notes. It aims to study how pen-based technology can be used to annotate a moving objects and how to maintain the association between a pen-based annotations and the annotated moving object The second concept replaces digital ink by video content, studding how pen gestures and pressure can be used on video editing and what kind of changes are needed in the interface, in order to provide a more familiar and creative interaction in this usage context.This work was partially funded by the UTAustin-Portugal, Digital Media, Program (Ph.D. grant: SFRH/BD/42662/2007 - FCT/MCTES); by the HP Technology for Teaching Grant Initiative 2006; by the project "TKB - A Transmedia Knowledge Base for contemporary dance" (PTDC/EAT/AVP/098220/2008 funded by FCT/MCTES); and by CITI/DI/FCT/UNL (PEst-OE/EEI/UI0527/2011