Detection of tongue protrusion gestures from videos

We propose a system that, using video information, segments the mouth region from a face image and then detects the protrusion of the tongue from inside the oral cavity. Initially, under the assumption that the mouth is closed, we detect both mouth corners. We use a set of specifically oriented Gabor filters for enhancing horizontal features corresponding to the shadow existing between the upper and lower lips. After applying the Hough line detector, the extremes of the line that was found are regarded as the mouth corners. Detection rate for mouth corner localization is 85.33%. These points are then input to a mouth appearance model which fits a mouth contour to the image. By segmenting its bounding box we obtain a mouth template. Next, considering the symmetric nature of the mouth, we divide the template into right and left halves. Thus, our system makes use of three templates. We track the mouth in the following frames using normalized correlation for mouth template matching. Changes happening in the mouth region are directly described by the correlation value, i.e., the appearance of the tongue in the surface of the mouth will cause a decrease in the correlation coefficient through time. These coefficients are used for detecting the tongue protrusion. The right and left tongue protrusion positions will be detected by analyzing similarity changes between the right and left half-mouth templates and the currently tracked ones. Detection rates under the default parameters of our system are 90.20% for the tongue protrusion regardless of the position, and 84.78% for the right and left tongue protrusion positions. Our results demonstrate the feasibility of real-time tongue protrusion detection in vision-based systems and motivates further investigating the usage of this new modality in human-computer communication

The role of HCI in the construction of disability

As a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use, and with the study of major phenomena surrounding them, human computer interaction (HCI) is involved in the phenomenon of disability. For an interaction between humans and computers to take place, there should be an interface mediating between both parties. The design of such an interface may inadvertently impose access barriers to some people. HCI literature addresses the relationship between the theory and practice of HCI and disability from different angles, some of which are diametrically opposed. This thesis explores three modern conceptions, or models, of disability — the individualistic medical, the biopsychosocial and the social models —, investigates which model predominates in the HCI literature, and analyzes why choosing a particular model may determine and constrain the classes of problems that can be identified during a solution discovery process. Departing from HCI’s traditional discourse, which interprets the phenomenon of disability as a problem in the human body, the author, leading a team of engineers and psychologists, carried out a project in a school for children with cerebral palsy. The project was aimed to improve different areas of child development, using non conventional user interfaces — i.e. user interfaces that use other input/output devices than the keyboard, mouse or screen. After two years working directly within the field of operations”, the author had the opportunity to contrast the theory underpinning HCI’s methods with real practice and to expand his understandings about the relationships between HCI and disability. The research process involved an action research approach, which allowed the author and the team of experimenters to formulate new hypotheses as they learned more about the context, to review the process and, ultimately and most importantly,to readapt their actions to better serve the end beneficiaries. The experiences and learnings gathered throughout the process have been included in this thesis as a case study, for the purpose of helping HCI researchers embarking on projects relatable to the one described. Finally, the author urges the HCI community to update its discourse and to connect it with the vast literature related to modern conceptions of the phenomenon of disability