Indian Sign Language Recognition System for Differently-able People

Sign languages commonly develop in deaf communities, that can include interpreters and friends and families of deaf people as well as people who are deaf or hard of hearing themselves. Sign Language Recognition is one of the most growing fields of research today. There are Many new techniques that have been developed recently in these fields. Here in this paper, we will propose a system for conversion of Indian sign language to text using Open CV. OpenCV designed to generate motion template images that can be used to rapidly determine where that motion occurred, how that motion occurred, and in which direction it occurred. There is also support for static gesture recognition in OpenCV which can locate hand position and define orientation (right or left) in image and create hand mask image. In this we will use image processing in which captured image will be processed which are digital in nature by the digital computer. By this we will enhance the quality of a picture so that it looks better. Our aim is to design a human computer interface system that can recognize language of the deaf and dumb accurately

Human Skin Detection Using RGB, HSV and YCbCr Color Models

Human Skin detection deals with the recognition of skin-colored pixels and regions in a given image. Skin color is often used in human skin detection because it is invariant to orientation and size and is fast to process. A new human skin detection algorithm is proposed in this paper. The three main parameters for recognizing a skin pixel are RGB (Red, Green, Blue), HSV (Hue, Saturation, Value) and YCbCr (Luminance, Chrominance) color models. The objective of proposed algorithm is to improve the recognition of skin pixels in given images. The algorithm not only considers individual ranges of the three color parameters but also takes into ac- count combinational ranges which provide greater accuracy in recognizing the skin area in a given image.Comment: ICCASP/ICMMD-2016. Published by Atlantic Press. Part of series: AISR ISBN: 978-94-6252-305-0 ISSN: 1951-685

Multimodaalsel emotsioonide tuvastamisel põhineva inimese-roboti suhtluse arendamine

Väitekirja elektrooniline versioon ei sisalda publikatsiooneÜks afektiivse arvutiteaduse peamistest huviobjektidest on mitmemodaalne emotsioonituvastus, mis leiab rakendust peamiselt inimese-arvuti interaktsioonis. Emotsiooni äratundmiseks uuritakse nendes süsteemides nii inimese näoilmeid kui kakõnet. Käesolevas töös uuritakse inimese emotsioonide ja nende avaldumise visuaalseid ja akustilisi tunnuseid, et töötada välja automaatne multimodaalne emotsioonituvastussüsteem. Kõnest arvutatakse mel-sageduse kepstri kordajad, helisignaali erinevate komponentide energiad ja prosoodilised näitajad. Näoilmeteanalüüsimiseks kasutatakse kahte erinevat strateegiat. Esiteks arvutatakse inimesenäo tähtsamate punktide vahelised erinevad geomeetrilised suhted. Teiseks võetakse emotsionaalse sisuga video kokku vähendatud hulgaks põhikaadriteks, misantakse sisendiks konvolutsioonilisele tehisnärvivõrgule emotsioonide visuaalsekseristamiseks. Kolme klassifitseerija väljunditest (1 akustiline, 2 visuaalset) koostatakse uus kogum tunnuseid, mida kasutatakse õppimiseks süsteemi viimasesetapis. Loodud süsteemi katsetati SAVEE, Poola ja Serbia emotsionaalse kõneandmebaaside, eNTERFACE’05 ja RML andmebaaside peal. Saadud tulemusednäitavad, et võrreldes olemasolevatega võimaldab käesoleva töö raames loodudsüsteem suuremat täpsust emotsioonide äratundmisel. Lisaks anname käesolevastöös ülevaate kirjanduses väljapakutud süsteemidest, millel on võimekus tunda äraemotsiooniga seotud ̆zeste. Selle ülevaate eesmärgiks on hõlbustada uute uurimissuundade leidmist, mis aitaksid lisada töö raames loodud süsteemile ̆zestipõhiseemotsioonituvastuse võimekuse, et veelgi enam tõsta süsteemi emotsioonide äratundmise täpsust.Automatic multimodal emotion recognition is a fundamental subject of interest in affective computing. Its main applications are in human-computer interaction. The systems developed for the foregoing purpose consider combinations of different modalities, based on vocal and visual cues. This thesis takes the foregoing modalities into account, in order to develop an automatic multimodal emotion recognition system. More specifically, it takes advantage of the information extracted from speech and face signals. From speech signals, Mel-frequency cepstral coefficients, filter-bank energies and prosodic features are extracted. Moreover, two different strategies are considered for analyzing the facial data. First, facial landmarks' geometric relations, i.e. distances and angles, are computed. Second, we summarize each emotional video into a reduced set of key-frames. Then they are taught to visually discriminate between the emotions. In order to do so, a convolutional neural network is applied to the key-frames summarizing the videos. Afterward, the output confidence values of all the classifiers from both of the modalities are used to define a new feature space. Lastly, the latter values are learned for the final emotion label prediction, in a late fusion. The experiments are conducted on the SAVEE, Polish, Serbian, eNTERFACE'05 and RML datasets. The results show significant performance improvements by the proposed system in comparison to the existing alternatives, defining the current state-of-the-art on all the datasets. Additionally, we provide a review of emotional body gesture recognition systems proposed in the literature. The aim of the foregoing part is to help figure out possible future research directions for enhancing the performance of the proposed system. More clearly, we imply that incorporating data representing gestures, which constitute another major component of the visual modality, can result in a more efficient framework

Developing a Prototype to Translate Pakistan Sign Language into Text and Speech While Using Convolutional Neural Networking

The purpose of the study is to provide a literature review of the work done on sign language in Pakistan and the world. This study also provides a framework of an already developed prototype to translate Pakistani sign language into speech and text while using convolutional neural networking (CNN) to facilitate unimpaired teachers to bridge the communication gap among the deaf learners and unimpaired teachers. Due to the lack of sign language teaching, unimpaired teachers face difficulty in communicating with impaired learners. This communication gap can be filled with the help of this translation tool. Research indicates that a prototype has been evolved that can translate the English textual content into sign language and highlighted that there is a need for translation tool which can translate the signs into English text. The current study will provide an architectural framework of the Pakistani sign language to English text translation tool that how different components of technology like deep learning, convolutional neural networking, python, tensor Flow, and NumPy, InceptionV3 and transfer learning, eSpeak text to speech help in the development of a translation tool prototype. Keywords: Pakistan sign language (PSL), sign language (SL), translation, deaf, unimpaired, convolutional neural networking (CNN). DOI: 10.7176/JEP/10-15-18 Publication date:May 31st 201

Event Transformation for Browser Based Web Devices

Today a smartphone or tablet supports seven to eight ways by which user can interact with it. These interaction methods are touch, mouse, keyboard, voice, gestures, hover & stylus. Future is going towards IoE (Internet of everything) but if we really want to realize this vision then we need someone who can deal with these various existing and upcoming device interaction methods. This paper talks about a custom JavaScript library, which is accountable for registering native events coming from different event sources and maps it with the user defined key map to form a proper gesture. It is not a plain mapping because it takes care of many parameters like event state, occurrence, time interval of key press etc. If the events are coming from touch screen device then complexity increases many folds because forming a touch gesture involves all mathematical steps related to identification of swipe direction. Also in order to support the acceleration, its required to know till how long key was pressed and when it was released else no gesture will be formed and all events will be discarded. Based on device capability supported events could be discarded to completely knock off a device interaction method. It could be touch, mouse or key anything. This paper investigates heterogeneity of device interaction method events to form uniform gestures so that application developer need not to write code for each and every device interaction method