Reconhecimento das configurações de mão da LIBRAS a partir de malhas 3D

Orientador: Prof. Dr. Daniel WeingaertnerDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Curso de Pós-Graduaçao em Informática. Defesa: Curitiba, 13/03/2013Bibliografia: fls. 68-73Resumo: O reconhecimento automático de sinais e um processo importante para uma boa utilização dos meios de comunicacão digitais por deficientes auditivos e, alem disso, favorece a comunicacao entre surdos e ouvintes que nao compreendem a língua de sinais. A abordagem de reconhecimento de sinais utilizada neste trabalho baseia-se nos parâmetros globais da LIBRAS - língua brasileira de sinais: configuracão de mão, locacao ou ponto de articulaçao, movimento, orientacao da palma da mao e expressão facial. A uniao de parâmetros globais forma sinais assim como fonemas formam palavras na língua falada. Este trabalho apresenta uma forma de reconhecer um dos parâmetros globais da LIBRAS, a configuracão de mao, a partir de malhas tridimensionais. A língua brasileira de sinais conta com 61 configuracoes de mao[16], este trabalho fez uso de uma base de dados contendo 610 vídeos de 5 usuarios distintos em duas tomadas, totalizando 10 capturas para cada configuracao de mao. De cada vídeo foram extraídos manualmente dois quadros retratando as visoes frontal e lateral da mao que, após segmentados e pré-processados, foram utilizados como entrada para o processamento de reconstrucao 3D. A geracao da malha 3D a partir das visães frontal e lateral da mão foi feita com o uso da tecnica de reconstruçao por silhueta[7]. O reconhecimento das configuracoes de mao a partir das malhas 3D foi feito com o uso do classificador SVM - Support Vector Machine. As características utilizadas para distinguir as malhas foram obtidas com o metodo Spherical Harmonics[25], um descritor de malhas 3D invariante à rotacao, translacao e escala. Os resultados atingiram uma taxa de acerto media de 98.52% com Ranking 5 demonstrando a eficiencia do metodo.Abstract: Automatic recognition of Sign Language signs is an important process that enhances the quality of use of digital media by hearing impaired people. Additionally, sign recognition enables a way of communication between deaf and hearing people who do not understand Sign Language. The approach of sign recognition used in this work is based on the global parameters of LIBRAS (Brazilian Sign Language): hand configuration, location or point of articulation, movement, palm orientation and facial expression. These parameters are combined to comprise signs, in a similar manner that phonemes are used to form words in spoken (oral) language. This paper presents a way to recognize one of the LIBRAS global parameters, the hand configuration, from 3D meshes. The Brazilian Sign Language has 61 hand configurations [16]. This work made use of a database containing 610 videos of 5 different users signing each hand configuration twice at distinct times, totaling 10 captures for each hand configuration. Two pictures depicting the front and the side views of the hand were manually extracted from each video. These pictures were segmented and pre-processed, after which they were used as input to the 3D reconstruction processing. The generation of the 3D meshes from the front and side images of the hand configuration was done using the Shape from Silhouette technique[7]. The recognition of the hand configurations from the 3D meshes was done with the use of SVM classifier - Support Vector Machine. The characteristics used to distinguish the mesh were obtained using the Spherical Harmonics [25] method: a 3D mesh descriptor that is rotation, translation and scale invariant. Results achieved a hit rate average of 98.52% with Rank 5, demonstrating the efficiency of the method

Benchmarking shape signatures against human perceptions of geometric similarity

Manual indexing of large databases of geometric information is both costly and difficult. Because of this, research into automated retrieval and indexing schemes has focused on the development of methods for characterising 3D shapes with a relatively small number of parameters (e.g. histograms) that allow ill-defined properties such as "geometric similarity" to be computed. However although many methods of generating these so called shape signatures have been proposed, little work on assessing how closely these measures match human perceptions of geometric similarity has been reported. This paper details the results of a trial that compared the part families identified by both human subjects and three published shape signatures. To do this a similarity matrix for the Drexel benchmark datasets was created by averaging the results of twelve manual inspections. Three different shape signatures (D2 shape distribution, spherical harmonics and surface portioning spectrum) were computed for each component in the dataset, and then used as input to a competitive neural network that sorted the objects into numbers of "similar" clusters. Comparison of human and machine generated clusters (i.e. families) of similar components allows the effectiveness of the signatures at duplicating human perceptions of shapes to be quantified. The work reported makes two contributions. Firstly the results of the human perception test suggest that the Drexel dataset contains objects whose perceived similarity levels ranged across the recorded spectrum (i.e. 0.1 to 0.9); Secondly the results obtained from benchmarking the three shape signatures against human perception demonstrate a low rate of false positives for all three signatures and a false negative rate that varied almost linearly with the amount of perceived similarity. In other words the shape signatures studied were reasonably effective at matching human perception in that they returned few wrong results and excluded parts in direct proportion to the level of similarity demanded by the user