4 research outputs found
Portuguese sign language recognition via computer vision and depth sensor
Sign languages are used worldwide by a multitude of individuals. They are mostly used by the deaf communities and their teachers, or people associated with them by ties of friendship or family. Speakers are a minority of citizens, often segregated, and over the years not much attention has been given to this form of communication, even by the scientific community. In fact, in Computer Science there is some, but limited, research and development in this area. In the particular case of sign Portuguese Sign Language-PSL that fact is more evident and, to our knowledge there isn’t yet an efficient system to perform the automatic recognition of PSL signs. With the advent and wide spreading of devices such as depth sensors, there are new possibilities to address this problem.
In this thesis, we have specified, developed, tested and preliminary evaluated, solutions that we think will bring valuable contributions to the problem of Automatic Gesture Recognition, applied to Sign Languages, such as the case of Portuguese Sign Language.
In the context of this work, Computer Vision techniques were adapted to the case of Depth Sensors. A proper gesture taxonomy for this problem was proposed, and techniques for feature extraction, representation, storing and classification were presented. Two novel algorithms to solve the problem of real-time recognition of isolated static poses were specified, developed, tested and evaluated. Two other algorithms for isolated dynamic movements for gesture recognition (one of them novel), have been also specified, developed, tested and evaluated. Analyzed results compare well with the literature.As LÃnguas Gestuais são utilizadas em todo o Mundo por uma imensidão de indivÃduos. Trata-se na sua grande maioria de surdos e/ou mudos, ou pessoas a eles associados por laços familiares de amizade ou professores de LÃngua Gestual. Tratando-se de uma minoria, muitas vezes segregada, não tem vindo a ser dada ao longo dos anos pela comunidade cientÃfica, a devida atenção a esta forma de comunicação.
Na área das Ciências da Computação existem alguns, mas poucos trabalhos de investigação e desenvolvimento. No caso particular da LÃngua Gestual Portuguesa - LGP esse facto é ainda mais evidente não sendo nosso conhecimento a existência de um sistema eficaz e efetivo para fazer o reconhecimento automático de gestos da LGP.
Com o aparecimento ou massificação de dispositivos, tais como sensores de profundidade, surgem novas possibilidades para abordar este problema.
Nesta tese, foram especificadas, desenvolvidas, testadas e efectuada a avaliação preliminar de soluções que acreditamos que trarão valiosas contribuições para o problema do Reconhecimento Automático de Gestos, aplicado à s LÃnguas Gestuais, como é o caso da LÃngua Gestual Portuguesa.
Foram adaptadas técnicas de Visão por Computador ao caso dos Sensores de Profundidade.
Foi proposta uma taxonomia adequada ao problema, e apresentadas técnicas para a extração, representação e armazenamento de caracterÃsticas. Foram especificados, desenvolvidos, testados e avaliados dois algoritmos para resolver o problema do reconhecimento em tempo real de poses estáticas isoladas. Foram também especificados, desenvolvidos, testados e avaliados outros dois algoritmos para o Reconhecimento de Movimentos Dinâmicos Isolados de Gestos(um deles novo).Os resultados analisados são comparáveis à literatura.Las lenguas de Signos se utilizan en todo el Mundo por una multitud de personas. En su mayorÃa son personas sordas y/o mudas, o personas asociadas con ellos por vÃnculos de amistad o familiares y profesores de Lengua de Signos. Es una minorÃa de personas, a menudo segregadas, y no se ha dado en los últimos años por la comunidad cientÃfica, la atención debida a esta forma de comunicación.
En el área de Ciencias de la Computación hay alguna pero poca investigación y desarrollo. En el caso particular de la Lengua de Signos Portuguesa - LSP, no es de nuestro conocimiento la existencia de un sistema eficiente y eficaz para el reconocimiento automático.
Con la llegada en masa de dispositivos tales como Sensores de Profundidad, hay nuevas posibilidades para abordar el problema del Reconocimiento de Gestos.
En esta tesis se han especificado, desarrollado, probado y hecha una evaluación preliminar de soluciones, aplicada a las Lenguas de Signos como el caso de la Lengua de Signos Portuguesa - LSP.
Se han adaptado las técnicas de Visión por Ordenador para el caso de los Sensores de Profundidad.
Se propone una taxonomÃa apropiada para el problema y se presentan técnicas para la extracción, representación y el almacenamiento de caracterÃsticas.
Se desarrollaran, probaran, compararan y analizan los resultados de dos nuevos algoritmos para resolver el problema del Reconocimiento Aislado y Estático de Posturas. Otros dos algoritmos (uno de ellos nuevo) fueran también desarrollados, probados, comparados y analizados los resultados, para el Reconocimiento de Movimientos Dinámicos Aislados de los Gestos
Towards the creation of an annotation system and a digital archive platform for contemporary dance
Tese de mestrado. Multimédia. Universidade do Porto. Faculdade de Engenharia. 201
Universal Gesture Tracking Framework in OpenISS and ROS and its Applications
In this research work, we present a common and extensible framework that abstracts different vision-based gesture recognition middleware and provides uniform gesture recognition data obtained from those via its simpler API to enable hand gesture interaction in different kinds of applications. We demonstrate various aspects of our framework via instrumentation and enable gesture interaction for our two specific yet different needs.
Firstly, we alleviate limited gesture tracking functionality in ISSv2 aka. (Illimitable Space System v2), an interactive and configurable artists' toolbox that is used to create music visualizations, visual effects and interactive documentary film based on the inputs from users such as gestures, voice, motion, etc.
Secondly, we provide a proof-of-concept solution to enhance and demonstrate limited language usability of the FORENSIC LUCID language's composition and compiler interactivity by enabling a forensic investigator to create partial FORENSIC LUCID encoded programs which require manipulation of preloaded digital evidence objects in a 3D warehouse-like application (DigiEVISS) via hand gesture interaction.
We also leverage Robot Operating System (ROS), an open source set of tools and libraries for its communication middleware to broadcast our framework data over the network. We provide this framework as a specialization of the OpenISS core framework and evaluate our framework on various aspects. We employ metrics such as effective frame rate and delay to evaluate our exemplified scenarios that represent our needs
Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space
This major component of the research described in this thesis is 3D computer
graphics, specifically the realistic physics-based softbody simulation and
haptic responsive environments. Minor components include advanced
human-computer interaction environments, non-linear documentary storytelling,
and theatre performance. The journey of this research has been unusual because
it requires a researcher with solid knowledge and background in multiple
disciplines; who also has to be creative and sensitive in order to combine the
possible areas into a new research direction. [...] It focuses on the advanced
computer graphics and emerges from experimental cinematic works and theatrical
artistic practices. Some development content and installations are completed to
prove and evaluate the described concepts and to be convincing. [...] To
summarize, the resulting work involves not only artistic creativity, but
solving or combining technological hurdles in motion tracking, pattern
recognition, force feedback control, etc., with the available documentary
footage on film, video, or images, and text via a variety of devices [....] and
programming, and installing all the needed interfaces such that it all works in
real-time. Thus, the contribution to the knowledge advancement is in solving
these interfacing problems and the real-time aspects of the interaction that
have uses in film industry, fashion industry, new age interactive theatre,
computer games, and web-based technologies and services for entertainment and
education. It also includes building up on this experience to integrate Kinect-
and haptic-based interaction, artistic scenery rendering, and other forms of
control. This research work connects all the research disciplines, seemingly
disjoint fields of research, such as computer graphics, documentary film,
interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm