26 research outputs found
TS-RGBD Dataset: a Novel Dataset for Theatre Scenes Description for People with Visual Impairments
Computer vision was long a tool used for aiding visually impaired people to
move around their environment and avoid obstacles and falls. Solutions are
limited to either indoor or outdoor scenes, which limits the kind of places and
scenes visually disabled people can be in, including entertainment places such
as theatres. Furthermore, most of the proposed computer-vision-based methods
rely on RGB benchmarks to train their models resulting in a limited performance
due to the absence of the depth modality.
In this paper, we propose a novel RGB-D dataset containing theatre scenes
with ground truth human actions and dense captions annotations for image
captioning and human action recognition: TS-RGBD dataset. It includes three
types of data: RGB, depth, and skeleton sequences, captured by Microsoft
Kinect.
We test image captioning models on our dataset as well as some skeleton-based
human action recognition models in order to extend the range of environment
types where a visually disabled person can be, by detecting human actions and
textually describing appearances of regions of interest in theatre scenes
Theater Aid System for the Visually Impaired Through Transfer Learning of Spatio-Temporal Graph Convolution Networks
The aim of this research is to recognize human actions performed on stage to
aid visually impaired and blind individuals. To achieve this, we have created a
theatre human action recognition system that uses skeleton data captured by
depth image as input. We collected new samples of human actions in a theatre
environment, and then tested the transfer learning technique with three
pre-trained Spatio-Temporal Graph Convolution Networks for skeleton-based human
action recognition: the spatio-temporal graph convolution network, the
two-stream adaptive graph convolution network, and the multi-scale disentangled
unified graph convolution network. We selected the NTU-RGBD human action
benchmark as the source domain and used our collected dataset as the target
domain. We analyzed the transferability of the pre-trained models and proposed
two configurations to apply and adapt the transfer learning technique to the
diversity between the source and target domains. The use of transfer learning
helped to improve the performance of the human action system within the context
of theatre. The results indicate that Spatio-Temporal Graph Convolution
Networks is positively transferred, and there was an improvement in performance
compared to the baseline without transfer learning.Comment: 28 pages, 18 figures, research paper not publishe
Detection and analysis of symmetrical parts on face for head pose estimation
International audienc
Proceedings of IEEE International Conference on Machine and Web Intelligence (ICMWI-2010)
International audienc