Augmented Reality based 3D Human Hands Tracking from Monocular True Images Using Convolutional Neural Network

Precise modeling of hand tracking from monocular moving camera calibration parameters using semantic cues is an active area of research concern for the researchers due to lack of accuracy and computational overheads. In this context, deep learning based framework, i.e. convolutional neural network based human hands tracking as well as recognizing pose of hands in the current camera frame become active research problem. In addition, tracking based on monocular camera needs to be addressed due to updated technology such as Unity3D engine and other related augmented reality plugins. This research aims to track human hands in continuous frame by using the tracked points to draw 3D model of the hands as an overlay in the original tracked image. In the proposed methodology, Unity3D environment was used for localizing hand object in augmented reality (AR). Later, convolutional neural network was used to detect hand palm and hand keypoints based on cropped region of interest (ROI). Proposed method by this research achieved accuracy rate of 99.2% where single monocular true images were used for tracking. Experimental validation shows the efficiency of the proposed methodology.Peer reviewe

Reconocimiento biométrico egocéntrico para entornos de realidad virtual

En este trabajo se presenta el primer entorno experimental para desarrollar sistemas biométricos de reconocimiento palmar en entornos virtuales. El entorno propuesto consta de una base de datos y de un sistema de reconocimiento inicial que sirva como base para futuros desarrollos. El sistema se divide en tres bloques principales: detección de pose de la mano, extracción de la palma y comparación entre palmas. Se crea uno automático que no necesita de supervisión humana, y otro donde la detección de pose se hace manualmente. El objetivo es crear un entorno que sirva como punto de partida para estudios futuros y que proponga distintas alternativas válidas para la implementación de estos sistemas. También intentar acompañar el auge de los entornos virtuales con un reconocimiento biométrico necesario en ciertas aplicaciones. Para lograr esto se ha hecho, en primer lugar, un estudio del estado del arte de la biometría y en concreto del reconocimiento palmar. A continuación, se han planteado los retos de este tipo de sistema y a partir de ellos se ha desarrollado el sistema completo. Por último, se han analizado los resultados y se han planteados posibles mejora

UV-Based 3D Hand-Object Reconstruction with Grasp Optimization

We propose a novel framework for 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image. The representation of hand-object contact regions is critical for accurate reconstructions. Instead of approximating the contact regions with sparse points, as in previous works, we propose a dense representation in the form of a UV coordinate map. Furthermore, we introduce inference-time optimization to fine-tune the grasp and improve interactions between the hand and the object. Our pipeline increases hand shape reconstruction accuracy and produces a vibrant hand texture. Experiments on datasets such as Ho3D, FreiHAND, and DexYCB reveal that our proposed method outperforms the state-of-the-art.Comment: BMVC 2022 Spotligh