Automatic measurement of hand dimensions using consumer 3D cameras

This article describes the metrological characterisation of two prototypes that use the point clouds acquired by consumer 3D cameras for the measurement of the human hand geometrical parameters. The initial part of the work is focused on the general description of algorithms that allow for the derivation of dimensional parameters of the hand. Algorithms were tested on data acquired using Microsoft Kinect v2 and Intel RealSense D400 series sensors. The accuracy of the proposed measurement methods has been evaluated in different tests aiming to identify bias errors deriving from point-cloud inaccuracy and at the identification of the effect of the hand pressure and the wrist flexion/extension. Results evidenced an accuracy better than 1 mm in the identification of the hand’s linear dimension and better than 20 cm3 for hand volume measurements. The relative uncertainty of linear dimensions, areas, and volumes was in the range of 1-10 %. Measurements performed with the Intel RealSense D400 were, on average, more repeatable than those performed with Microsoft Kinect. The uncertainty values limit the use of these devices to applications where the requested accuracy is larger than 5 % (volume measurements), 3 % (area measurements), and 1 mm (hands’ linear dimensions and thickness)

A new metrological characterization strategy for 3D multi-camera systems

AbstractThe objective of this study is to establish a new methodology for the metrological characterization of interactive multi-camera systems. In the case of 3D system highly adapted to specific needs the accuracy evaluation cannot be performed using standard state-of-the-art techniques. To this end, the metrological characterization techniques used in the literature were investigated in order to define a new methodology that can be adjusted to each device by making the appropriate modifications. The proposed strategy is adopted for the metrological characterization of a new interactive multi-camera system for the acquisition of the arm

A Fast and Reliable Optical 3D Scanning System for Human Arm

AbstractThe article discusses the design of an acquisition system for the 3D surface of human arms. The system is composed by a 3D optical scanner implementing stereoscopic depth sensors and by an acquisition software responsible for the processing of the raw data. The 3D data acquired by the scanner is used as starting point for the manufacturing of custom-made 3D printed casts. Specifically, the article discusses the choices made in the development of an improved version of an existing system presented in [1] and presents the results achieved by the devised system

Sensor architectures and technologies for upper limb 3d surface reconstruction: A review

3D digital models of the upper limb anatomy represent the starting point for the design process of bespoke devices, such as orthoses and prostheses, which can be modeled on the actual patient’s anatomy by using CAD (Computer Aided Design) tools. The ongoing research on optical scanning methodologies has allowed the development of technologies that allow the surface reconstruction of the upper limb anatomy through procedures characterized by minimum discomfort for the patient. However, the 3D optical scanning of upper limbs is a complex task that requires solving problematic aspects, such as the difficulty of keeping the hand in a stable position and the presence of artefacts due to involuntary movements. Scientific literature, indeed, investigated different approaches in this regard by either integrating commercial devices, to create customized sensor architectures, or by developing innovative 3D acquisition techniques. The present work is aimed at presenting an overview of the state of the art of optical technologies and sensor architectures for the surface acquisition of upper limb anatomies. The review analyzes the working principles at the basis of existing devices and proposes a categorization of the approaches based on handling, pre/post-processing effort, and potentialities in real-time scanning. An in-depth analysis of strengths and weaknesses of the approaches proposed by the research community is also provided to give valuable support in selecting the most appropriate solution for the specific application to be addressed

Analysis of RGB-D camera technologies for supporting different facial usage scenarios

AbstractRecently a wide variety of applications has been developed integrating 3D functionalities. Advantages given by the possibility of relying on depth information allows the developers to design new algorithms and to improve the existing ones. In particular, for what concerns face morphology, 3D has led to the possibility to obtain face depth maps highly close to reality and consequently an improvement of the starting point for further analysis such as Face Detection, Face Authentication, Face Identification and Face Expression Recognition. The development of the aforementioned applications would have been impossible without the progress of sensor technologies for obtaining 3D information. Several solutions have been adopted over time. In this paper, emphasis is put on passive stereoscopy, structured light, time-of-flight (ToF) and active stereoscopy, namely the most used technologies for the cameras design and fulfilment according to the literature. The aim of this article is to investigate facial applications and to examine 3D camera technologies to suggest some guidelines for addressing the correct choice of a 3D sensor according to the application that has to be developed

Collaborative robot control with hand gestures

Mestrado de dupla diplomação com a Université Libre de TunisThis thesis focuses on hand gesture recognition by proposing an architecture to control a collaborative robot in real-time vision based on hand detection, tracking, and gesture recognition for interaction with an application via hand gestures. The first stage of our system allows detecting and tracking a bar e hand in a cluttered background using skin detection and contour comparison. The second stage allows recognizing hand gestures using a Machine learning method algorithm. Finally an interface has been developed to control the robot over. Our hand gesture recognition system consists of two parts, in the first part for every frame captured from a camera we extract the keypoints for every training image using a machine learning algorithm, and we appoint the keypoints from every image into a keypoint map. This map is treated as an input for our processing algorithm which uses several methods to recognize the fingers in each hand. In the second part, we use a 3D camera with Infrared capabilities to get a 3D model of the hand to implement it in our system, after that we track the fingers in each hand and recognize them which made it possible to count the extended fingers and to distinguish each finger pattern. An interface to control the robot has been made that utilizes the previous steps that gives a real-time process and a dynamic 3D representation.Esta dissertação trata do reconhecimento de gestos realizados com a mão humana, propondo uma arquitetura para interagir com um robô colaborativo, baseado em visão computacional, rastreamento e reconhecimento de gestos. O primeiro estágio do sistema desenvolvido permite detectar e rastrear a presença de uma mão em um fundo desordenado usando detecção de pele e comparação de contornos. A segunda fase permite reconhecer os gestos das mãos usando um algoritmo do método de aprendizado de máquina. Finalmente, uma interface foi desenvolvida para interagir com robô. O sistema de reconhecimento de gestos manuais está dividido em duas partes. Na primeira parte, para cada quadro capturado de uma câmera, foi extraído os pontos-chave de cada imagem de treinamento usando um algoritmo de aprendizado de máquina e nomeamos os pontos-chave de cada imagem em um mapa de pontos-chave. Este mapa é tratado como uma entrada para o algoritmo de processamento que usa vários métodos para reconhecer os dedos em cada mão. Na segunda parte, foi utilizado uma câmera 3D com recursos de infravermelho para obter um modelo 3D da mão para implementá-lo em no sistema desenvolvido, e então, foi realizado os rastreio dos dedos de cada mão seguido pelo reconhecimento que possibilitou contabilizar os dedos estendidos e para distinguir cada padrão de dedo. Foi elaborado uma interface para interagir com o robô manipulador que utiliza as etapas anteriores que fornece um processo em tempo real e uma representação 3D dinâmica

Analysis of RGB-D camera technologies for supporting different facial usage scenarios

Recently a wide variety of applications has been developed integrating 3D functionalities. Advantages given by the possibility of relying on depth information allows the developers to design new algorithms and to improve the existing ones. In particular, for what concerns face morphology, 3D has led to the possibility to obtain face depth maps highly close to reality and consequently an improvement of the starting point for further analysis such as Face Detection, Face Authentication, Face Identification and Face Expression Recognition. The development of the aforementioned applications would have been impossible without the progress of sensor technologies for obtaining 3D information. Several solutions have been adopted over time. In this paper, emphasis is put on passive stereoscopy, structured light, time-of-flight (ToF) and active stereoscopy, namely the most used technologies for the cameras design and fulfilment according to the literature. The aim of this article is to investigate facial applications and to examine 3D camera technologies to suggest some guidelines for addressing the correct choice of a 3D sensor according to the application that has to be developed

ANALYSIS OF AN RGBD CAMERA FOR MANHOLE INSPECTION

As the service life of ducts and manholes reaches their end, there is a growing need to preserve the structures. In order to prevent casualties and service interruption, more frequent inspections are advised. To this day most of the inspections are still, manually made. These inspectors need to be highly qualified, and the inspections are done in an hazardous environment so, automating the inspection process would lead to a healthier workplace. Given the recent development in RGBD cameras, with smaller size factor, cost, and weight, this project aims to evaluate the application of one of the more recent models in a manhole inspection environment. Several analysis are done to assess the RGBD sensor performance for 3D model reconstruction, including the comparison with a ground-truth 3D model obtained using a laser point profile sensor with an industrial robot