3-D Hand Pose Estimation from Kinect's Point Cloud Using Appearance Matching

We present a novel appearance-based approach for pose estimation of a human hand using the point clouds provided by the low-cost Microsoft Kinect sensor. Both the free-hand case, in which the hand is isolated from the surrounding environment, and the hand-object case, in which the different types of interactions are classified, have been considered. The hand-object case is clearly the most challenging task having to deal with multiple tracks. The approach proposed here belongs to the class of partial pose estimation where the estimated pose in a frame is used for the initialization of the next one. The pose estimation is obtained by applying a modified version of the Iterative Closest Point (ICP) algorithm to synthetic models to obtain the rigid transformation that aligns each model with respect to the input data. The proposed framework uses a "pure" point cloud as provided by the Kinect sensor without any other information such as RGB values or normal vector components. For this reason, the proposed method can also be applied to data obtained from other types of depth sensor, or RGB-D camera

Markerless Augmented Reality via Stereo Video See-Through Head-Mounted Display Device

Conventionally, the camera localization for augmented reality (AR) relies on detecting a known pattern within the captured images. In this study, a markerless AR scheme has been designed based on a Stereo Video See-Through Head-Mounted Display (HMD) device. The proposed markerless AR scheme can be utilized for medical applications such as training, telementoring, or preoperative explanation. Firstly, a virtual model for AR visualization is aligned to the target in physical space by an improved Iterative Closest Point (ICP) based surface registration algorithm, with the target surface structure reconstructed by a stereo camera pair; then, a markerless AR camera localization method is designed based on the Kanade-Lucas-Tomasi (KLT) feature tracking algorithm and the Random Sample Consensus (RANSAC) correction algorithm. Our AR camera localization method is shown to be better than the traditional marker-based and sensor-based AR environment. The demonstration system was evaluated with a plastic dummy head and the display result is satisfactory for a multiple-view observation

Avaliação de uso de um LiDAR e algoritmos ICP (Iterative Closest Point) para inspeção de curta distância

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Mecânica, Florianópolis, 2023.A indústria de petróleo brasileira é uma das maiores adeptas ao uso de técnicas de extração de petróleo com o uso de risers flexíveis, que são estruturas tubulares utilizadas para o transporte seguro do material entre os poços de exploração de petróleo e as plataformas offshore. Dada sua importância, é essencial que sejam inspecionados periodicamente. Os procedimentos de inspeção atuais utilizados para a região dos dutos acima da lâmina d?agua estão, em sua maioria, atreladas a técnicas de alpinismo industrial, que emprega alto risco nas operações, além de custo e tempo elevados. Como forma de reduzir a periculosidade das inspeções, aeronaves remotamente pilotadas (UAV) podem ser utilizadas para inspecionar a camada externa dos risers com o uso de métodos ópticos. Comumente, câmeras são utilizadas para esse tipo de inspeção, mas há outras opções de sensores e sistemas ópticos a serem utilizados, como é o caso dos radares a laser (LiDAR). Esses sensores utilizam princípios de medição ativos, e emitem feixes laser para capturar a distância entre o mensurando e o sistema de medição, e podem ser utilizados para avaliar quantitativamente as superfícies que o sensor mede através do processamento de nuvens de pontos. As vantagens de uso de um LiDAR para inspeções sem contato se dão por se tratar de um equipamento que não depende de iluminação ambiente para seu uso, podendo ser usado em ambientes de baixa incidência de luz visível; por ter uma alta taxa de aquisição de informação e por seu tempo de processamento ser relativamente baixo quando comparado com outras técnicas de inspeção óptica. No entanto, radares a laser normalmente não possuem sistemas de navegação em sua montagem de fábrica, necessitando muitas vezes a integração com sensores inerciais e de navegação ou de operações matemáticas de refinamento de trajetória, como as técnicas de Iterative Closest Points (ICP) e filtros de Kalman. Essa dissertação de mestrado trata do desenvolvimento de técnicas de inspeção de curta distância utilizando um LiDAR como sistema de medição, e técnicas de ICP ponto a ponto e ponto a plano para definir posição e orientação do sistema. As análises foram feitas a partir de capturas de nuvens de pontos consecutivas, cada qual com sua respectiva localização, realizando assim a concatenação das nuvens de pontos e a reconstrução 3D de simulacros de riser, possibilitando sua inspeção dimensional. Durante o desenvolvimento, foram exploradas técnicas de estimação de estado e metodologias para calcular incertezas de medição a curtas distâncias utilizando um LiDAR. O sistema foi validado experimentalmente através da comparação entre os resultados adquiridos com o sistema de medição e as superfícies e geometrias de referência, possibilitando avaliar o desempenho de medição a curta distância e a influência do uso de algoritmos de refinamento de trajetória.Abstract: The Brazilian oil and gas industry is one of the biggest users of oil extraction techniques based on flexible risers, which are cylindrical structures used for safe extraction between offshore platforms and the wells. Giving its importance, it's essential that the equipment be inspected periodically. The actual inspection procedures for surface above water are related to industrial climbing, which demands not just high-risk operations, but high cost and time. As a way to reduce inspections dangerousness, unmanned aerial vehicles (UAV) can be used to inspect the risers' external surface with the usage of optical methods, like Light Detection and Ranging (LiDAR). These sensors use active measurement principles and emit laser beams to capture the distance between measurand and measurement system and can be used to quantitatively evaluate surfaces through point cloud processing. The advantages of using a LiDAR for touchless inspection are because it's an equipment that doesn't depend on environment lightning during the usage, which can be used at night or in low light environments; because has high frequency data acquisition and because of its time processing when compared with other optical techniques. However, LiDARs normally don't have navigation systems in themselves, demanding inertial sensors or state estimation mathematical operations to define trajectory, like Iterative Closest Points (ICP) and Kalman filters. This master dissertation talks about the development of short-range inspection techniques using a LiDAR as the measurement system and ICP techniques, point to point and point to plane, to define position and orientation of the system. The analysis was made based on the result of the consecutive point clouds captures, each one with its respective estimated localization, realizing merge of the point clouds and risers 3D reconstruction, being able to do dimensional inspection. During the development, state estimation techniques were explored, as well as methodologies to calculate short range measurement uncertainties using LiDAR. The system was experimentally validated through comparison between acquired results with the measurement system and reference geometries, making possible evaluate short range measurement performance and the influence of the usage of ICP algorithms to trajectory refinement

A Stochastic Iterative Closest Point Algorithm (stochastICP)

© Springer-Verlag Berlin Heidelberg 2001. We present a modification to the iterative closest point algorithm which improves the algorithm’s robustness and precision. At the start of each iteration, before point correspondence is calculated between the two feature sets, the algorithm randomly perturbs the point positions in one feature set. These perturbations allow the algorithm to move out of some local minima to find a minimum with a lower residual error. The size of this perturbation is reduced during the registration process. The algorithm has been tested using multiple starting positions to register three sets of data: a surface of a femur, a skull surface and a registration to hepatic vessels and a liver surface. Our results show that, if local minima are present, the stochastic ICP algorithm is more robust and is more precise than the standard ICP algorithm