An integrated python-based open-source Digital Image Correlation Software for in-plane and out-of-plane measurements (Part 2)

Funding Information: The authors would like to acknowledge Fundação para a Ciência e a Tecnologia (FCT-MCTES) throughout the project PTDC/EMD-EMD/1230/2021 (AneurysmTool) and UIDB/00667/2020 (UNIDEMI) and the support provided by the Brazilian Government funding agencies CAPES, Brazil , FAPERJ, Brazil and CNPq, Brazil . Publisher Copyright: © 2022 The AuthorsThis work presents an out-of-the-box python-based open-source 3D Digital Image Correlation (3D-DIC) software for both in-plane and out-of-plane full-field measurements, denoted by iCorrVision-3D. The software includes an integrated stereo grabber for image acquisition, stereo calibration, numerical stereo correlation and post-processing modules. The main objective is to provide a complete integrated 3D-DIC system for users. All important DIC setting parameters can be easily controlled by the user from an intuitive graphical interface. For instance, the interpolation strategy and correlation techniques that are usually not open for users are available for modifications in iCorrVision-3D. The proposed software can be used in a great number of applications in engineering. Results indicated that the iCorrVision-3D software is robust and accurate in reconstructing the 3D shape of objects and in evaluating the out-of-plane full-field displacement of specimens being tested.publishersversionpublishe

Stereo-Based Head Pose Tracking Using Iterative Closest Point and Normal Flow Constraint

In this text, we present two stereo-based head tracking techniques along with a fast 3D model acquisition system. The first tracking technique is a robust implementation of stereo-based head tracking designed for interactive environments with uncontrolled lighting. We integrate fast face detection and drift reduction algorithms with a gradient-based stereo rigid motion tracking technique. Our system can automatically segment and track a user's head under large rotation and illumination variations. Precision and usability of this approach are compared with previous tracking methods for cursor control and target selection in both desktop and interactive room environments. The second tracking technique is designed to improve the robustness of head pose tracking for fast movements. Our iterative hybrid tracker combines constraints from the ICP (Iterative Closest Point) algorithm and normal flow constraint. This new technique is more precise for small movements and noisy depth than ICP alone, and more robust for large movements than the normal flow constraint alone. We present experiments which test the accuracy of our approach on sequences of real and synthetic stereo images. The 3D model acquisition system we present quickly aligns intensity and depth images, and reconstructs a textured 3D mesh. 3D views are registered with shape alignment based on our iterative hybrid tracker. We reconstruct the 3D model using a new Cubic Ray Projection merging algorithm which takes advantage of a novel data structure: the linked voxel space. We present experiments to test the accuracy of our approach on 3D face modelling using real-time stereo images

3D reconstruction of particle agglomerates using multiple scanning electron microscope stereo-pair images

Scanning electron microscopes (SEM) allow a detailed surface analysis of a wide variety of specimen. However, SEM image data does not provide depth information about a captured scene. This limitation can be overcome by recovering the hidden third dimension of the acquired SEM micrographs, for instance to fully characterize a particle agglomerate's morphology. In this paper, we present a method that allows the three-dimensional (3D) reconstruction of investigated particle agglomerates using an uncalibrated stereo vision approach that is applied to multiple stereo-pair images. The reconstruction scheme starts with a feature detection and subsequent matching in each pair of stereo images. Based on these correspondences, a robust estimate of the epipolar geometry is determined. A following rectification allows a reduction of the dense correspondence problem to a one-dimensional search along conjugate epipolar lines. So the disparity maps can be obtained using a dense stereo matching algorithm. To remove outliers while preserving edges and individual structures, a disparity refinement is executed using suitable image filtering techniques. The investigated specimen's qualitative depth's information can be directly calculated from the determined disparity maps. In a final step the resulting point clouds are registered. State-of-the-art algorithms for 3D reconstruction of SEM micrographs mainly focus on structures whose image pairs contain hardly or even none-occluded areas. The acquisition of multiple stereo-pair images from different perspectives makes it possible to combine the obtained point clouds in order to overcome occurring occlusions. The presented approach thereby enables the 3D illustration of the investigated particle agglomerates. © 2018 SPIE

Mapping the Navigational Information Content of Insect Habitats

For developing and validating models of insect navigation it is essential to identify the visual input insects experience in their natural habitats. Here we report on the development of methods to reconstruct what insects see when making navigational decisions and critically assess the current limitations of such methods. We used a laser-range finder as well as camera-based methods to capture the 3D structure and the appearance of outdoor environments. Both approaches produce coloured point clouds that allow within the model scale the reconstruction of views at defined positions and orientations. For instance, we filmed bees and wasps with a high-speed stereo camera system to estimate their 3D flight paths and gaze direction. The high-speed system is registered with a 3D model of the same environment, such that panoramic images can be rendered along the insects’ flight paths (see accompanying abstract “Benchmark 3D-models of natural navigation environments @ www.InsectVision.org” by Mair et al.). The laser-range finder (see figure A) is equipped with a rotating camera that provides colour information for the measured 3D points. This system is robust and easy-to-use in the field generating high resolution data (about 50 × 106 points) with large field of view, up to a distance of 80 m at typical acquisition times of about 8 minutes. However, a large number of scans at different locations has to be recorded and registered to account for occlusions. In comparison, data acquisition in camera-based reconstruction from multiple view-points is fast, but model generation is computationally more complex due to bundle adjustment and dense pair-wise stereo computation (see figure B, C for views rendered from a 3D model based on 6 image pairs). In addition it is non-trivial and often time-consuming in the field to ensure the acquisition of sufficient information. We are currently developing the tools that will allow us to combine the results of laser-scanner and camera-based 3D reconstruction methods

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

A Novel Framework for Highlight Reflectance Transformation Imaging

We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091DSURF project (PRIN 2015) funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa