On-barn pig weight estimation based on body measurements by structure-from-motion (SfM)

Information on the body shape of pigs is a key indicator to monitor their performance and health and to control or predict their market weight. Manual measurements are among the most common ways to obtain an indication of animal growth. However, this approach is laborious and difficult, and it may be stressful for both the pigs and the stockman. The present paper proposes the implementation of a Structure from Motion (SfM) photogrammetry approach as a new tool for on-barn animal reconstruction applications. This is possible also to new software tools allowing automatic estimation of camera parameters during the reconstruction process even without a preliminary calibration phase. An analysis on pig body 3D SfM characterization is here proposed, carried out under different conditions in terms of number of camera poses and animal movements. The work takes advantage of the total reconstructed surface as reference index to quantify the quality of the achieved 3D reconstruction, showing how as much as 80% of the total animal area can be characterized

Articulation-aware Canonical Surface Mapping

We tackle the tasks of: 1) predicting a Canonical Surface Mapping (CSM) that indicates the mapping from 2D pixels to corresponding points on a canonical template shape, and 2) inferring the articulation and pose of the template corresponding to the input image. While previous approaches rely on keypoint supervision for learning, we present an approach that can learn without such annotations. Our key insight is that these tasks are geometrically related, and we can obtain supervisory signal via enforcing consistency among the predictions. We present results across a diverse set of animal object categories, showing that our method can learn articulation and CSM prediction from image collections using only foreground mask labels for training. We empirically show that allowing articulation helps learn more accurate CSM prediction, and that enforcing the consistency with predicted CSM is similarly critical for learning meaningful articulation.Comment: To appear at CVPR 2020, project page https://nileshkulkarni.github.io/acsm

High-Throughput and Accurate 3D Scanning of Cattle Using Time-of-Flight Sensors and Deep Learning

We introduce a high throughput 3D scanning solution specifically designed to precisely measure cattle phenotypes. This scanner leverages an array of depth sensors, i.e. time-of-flight (Tof) sensors, each governed by dedicated embedded devices. The system excels at generating high-fidelity 3D point clouds, thus facilitating an accurate mesh that faithfully reconstructs the cattle geometry on the fly. In order to evaluate the performance of our system, we have implemented a two-fold validation process. Initially, we test the scanner's competency in determining volume and surface area measurements within a controlled environment featuring known objects. Secondly, we explore the impact and necessity of multi-device synchronization when operating a series of time-of-flight sensors. Based on the experimental results, the proposed system is capable of producing high-quality meshes of untamed cattle for livestock studies

Monocular 3d Object Recognition

Object recognition is one of the fundamental tasks of computer vision. Recent advances in the field enable reliable 2D detections from a single cluttered image. However, many challenges still remain. Object detection needs timely response for real world applications. Moreover, we are genuinely interested in estimating the 3D pose and shape of an object or human for the sake of robotic manipulation and human-robot interaction. In this thesis, a suite of solutions to these challenges is presented. First, Active Deformable Part Models (ADPM) is proposed for fast part-based object detection. ADPM dramatically accelerates the detection by dynamically scheduling the part evaluations and efficiently pruning the image locations. Second, we unleash the power of marrying discriminative 2D parts with an explicit 3D geometric representation. Several methods of such scheme are proposed for recovering rich 3D information of both rigid and non-rigid objects from monocular RGB images. (1) The accurate 3D pose of an object instance is recovered from cluttered images using only the CAD model. (2) A global optimal solution for simultaneous 2D part localization, 3D pose and shape estimation is obtained by optimizing a unified convex objective function. Both appearance and geometric compatibility are jointly maximized. (3) 3D human pose estimation from an image sequence is realized via an Expectation-Maximization algorithm. The 2D joint location uncertainties are marginalized out during inference and 3D pose smoothness is enforced across frames. By bridging the gap between 2D and 3D, our methods provide an end-to-end solution to 3D object recognition from images. We demonstrate a range of interesting applications using only a single image or a monocular video, including autonomous robotic grasping with a single image, 3D object image pop-up and a monocular human MoCap system. We also show empirical start-of-art results on a number of benchmarks on 2D detection and 3D pose and shape estimation

Modern post-mortem imaging: an update on recent developments

Modern post-mortem investigations use an increasing number of digital imaging methods, which can be collected under the term “post-mortem imaging”. Most methods of forensic imaging are from the radiology field and are therefore techniques that show the interior of the body with technologies such as X-ray or magnetic resonance imaging. To digitally image the surface of the body, other techniques are regularly applied, e.g. three-dimensional (3D) surface scanning (3DSS) or photogrammetry. Today's most frequently used techniques include post-mortem computed tomography (PMCT), post-mortem magnetic resonance imaging (PMMR), post-mortem computed tomographic angiography (PMCTA) and 3DSS or photogrammetry. Each of these methods has specific advantages and limitations. Therefore, the indications for using each method are different. While PMCT gives a rapid overview of the interior of the body and depicts the skeletal system and radiopaque foreign bodies, PMMR allows investigation of soft tissues and parenchymal organs. PMCTA is the method of choice for viewing the vascular system and detecting sources of bleeding. However, none of those radiological methods allow a detailed digital view of the body's surface, which makes 3DSS the best choice for such a purpose. If 3D surface scanners are not available, photogrammetry is an alternative. This review article gives an overview of different imaging techniques and explains their applications, advantages and limitations. We hope it will improve understanding of the methods

Discovering Relationships between Object Categories via Universal Canonical Maps

We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects. However, training such models requires to initialize inter-category correspondences by hand. This is suboptimal and the resulting models fail to maintain correct correspondences as individual categories are learned. In this paper, we show that improved correspondences can be learned automatically as a natural byproduct of learning category-specific dense pose predictors. To do this, we express correspondences between different categories and between images and categories using a unified embedding. Then, we use the latter to enforce two constraints: symmetric inter-category cycle consistency and a new asymmetric image-to-category cycle consistency. Without any manual annotations for the inter-category correspondences, we obtain state-of-the-art alignment results, outperforming dedicated methods for matching 3D shapes. Moreover, the new model is also better at the task of dense pose prediction than prior work.Comment: Accepted at CVPR 2021; Project page: https://gdude.de/discovering-3d-obj-re

Visualization Challenges of Virtual Reality 3D Images in New Media Environments

This paper proposes a three-dimensional image visualization process to face-drawing three-dimensional image reconstruction algorithm to obtain the data field with three-dimensional space, using color adjustment based on global color correction and local Poisson fusion to optimize the splicing seams between the texture color blocks and updating the visualization technology of three-dimensional images. Divide the digital display design and create a virtual reality visualization display using 3D modeling in combination with the new media environment. Propose design steps to visualize virtual reality three-dimensional images in the new media environment by combining the key algorithms of three-dimensional image visualization from the previous section. Combined with the application of new media displaying 3D images, the concept of artifact shape in reconstructed images is proposed to analyze the quality of 3D image reconstruction by taking the Herman model and Sheep-Logan model as the research object. Test experiments are conducted to examine the visual impact of texture mapping algorithms, and different sampling intervals are set to measure the drawing time of 3D reconstruction. For the data size and number of pictures of other organizations, the processing time of the 3D image reconstruction algorithm based on surface drawing is no more than 2s. The denser the sampling points are, the higher the degree of fitting, the more complete the preservation of isosurface information is, the finer the effect of 3D reconstruction, and the higher the quality of the image

Using Structure from Motion Mapping to Record and Analyze Details of the Colossal Hats (Pukao) of Monumental Statues on Rapa Nui (Easter Island)

Structure from motion (SfM) mapping is a photogrammetric technique that offers a cost-effective means of creating three-dimensional (3-D) visual representations from overlapping digital photographs. The technique is now used more frequently to document the archaeological record. We demonstrate the utility of SfM by studying red scoria bodies known as pukao from Rapa Nui (Easter Island, Chile). We created 3-D images of 50 pukao that once adorned the massive statues (moai) of Rapa Nui and compare them to 13 additional pukao located in Puna Pau, the island’s red scoria pukao quarry. Through SfM, we demonstrate that the majority of these bodies have petroglyphs and other surface features that are relevant to archaeological explanation and are currently at risk of continued degradation

The Pukao of Rapa Nui (Easter Island, Chile)

Structure from motion (SfM) mapping is a photogrammetric technique that offers a cost-effective means of creating three-dimensional visual representations from overlapping digital photographs. The technique has seen increasing uses for documenting the archaeological record. We demonstrate the utility of SfM through a study of the form of red scoria bodies known as pukao from Rapa Nui (Easter Island, Chile). We study 50 pukao that once adorned the massive statues (moai) of Rapa Nui, and compare them to 13 additional pukao located in Puna Pau: the island’s red scoria pukao quarry. Through SfM, we demonstrate that the majority of these bodies have petroglyphs and other surface features that are relevant to archaeological explanation and are currently at risk of continued degradation