AUTOMATED DETECTION AND LAYOUT REGULARIZATION OF SIMILAR FEATURES IN INDOOR POINT CLOUD

Automated identification of high-level structures in unorganized point cloud of indoor spaces Indoor space is an important aspect of scene analysis that provides essential information for many applications, such as building digitization, indoor navigation and evacuation route planning. In addition, detection of repetition and regularities in the organization indoor environments, such as rooms, can be used to provide a contextual relationship in the reconstruction phase. However, retrieving high-level information is a challenging task due to the unorganized nature of the raw data, poor-quality of the input data that are in many cases contaminated with noise and outliers. in point benefit from the apparent regularities and strong contextual relationships in façades. The main observation exploited in this paper is the fact that building indoor is generally constituted by a set of basic shapes repeated several times in regular layouts. Building elements can be considered as similar if they share a set of features and elements in an idealized layout exhibiting some regularities. Starting from this main assumption a recursive adaptive partitioning of the indoor point cloud is carried out to automatically derive a flexible and hierarchical 3D representation of the building space. The presented methodology is tested on a synthetic dataset with Gaussian noise. The reconstructed pattern shows a close correspondence with the synthetic one showing the viability of the proposed approach

Equivariant geometric learning for digital rock physics: estimating formation factor and effective permeability tensors from Morse graph

We present a SE(3)-equivariant graph neural network (GNN) approach that directly predicting the formation factor and effective permeability from micro-CT images. FFT solvers are established to compute both the formation factor and effective permeability, while the topology and geometry of the pore space are represented by a persistence-based Morse graph. Together, they constitute the database for training, validating, and testing the neural networks. While the graph and Euclidean convolutional approaches both employ neural networks to generate low-dimensional latent space to represent the features of the micro-structures for forward predictions, the SE(3) equivariant neural network is found to generate more accurate predictions, especially when the training data is limited. Numerical experiments have also shown that the new SE(3) approach leads to predictions that fulfill the material frame indifference whereas the predictions from classical convolutional neural networks (CNN) may suffer from spurious dependence on the coordinate system of the training data. Comparisons among predictions inferred from training the CNN and those from graph convolutional neural networks (GNN) with and without the equivariant constraint indicate that the equivariant graph neural network seems to perform better than the CNN and GNN without enforcing equivariant constraints

Toward Constrained Animal Pose Estimation

Quantifying animal behavior is a crucial aspect of the ongoing neuroscientific endeavor to understand the brain, since it is a prerequisite for studying how neural computations relate to behavioral outputs. One method for obtaining an objective yet detailed description of an animal's unconstrained and therefore natural behavior is given by estimating its pose, i.e. the collective positions and orientations of all individual body parts in space at a given moment in time. While various approaches have been proposed for estimating the pose of a freely-moving animal, so far, studies relying on video cameras for recording the required behavioral data have neglected reconstructing the actual skeleton of an animal and only considered inferring the positions of anatomical landmarks located on its body surface. Additionally, many approaches lack incorporating mechanistic knowledge of an animal's anatomy, which leaves room for improving the resulting pose reconstruction accuracy. Consequently, methods for quantifying skeletal animal poses during free motion sequences are desirable tools for future neuroscientific studies. The work presented in this thesis tackles the problem of inferring skeletal poses from recorded video data of freely-moving animal subjects via a constrained animal pose estimation framework, which enables reconstructing underlying three-dimensional joint positions from observable surface markers while enforcing anatomical and temporal constraints. Anatomical constraints are implemented via a realistic skeleton model, which accounts for physiological joint angle limits, bone lengths and body symmetry. Besides, the realistic skeleton model allows for learning individual skeleton anatomies directly from recorded video data of behaving animals, taking into account subject-specific differences with respect to bone lengths and body-symmetry. Furthermore, to ensure that reconstructed joint positions follow smooth motion trajectories, the proposed animal pose estimation framework also enforces temporal constraints. Particularly, temporal constraints are implemented via an underlying state space model, which allows for deploying a Bayesian smoother for inferring bone rotations as well as an expectation-maximization algorithm for learning the unknown probabilistic hyper-parameters of the state space model. The proposed animal pose estimation framework is evaluated and tested with respect to its reconstruction accuracy and usability for quantifying a range of different behaviors. By comparing learned skeleton anatomies with ground truth data obtained via magnetic resonance imaging, it is shown that the framework offers the opportunity to learn three-dimensional joint positions and bone lengths solely from two-dimensional video data. Besides, to test whether poses of freely-moving animals are accurately inferred, independently measured paw positions are obtained using a frustrated total internal reflection imaging system and compared to their reconstructed counterparts, while the effects of the enforced anatomical and temporal constraints are analyzed. This analysis shows the advantages of constrained over unconstrained animal pose estimation, since enforcing constraints reduces errors with respect to reconstructed paw positions and orientations. Furthermore, to assess if the proposed pose estimation framework is capable of accurately quantifying common behaviors, periodic gait cycles are analyzed based on reconstructed skeletal poses, which shows that enforcing constraints is essential for successfully extracting characteristic movement patterns from recorded video data. Finally, the proposed pose estimation framework is also used to quantify complex gap-crossing behaviors, where animals jump over gaps of various distances. This analysis shows that reconstructing skeletal poses enables computing characteristic movement patterns during jumping and correlating skeletal kinematic quantities with each other as well as the jumped distances. In summary, this thesis proposes an animal pose estimation framework, which allows for reconstructing anatomically-plausible as well as time-consistent three-dimensional skeletal poses of freely-moving animals from two-dimensional video data. To achieve this, anatomical and temporal constraints are implemented into the proposed pose reconstruction framework, which transpired to be essential for obtaining accurate pose reconstruction results. Consequently, this thesis contains analyses, which demonstrate the importance of the implemented constraints in the context of animal pose estimation