Least squares optimization: From theory to practice

Nowadays, Nonlinear Least-Squares embodies the foundation of many Robotics and Computer Vision systems. The research community deeply investigated this topic in the last few years, and this resulted in the development of several open-source solvers to approach constantly increasing classes of problems. In this work, we propose a unified methodology to design and develop efficient Least-Squares Optimization algorithms, focusing on the structures and patterns of each specific domain. Furthermore, we present a novel open-source optimization system that addresses problems transparently with a different structure and designed to be easy to extend. The system is written in modern C++ and runs efficiently on embedded systemsWe validated our approach by conducting comparative experiments on several problems using standard datasets. The results show that our system achieves state-of-the-art performances in all tested scenarios

Multimodal Three Dimensional Scene Reconstruction, The Gaussian Fields Framework

The focus of this research is on building 3D representations of real world scenes and objects using different imaging sensors. Primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry, and multi-spectral image sequences including visual and thermal IR images that provide additional scene characteristics. The crucial technical challenge that we addressed is the automatic point-sets registration task. In this context our main contribution is the development of an optimization-based method at the core of which lies a unified criterion that solves simultaneously for the dense point correspondence and transformation recovery problems. The new criterion has a straightforward expression in terms of the datasets and the alignment parameters and was used primarily for 3D rigid registration of point-sets. However it proved also useful for feature-based multimodal image alignment. We derived our method from simple Boolean matching principles by approximation and relaxation. One of the main advantages of the proposed approach, as compared to the widely used class of Iterative Closest Point (ICP) algorithms, is convexity in the neighborhood of the registration parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. Physically the criterion is interpreted in terms of a Gaussian Force Field exerted by one point-set on the other. Such formulation proved useful for controlling and increasing the region of convergence, and hence allowing for more autonomy in correspondence tasks. Furthermore, the criterion can be computed with linear complexity using recently developed Fast Gauss Transform numerical techniques. In addition, we also introduced a new local feature descriptor that was derived from visual saliency principles and which enhanced significantly the performance of the registration algorithm. The resulting technique was subjected to a thorough experimental analysis that highlighted its strength and showed its limitations. Our current applications are in the field of 3D modeling for inspection, surveillance, and biometrics. However, since this matching framework can be applied to any type of data, that can be represented as N-dimensional point-sets, the scope of the method is shown to reach many more pattern analysis applications

Registration using Graphics Processor Unit

Data point set registration is an important operation in coordinate metrology. Registration is the operation by which sampled point clouds are aligned with a CAD model by a 4X4 homogeneous transformation (e.g., rotation and translation). This alignment permits validation of the produced artifact\u27s geometry. State-of-the-art metrology systems are now capable of generating thousands, if not millions, of data points during an inspection operation, resulting in increased computational power to fully utilize these larger data sets. The registration process is an iterative nonlinear optimization operation having an execution time directly related to the number of points processed and CAD model complexity. The objective function to be minimized by this optimization is the sum of the square distances between each point in the point cloud and the closest surface in the CAD model. A brute force approach to registration, which is often used, is to compute the minimum distance between each point and each surface in the CAD model. As point cloud sizes and CAD model complexity increase, this approach becomes intractable and inefficient. Highly efficient numerical and analytical gradient based algorithms exist and their goal is to convergence to an optimal solution in minimum time. This thesis presents a new approach to efficiently perform the registration process by employing readily available computer hardware, the graphical processor unit (GPU). The data point set registration time for the GPU shows a significant improvement (around 15-20 times) over typical CPU performance. Efficient GPU programming decreases the complexity of the steps and improves the rate of convergence of the existing algorithms. The experimental setup reveals the exponential increasing nature of the CPU and the linear performance of the GPU in various aspects of an algorithm. The importance of CPU in the GPU programming is highlighted. The future implementations disclose the possible extensions of a GPU for higher order and complex coordinate metrology algorithms

Single View Modeling and View Synthesis

This thesis develops new algorithms to produce 3D content from a single camera. Today, amateurs can use hand-held camcorders to capture and display the 3D world in 2D, using mature technologies. However, there is always a strong desire to record and re-explore the 3D world in 3D. To achieve this goal, current approaches usually make use of a camera array, which suffers from tedious setup and calibration processes, as well as lack of portability, limiting its application to lab experiments. In this thesis, I try to produce the 3D contents using a single camera, making it as simple as shooting pictures. It requires a new front end capturing device rather than a regular camcorder, as well as more sophisticated algorithms. First, in order to capture the highly detailed object surfaces, I designed and developed a depth camera based on a novel technique called light fall-off stereo (LFS). The LFS depth camera outputs color+depth image sequences and achieves 30 fps, which is necessary for capturing dynamic scenes. Based on the output color+depth images, I developed a new approach that builds 3D models of dynamic and deformable objects. While the camera can only capture part of a whole object at any instance, partial surfaces are assembled together to form a complete 3D model by a novel warping algorithm. Inspired by the success of single view 3D modeling, I extended my exploration into 2D-3D video conversion that does not utilize a depth camera. I developed a semi-automatic system that converts monocular videos into stereoscopic videos, via view synthesis. It combines motion analysis with user interaction, aiming to transfer as much depth inferring work from the user to the computer. I developed two new methods that analyze the optical flow in order to provide additional qualitative depth constraints. The automatically extracted depth information is presented in the user interface to assist with user labeling work. In this thesis, I developed new algorithms to produce 3D contents from a single camera. Depending on the input data, my algorithm can build high fidelity 3D models for dynamic and deformable objects if depth maps are provided. Otherwise, it can turn the video clips into stereoscopic video

Approximating displacements in R^3 by rotations in R^4 and its application to pointcloud registration

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksNo proper norm exists to measure the distance between two object poses essentially because a general pose is defined by a rotation and a translation, and thus it involves magnitudes with different units. As a means to solve this dimensional-inhomogeneity problem, the concept of characteristic length has been put forward in the area of kinematics. The idea consists in scaling translations according to this characteristic length and then approximating the corresponding displacement defining the object pose in R^3 by a rotation in R^4, for which a norm exists. This paper sheds new light on this kind of approximations which permits simplifying optimization problem whose cost functions involve translations and rotations simultaneously. A good example of this kind of problems is the pointcloud registration problem in which the optimal rotation and translation between two sets of corresponding 3D point data, so that they are aligned/registered, have to be found. As a result, a simple closed-form formula for solving this problem is presented which is shown to be an attractive alternative to the previous approaches.Peer ReviewedPostprint (author's final draft

A computational study of thrust augmenting ejectors based on a viscous-inviscid approach

A viscous-inviscid interaction technique is advocated as both an efficient and accurate means of predicting the performance of two-dimensional thrust augmenting ejectors. The flow field is subdivided into a viscous region that contains the turbulent jet and an inviscid region that contains the ambient fluid drawn into the device. The inviscid region is computed with a higher-order panel method, while an integral method is used for the description of the viscous part. The strong viscous-inviscid interaction present within the ejector is simulated in an iterative process where the two regions influence each other en route to a converged solution. The model is applied to a variety of parametric and optimization studies involving ejectors having either one or two primary jets. The effects of nozzle placement, inlet and diffuser shape, free stream speed, and ejector length are investigated. The inlet shape for single jet ejectors is optimized for various free stream speeds and Reynolds numbers. Optimal nozzle tilt and location are identified for various dual-ejector configurations

Methods and algorithms for quantitative analysis of metallomic images to assess traumatic brain injury

The primary aim of this thesis is to develop image processing algorithms to quantitatively determine the link between traumatic brain injury (TBI) severity and chronic traumatic encephalopathy (CTE) neuropathology, specifically looking into the role of blood-brain barrier disruption following TBI. In order to causally investigate the relationship between the tau protein neurodegenerative disease CTE and TBI, mouse models of blast neurotrauma (BNT) and impact neurotrauma (INT) are investigated. First, a high-speed video tracking algorithm is developed based on K-means clustering, active contours and Kalman filtering to comparatively study the head kinematics in blast and impact experiments. Then, to compare BNT and INT neuropathology, methods for quantitative analysis of macroscopic optical images and fluorescent images are described. The secondary aim of this thesis focuses on developing methods for a novel application of metallomic imaging mass spectrometry (MIMS) to biological tissue. Unlike traditional modalities used to assess neuropathology, that suffer from limited sensitivity and analytical capacity, MIMS uses a mass spectrometer -- an analytical instrument for measuring elements and isotopes with high dynamic range, sensitivity and specificity -- as the imaging sensor to generate spatial maps with spectral (vector-valued) data per pixel. Given the vector nature of MIMS data, a unique end-to-end processing pipeline is designed to support data acquisition, visualization and interpretation. A novel multi-modal and multi-channel image registration (MMMCIR) method using multi-variate mutual information as a similarity metric is developed in order to establish correspondence between two images of arbitrary modality. The MMMCIR method is then used to automatically segment MIMS images of the mouse brain and systematically evaluate the levels of relevant elements and isotopes after experimental closed-head impact injury on the impact side (ipsilateral) and opposing side (contralateral) of the brain. This method quantifiably confirms observed differences in gadolinium levels for a cohort of images. Finally, MIMS images of human lacrimal sac biopsy samples are used for preliminary clinicopathological assessments, supporting the utility of the unique insights MIMS provides by correlating areas of inflammation to areas of elevated toxic metals. The image processing methods developed in this work demonstrate the significant capabilities of MIMS and its role in enhancing our understanding of the underlying pathological mechanisms of TBI and other medical conditions.2019-07-09T00:00:00

Signal processing with Fourier analysis, novel algorithms and applications

Fourier analysis is the study of the way general functions may be represented or approximated by sums of simpler trigonometric functions, also analogously known as sinusoidal modeling. The original idea of Fourier had a profound impact on mathematical analysis, physics and engineering because it diagonalizes time-invariant convolution operators. In the past signal processing was a topic that stayed almost exclusively in electrical engineering, where only the experts could cancel noise, compress and reconstruct signals. Nowadays it is almost ubiquitous, as everyone now deals with modern digital signals. Medical imaging, wireless communications and power systems of the future will experience more data processing conditions and wider range of applications requirements than the systems of today. Such systems will require more powerful, efficient and flexible signal processing algorithms that are well designed to handle such needs. No matter how advanced our hardware technology becomes we will still need intelligent and efficient algorithms to address the growing demands in signal processing. In this thesis, we investigate novel techniques to solve a suite of four fundamental problems in signal processing that have a wide range of applications. The relevant equations, literature of signal processing applications, analysis and final numerical algorithms/methods to solve them using Fourier analysis are discussed for different applications in the electrical engineering/computer science. The first four chapters cover the following topics of central importance in the field of signal processing: • Fast Phasor Estimation using Adaptive Signal Processing (Chapter 2) • Frequency Estimation from Nonuniform Samples (Chapter 3) • 2D Polar and 3D Spherical Polar Nonuniform Discrete Fourier Transform (Chapter 4) • Robust 3D registration using Spherical Polar Discrete Fourier Transform and Spherical Harmonics (Chapter 5) Even though each of these four methods discussed may seem completely disparate, the underlying motivation for more efficient processing by exploiting the Fourier domain signal structure remains the same. The main contribution of this thesis is the innovation in the analysis, synthesis, discretization of certain well known problems like phasor estimation, frequency estimation, computations of a particular non-uniform Fourier transform and signal registration on the transformed domain. We conduct propositions and evaluations of certain applications relevant algorithms such as, frequency estimation algorithm using non-uniform sampling, polar and spherical polar Fourier transform. The techniques proposed are also useful in the field of computer vision and medical imaging. From a practical perspective, the proposed algorithms are shown to improve the existing solutions in the respective fields where they are applied/evaluated. The formulation and final proposition is shown to have a variety of benefits. Future work with potentials in medical imaging, directional wavelets, volume rendering, video/3D object classifications, high dimensional registration are also discussed in the final chapter. Finally, in the spirit of reproducible research we release the implementation of these algorithms to the public using Github