Data Driven Multispectral Image Registration Framework

Multispectral imaging is widely used in remote sensing applications from UAVs and ground-based platforms. Multispectral cameras often use a physically different camera for each wavelength causing misalignment in the images for different imaging bands. This misalignment must be corrected prior to concurrent multi-band image analysis. The traditional approach for multispectral image registration process is to select a target channel and register all other image channels to the target. There is no objective evidence-based method to select a target channel. The possibility of registration to some intermediate channel before registering to the target is not usually considered, but could be beneficial if there is no target channel for which direct registration performs well for every other channel. In this paper, we propose an automatic data-driven multispectral image registration framework that determines a target channel, and possible intermediate registration steps based on the assumptions that 1) some reasonable minimum number of control-points correspondences between two channels is needed to ensure a low-error registration; 2) a greater number of such correspondences generally results in higher registration performance. Our prototype is tested on five multispectral datasets captured with UAV-mounted multispectral cameras. The output of the prototype is a registration scheme in the form of a directed acyclic graph (actually a tree) that represents the target channel and the process to register other image channels. The resulting registration schemes had more control point correspondences on average than the traditional register-all-to-one-targetchannel approach. Data-driven registration scheme consistently showed low back-projection error across all the image channel pairs in most of the experiments. Our data-driven framework has generated registration schemes with the best control point extraction algorithm for each image channel pair and registering images in a data-driven approach. The data-driven image registration framework is dataset independent, and it performs on datasets with any number of image channels. With the growing need of remote sensing and the lack of a proper evidence-based method to register multispectral image channels, a data-driven registration framework is an essential tool in the field of image registration and multispectral imaging

Correction, Validation, and Characterization of Motion in Resting-State Functional Magnetic Resonance Images of Pediatric Patients

There are many scenarios, for both clinical and research applications, in which we would like to examine a patient's neurodevelopmental status. Generally, neurodevelopmental evaluations can be performed through psychological testing or in-person assessment with a psychologist. However, these approaches are not applicable in all cases, particularly for many pediatric populations. Researchers are beginning to turn to medical imaging approaches for objectively quantifying a patient's neurodevelopmental status. Resting-state functional magnetic resonance images (rs-fMRIs) can be used to study neuronal networks that are active even when a person is not performing a specific task or reacting to particular stimuli. These image sequences are highly sensitive to motion. Techniques have been developed to prevent patients from moving as well as monitor motion during the scan and correct for the patient's movement after the scan. We focus on the first step of retrospective motion correction: volume registration. The purpose of volume registration is to align the contents of all of the image volumes in the image sequence to the contents of a single volume. Traditionally, all image volumes are directly registered to the chosen stationary image volume. However, this approach does not account for significant differences in patient position between the stationary volume and the other volumes in the sequence. We developed a registration framework based on the concept of a directed acyclic graph. We treat the volumes in the sequence as nodes in a graph where pairs of subsequent volumes are connected via directed edges. This perspective allows us to model the relationships between subsequent volumes and account for them during registration. We applied both registration frameworks to a set of simulated images as well as neurological rs-fMRIs from three clinical populations. The clinical populations were preadolescent, neonatal, and fetal subjects who either were healthy or had congenital heart disease (CHD). The original and registered sequences were compared with respect to their local and global motion. The local motion was measured between every pair of image volumes $i$ and $i+1$ in each sequence using the framewise displacement (FD) and the derivative of the root mean square variance of the signal (DVARS). The global motion across each sequence was measured by calculating the similarity between every pair of image volumes in each sequence. The local motion parameters were compared to a pair of gold standard usability thresholds to determine how each registration framework impacted the usability of every image volume. Both the local and global motion parameters were used to determine how many sequences had statistically significant differences in their motion distributions before and after registration. Additionally, the local and global metrics of the original sequences were clustered to determine if a computer could identify groups of subjects based on their motion parameters. The registration frameworks had different effects on each age group of subjects. We found that the neonatal subjects contained the least amount of motion, while the fetal subjects contained the most motion. The DAG-based registration was most effective at reducing motion in the fetal images. Our clustering analysis showed that the different age groups have different global motion parameters, though lifespan-level patterns related to CHD status could not be detected

Modeling Brain Circuitry over a Wide Range of Scales

If we are ever to unravel the mysteries of brain function at its most fundamental level, we will need a precise understanding of how its component neurons connect to each other. Electron Microscopes (EM) can now provide the nanometer resolution that is needed to image synapses, and therefore connections, while Light Microscopes (LM) see at the micrometer resolution required to model the 3D structure of the dendritic network. Since both the topology and the connection strength are integral parts of the brain's wiring diagram, being able to combine these two modalities is critically important. In fact, these microscopes now routinely produce high-resolution imagery in such large quantities that the bottleneck becomes automated processing and interpretation, which is needed for such data to be exploited to its full potential. In this paper, we briefly review the Computer Vision techniques we have developed at EPFL to address this need. They include delineating dendritic arbors from LM imagery, segmenting organelles from EM, and combining the two into a consistent representation

Improving 6D Pose Estimation of Objects in Clutter via Physics-aware Monte Carlo Tree Search

This work proposes a process for efficiently searching over combinations of individual object 6D pose hypotheses in cluttered scenes, especially in cases involving occlusions and objects resting on each other. The initial set of candidate object poses is generated from state-of-the-art object detection and global point cloud registration techniques. The best-scored pose per object by using these techniques may not be accurate due to overlaps and occlusions. Nevertheless, experimental indications provided in this work show that object poses with lower ranks may be closer to the real poses than ones with high ranks according to registration techniques. This motivates a global optimization process for improving these poses by taking into account scene-level physical interactions between objects. It also implies that the Cartesian product of candidate poses for interacting objects must be searched so as to identify the best scene-level hypothesis. To perform the search efficiently, the candidate poses for each object are clustered so as to reduce their number but still keep a sufficient diversity. Then, searching over the combinations of candidate object poses is performed through a Monte Carlo Tree Search (MCTS) process that uses the similarity between the observed depth image of the scene and a rendering of the scene given the hypothesized pose as a score that guides the search procedure. MCTS handles in a principled way the tradeoff between fine-tuning the most promising poses and exploring new ones, by using the Upper Confidence Bound (UCB) technique. Experimental results indicate that this process is able to quickly identify in cluttered scenes physically-consistent object poses that are significantly closer to ground truth compared to poses found by point cloud registration methods.Comment: 8 pages, 4 figure