Optical Image Blending for Underwater Mosaics

Typical problems for creation of consistent underwater mosaic are misalignment and inhomogeneous illumination of the image frames, which causes visible seams and consequently complicates post-processing of the mosaics such as object recognition and shape extraction. Two recently developed image blending methods were explored in the literature: gradient domain stitching and graph-cut method, and they allow for improvement of illumination inconsistency and ghosting effects, respectively. However, due to the specifics of underwater imagery, these two methods cannot be used within a straightforward manner. In this paper, a new improved blending algorithm is proposed based on these two methods. By comparing with the previous methods from a perceptual point of view and as a potential input for pattern recognition algorithms, our results show an improvement in decreasing the mosaic degradation due to feature doubling and rapid illumination change

A Markov Random Field Based Approach to 3D Mosaicing and Registration Applied to Ultrasound Simulation

A novel Markov Random Field (MRF) based method for the mosaicing of 3D ultrasound volumes is presented in this dissertation. The motivation for this work is the production of training volumes for an affordable ultrasound simulator, which offers a low-cost/portable training solution for new users of diagnostic ultrasound, by providing the scanning experience essential for developing the necessary psycho-motor skills. It also has the potential for introducing ultrasound instruction into medical education curriculums. The interest in ultrasound training stems in part from the widespread adoption of point-of-care scanners, i.e. low cost portable ultrasound scanning systems in the medical community. This work develops a novel approach for producing 3D composite image volumes and validates the approach using clinically acquired fetal images from the obstetrics department at the University of Massachusetts Medical School (UMMS). Results using the Visible Human Female dataset as well as an abdominal trauma phantom are also presented. The process is broken down into five distinct steps, which include individual 3D volume acquisition, rigid registration, calculation of a mosaicing function, group-wise non-rigid registration, and finally blending. Each of these steps, common in medical image processing, has been investigated in the context of ultrasound mosaicing and has resulted in improved algorithms. Rigid and non-rigid registration methods are analyzed in a probabilistic framework and their sensitivity to ultrasound shadowing artifacts is studied. The group-wise non-rigid registration problem is initially formulated as a maximum likelihood estimation, where the joint probability density function is comprised of the partially overlapping ultrasound image volumes. This expression is simplified using a block-matching methodology and the resulting discrete registration energy is shown to be equivalent to a Markov Random Field. Graph based methods common in computer vision are then used for optimization, resulting in a set of transformations that bring the overlapping volumes into alignment. This optimization is parallelized using a fusion approach, where the registration problem is divided into 8 independent sub-problems whose solutions are fused together at the end of each iteration. This method provided a speedup factor of 3.91 over the single threaded approach with no noticeable reduction in accuracy during our simulations. Furthermore, the registration problem is simplified by introducing a mosaicing function, which partitions the composite volume into regions filled with data from unique partially overlapping source volumes. This mosaicing functions attempts to minimize intensity and gradient differences between adjacent sources in the composite volume. Experimental results to demonstrate the performance of the group-wise registration algorithm are also presented. This algorithm is initially tested on deformed abdominal image volumes generated using a finite element model of the Visible Human Female to show the accuracy of its calculated displacement fields. In addition, the algorithm is evaluated using real ultrasound data from an abdominal phantom. Finally, composite obstetrics image volumes are constructed using clinical scans of pregnant subjects, where fetal movement makes registration/mosaicing especially difficult. Our solution to blending, which is the final step of the mosaicing process, is also discussed. The trainee will have a better experience if the volume boundaries are visually seamless, and this usually requires some blending prior to stitching. Also, regions of the volume where no data was collected during scanning should have an ultrasound-like appearance before being displayed in the simulator. This ensures the trainee\u27s visual experience isn\u27t degraded by unrealistic images. A discrete Poisson approach has been adapted to accomplish these tasks. Following this, we will describe how a 4D fetal heart image volume can be constructed from swept 2D ultrasound. A 4D probe, such as the Philips X6-1 xMATRIX Array, would make this task simpler as it can acquire 3D ultrasound volumes of the fetal heart in real-time; However, probes such as these aren\u27t widespread yet. Once the theory has been introduced, we will describe the clinical component of this dissertation. For the purpose of acquiring actual clinical ultrasound data, from which training datasets were produced, 11 pregnant subjects were scanned by experienced sonographers at the UMMS following an approved IRB protocol. First, we will discuss the software/hardware configuration that was used to conduct these scans, which included some custom mechanical design. With the data collected using this arrangement we generated seamless 3D fetal mosaics, that is, the training datasets, loaded them into our ultrasound training simulator, and then subsequently had them evaluated by the sonographers at the UMMS for accuracy. These mosaics were constructed from the raw scan data using the techniques previously introduced. Specific training objectives were established based on the input from our collaborators in the obstetrics sonography group. Important fetal measurements are reviewed, which form the basis for training in obstetrics ultrasound. Finally clinical images demonstrating the sonographer making fetal measurements in practice, which were acquired directly by the Philips iU22 ultrasound machine from one of our 11 subjects, are compared with screenshots of corresponding images produced by our simulator

A Markov Random Field Based Approach to 3D Mosaicing and Registration Applied to Ultrasound Simulation

A novel Markov Random Field (MRF) based method for the mosaicing of 3D ultrasound volumes is presented in this dissertation. The motivation for this work is the production of training volumes for an affordable ultrasound simulator, which offers a low-cost/portable training solution for new users of diagnostic ultrasound, by providing the scanning experience essential for developing the necessary psycho-motor skills. It also has the potential for introducing ultrasound instruction into medical education curriculums. The interest in ultrasound training stems in part from the widespread adoption of point-of-care scanners, i.e. low cost portable ultrasound scanning systems in the medical community. This work develops a novel approach for producing 3D composite image volumes and validates the approach using clinically acquired fetal images from the obstetrics department at the University of Massachusetts Medical School (UMMS). Results using the Visible Human Female dataset as well as an abdominal trauma phantom are also presented. The process is broken down into five distinct steps, which include individual 3D volume acquisition, rigid registration, calculation of a mosaicing function, group-wise non-rigid registration, and finally blending. Each of these steps, common in medical image processing, has been investigated in the context of ultrasound mosaicing and has resulted in improved algorithms. Rigid and non-rigid registration methods are analyzed in a probabilistic framework and their sensitivity to ultrasound shadowing artifacts is studied. The group-wise non-rigid registration problem is initially formulated as a maximum likelihood estimation, where the joint probability density function is comprised of the partially overlapping ultrasound image volumes. This expression is simplified using a block-matching methodology and the resulting discrete registration energy is shown to be equivalent to a Markov Random Field. Graph based methods common in computer vision are then used for optimization, resulting in a set of transformations that bring the overlapping volumes into alignment. This optimization is parallelized using a fusion approach, where the registration problem is divided into 8 independent sub-problems whose solutions are fused together at the end of each iteration. This method provided a speedup factor of 3.91 over the single threaded approach with no noticeable reduction in accuracy during our simulations. Furthermore, the registration problem is simplified by introducing a mosaicing function, which partitions the composite volume into regions filled with data from unique partially overlapping source volumes. This mosaicing functions attempts to minimize intensity and gradient differences between adjacent sources in the composite volume. Experimental results to demonstrate the performance of the group-wise registration algorithm are also presented. This algorithm is initially tested on deformed abdominal image volumes generated using a finite element model of the Visible Human Female to show the accuracy of its calculated displacement fields. In addition, the algorithm is evaluated using real ultrasound data from an abdominal phantom. Finally, composite obstetrics image volumes are constructed using clinical scans of pregnant subjects, where fetal movement makes registration/mosaicing especially difficult. Our solution to blending, which is the final step of the mosaicing process, is also discussed. The trainee will have a better experience if the volume boundaries are visually seamless, and this usually requires some blending prior to stitching. Also, regions of the volume where no data was collected during scanning should have an ultrasound-like appearance before being displayed in the simulator. This ensures the trainee\u27s visual experience isn\u27t degraded by unrealistic images. A discrete Poisson approach has been adapted to accomplish these tasks. Following this, we will describe how a 4D fetal heart image volume can be constructed from swept 2D ultrasound. A 4D probe, such as the Philips X6-1 xMATRIX Array, would make this task simpler as it can acquire 3D ultrasound volumes of the fetal heart in real-time; However, probes such as these aren\u27t widespread yet. Once the theory has been introduced, we will describe the clinical component of this dissertation. For the purpose of acquiring actual clinical ultrasound data, from which training datasets were produced, 11 pregnant subjects were scanned by experienced sonographers at the UMMS following an approved IRB protocol. First, we will discuss the software/hardware configuration that was used to conduct these scans, which included some custom mechanical design. With the data collected using this arrangement we generated seamless 3D fetal mosaics, that is, the training datasets, loaded them into our ultrasound training simulator, and then subsequently had them evaluated by the sonographers at the UMMS for accuracy. These mosaics were constructed from the raw scan data using the techniques previously introduced. Specific training objectives were established based on the input from our collaborators in the obstetrics sonography group. Important fetal measurements are reviewed, which form the basis for training in obstetrics ultrasound. Finally clinical images demonstrating the sonographer making fetal measurements in practice, which were acquired directly by the Philips iU22 ultrasound machine from one of our 11 subjects, are compared with screenshots of corresponding images produced by our simulator

Structure-Aware Image Compositing

The classic task of image compositing is complicated by the fact that the source and target images need to be carefully aligned and adjusted. Otherwise, it is not possible to achieve convincing results. Visual artifacts are caused by image intensity mismatch, image distortion or structure misalignment even if the images have been globally aligned. In this paper we extend classic Poisson blending by a constrained structure deformation and propagation method. This approach can solve the above-mentioned problems and proves useful for a variety of applications, e.g. in de-ghosting of mosaic images, classic image compositing or other applications such as superresolution from image databases. Our method is based on the following basic steps. First, an optimal partitioning boundary is computed between the input images. Then, features along this boundary are robustly aligned and deformation vectors are computed. Starting at these features, salient edges are traced and aligned, serving as additional constraints for the smooth deformation field, which is propagated robustly and smoothly into the interior of the target image. If very different images are to be stitched, we propose to base the deformation constraints on the curvature of the salient edges for C1-continuity of the structures between the images. We present results that show the robustness of our method on a number of image stitching and compositing tasks.Bildkompositionen werden oft dadurch erschwert, dass die zwei zusammenzufügenden Bilder vorsichtig und genau justiert und angepasst werden müssen. Andernfalls können keine überzeugenden Ergebnisse erreicht werden. Visuelle Artefakte können durch unterschiedliche Intensitäten, Verzerrungen oder strukturelle Missanpassungen entstehen, selbst wenn die Bilder global ausgerichtet wurden. In diesem Report wird das klassische Poisson Blending erweitert durch eine gezielte Strukturdeformation und Propagierung derselben. Dieser Ansatz löst die oben genannten Probleme und ist sinnvoll einsetzbar in einer Reihe von Anwendungen wie Mosaikbilder, Bildkompositionen, oder sogar Super-Resolution durch Bilddatenbanken. Die Methode basiert auf folgenden Schritten: Zunächst wir eine optimale Partitionierung berechnet zwischen den Eingabebildern, dann werden die wichtigen Features entlang dieser Grenze in Einklang gebracht und entsprechende Deformationsvektoren berechnet. Ausgehend von diesen Features werden die auffälligen Kanten weiterverfolgt und in Einklang gebracht. Diese Deformationen werden dann über das gesamte Bild ausgebreitet. Wenn die Bilder stark unterschiedlich sind schlagen wir in diesem Report vor die Verformung auf Basis der Krümmung der Kanten zu durchzuführen. Es werden Ergebnisse präsentiert, die die Robustheit der Methode zeigen

Aspects of automation in the shoe industry

The shoe manufacturing industry has undergone a revolution during the last 50 years, due to the introduction of task specific machinery. Great technological strides have been made in the areas of shoe manufacture prior to actual component assembly. Computer systems are now becoming the norm for the design of shoes for today's market place. Technological innovations have also started to be applied in the assembly and construction processes of modern shoes. Computer controlled cutting machines calculate the optimum usage of leather from any given hide, new machines allow decorative stitch patterns to be associated with a given shape and size of component and automatically stitched on to the presented workpiece. However the majority of assembly operations have remained predominantly manual with technology playing a secondary role to the human operator due to complexities either in manipulation, control or sensing. In these machines electronic and mechanical innovations have been used to add new features to often simple machines and in some cases to simplify some of the more complex operations, thus increasing productivity but reducing the required dexterity and knowledge of an operator. Modern preferences in industry are to utilise fully automated machines, that are as operator independent as possible, thus improving quality, consistency and production speed whilst at the same time reducing production costs.Due to the nature of the shoe manufacturing industry and the complex operations that have to be performed in order to construct a shoe, machinery manufacturers who have ventured into this field of automation have generally struggled to gain acceptance from the shoe makers as the machinery is generally complex and slow in operation. This together with the fact that a large proportion of the world's main footwear production is centred in the far east, with their correspondingly low labour costs, has held back the automation of the shoe  manufacturing industry.This thesis examines a selection of operations encountered in the construction of a typical shoe. These include operations for processing single flat component parts as well as more complex three-dimensional operations encountered when lasting and soling a shoe. The aim of the research was to develop an understanding of processes encountered in specific areas within the shoe manufacturing industry in order to identify areas where further advances in automation could be achieved. This understanding has been applied to produce proposals and in some cases hardware, to allow for the development of working systems

Blending techniques for underwater photomosaics

The creation of consistent underwater photomosaics is typically hampered by local misalignments and inhomogeneous illumination of the image frames, which introduce visible seams that complicate post processing of the mosaics for object recognition and shape extraction. In this thesis, methods are proposed to improve blending techniques for underwater photomosaics and the results are compared with traditional methods. Five specific techniques drawn from various areas of image processing, computer vision, and computer graphics have been tested: illumination correction based on the median mosaic, thin plate spline warping, perspective warping, graph-cut applied in the gradient domain and in the wavelet domain. A combination of the first two methods yields globally homogeneous underwater photomosaics with preserved continuous features. Further improvements are obtained with the graph-cut technique applied in the spatial domain

Piecewise planar underwater mosaicing

A commonly ignored problem in planar mosaics, yet often present in practice, is the selection of a reference homography reprojection frame where to attach the successive image frames of the mosaic. A bad choice for the reference frame can lead to severe distortions in the mosaic and can degenerate in incorrect configurations after some sequential frame concatenations. This problem is accentuated in uncontrolled underwater acquisition setups as those provided by AUVs or ROVs due to both the noisy trajectory of the acquisition vehicle - with roll and pitch shakes - and to the non-flat nature of the seabed which tends to break the planarity assumption implicit in the mosaic construction. These scenarios can also introduce other undesired effects, such as light variations between successive frames, scattering and attenuation, vignetting, flickering and noise. This paper proposes a novel mosaicing pipeline, also including a strategy to select the best reference homography in planar mosaics from video sequences which minimizes the distortions induced on each image by the mosaic homography itself. Moreover, a new non-linear color correction scheme is incorporated to handle strong color and luminosity variations among the mosaic frames. Experimental evaluation of the proposed method on real, challenging underwater video sequences shows the validity of the approach, providing clear and visually appealing mosaic

Three-dimensional-printed gas dynamic virtual nozzles for x-ray laser sample delivery

Reliable sample delivery is essential to biological imaging using X-ray Free Electron Lasers (XFELs). Continuous injection using the Gas Dynamic Virtual Nozzle (GDVN) has proven valuable, particularly for time-resolved studies. However, many important aspects of GDVN functionality have yet to be thoroughly understood and/or refined due to fabrication limitations. We report the application of 2-photon polymerization as a form of high-resolution 3D printing to fabricate high-fidelity GDVNs with submicron resolution. This technique allows rapid prototyping of a wide range of different types of nozzles from standard CAD drawings and optimization of crucial dimensions for optimal performance. Three nozzles were tested with pure water to determine general nozzle performance and reproducibility, with nearly reproducible off-axis jetting being the result. X-ray tomography and index matching were successfully used to evaluate the interior nozzle structures and identify the cause of off-axis jetting. Subsequent refinements to fabrication resulted in straight jetting. A performance test of printed nozzles at an XFEL provided high quality femtosecond diffraction patterns. (C) 2016 Optical Society of Americ