Automated 3D object modeling from aerial video imagery

Research in physically accurate 3D modeling of a scene is gaining momentum because of its far reaching applications in civilian and defense sectors. The modeled 3D scene must conform both geometrically and spectrally to the real world for all the applications. Geometric modeling of a scene can be achieved in many ways of which the two most popular methods are - a) using multiple 2D passive images of the scene also called as stereo vision and b) using 3D point clouds like Lidar (Light detection and ranging) data. In this research work, we derive the 3D models of objects in a scene using passive aerial video imagery. At present, this geometric modeling requires a lot of manual intervention due to a variety of factors like sensor noise, low contrast conditions during image capture, etc. Hence long time periods, in the order of weeks and months, are required to model even a small scene. This thesis focuses on automating the process of geometric modeling of objects in a scene from passive aerial video imagery. The aerial video frames are stitched into stereo mosaics. These stereo mosaics not only provide the elevation information of a scene but also act as good 3D visualization tools. The 3D information obtained from the stereo mosaics is used to identify the various 3D objects, especially man-made buildings using probabilistic inference provided by Bayesian Networks. The initial 3D building models are further optimized by projecting them on to the individual video frames. The limitations of the state-of-art technology in attaining these goals are presented along with the techniques to overcome them. The improvement that can be achieved in the accuracy of the 3D models when Lidar data is fused with aerial video during the object identification process is also examined

LIMBUSTRACK: STABLE EYE-TRACKING IN IMPERFECT LIGHT CONDITIONS

We are aware of only one serious effort at development of a cheap, accurate, wearable eye tracker: the open source openEyes project. However, its method of ocular feature detection is such that it is prone to failure in variable lighting conditions. To address this deficiency, we have developed a cheap wearable eye tracker. At the heart of our development are novel techniques that allow operation under variable illumination

Multi-scale multi-dimensional imaging and characterization of oil shale pyrolysis

In recent years, oil shale has attracted renewed attention as an unconventional energy resource, with vast and largely untapped reserves. Oil shale is a fine-grained sedimentary rock containing a sufficiently high content of immature organic matter from which shale oil and combustible gas can be extracted through pyrolysis. Several complex physical and chemical changes occur during the pyrolysis of oil shale where macromolecular network structures of kerogen are thermally decomposed. The pyrolysis of oil shale leads to the formation of a microscopic pore network in which the oil and gas products flow. The pore structure and the connectivity are significant characteristics which determine fluid flow and ultimate hydrocarbon recovery. In this thesis, a state-of-the-art multi-scale multi-dimensional workflow was applied to image and quantify the Lacustrine Eocene Green River (Mahogany Zone) formation, the world’s largest oil shale deposit. Samples were imaged before, during and after pyrolysis using laboratory and synchrotron-based X-ray Micro-tomography (µCT), Optical Microscopy, Automated Ultra-High Resolution Scanning Electron Microscopy (SEM), MAPS Mineralogy (Modular Automated Processing System) and Focused Ion Beam Scanning Electron Microscopy (FIB-SEM). Results of image analysis using optical (2-D), SEM (2-D), and µCT (3-D) reveal a complex fine-grained microstructure dominated by organic-rich parallel laminations in a tightly bound heterogeneous mineral matrix. MAPS Mineralogy combined with ultrafast measurements highlighted mineralogic textures dominated by dolomite, calcite, K-feldspar, quartz, pyrite and illitic clays. From high resolution backscattered electron (BSE) images, intra-organic, inter-organic-mineral, intra and inter-mineral pores were characterised with varying sizes and geometries. A detailed X-ray µCT study with increasing pyrolysis temperature (300-500°C) at 12 µm, 2 µm and 0.8 µm voxel sizes illuminated the evolution of pore structure, which is shown to be a strong function of the spatial distribution of organic content. In addition, FIB-SEM 3-D visualisations showed an unconnected pore space of 0.5% with pores sizes between 15 nm and 22 nm for the un-pyrolysed sample and a well-connected pore space of 18.2% largely with pores of equivalent radius between 1.6 µm and 2.0 µm for the pyrolysed sample. Synchrotron 4-D results at a time resolution of 160 seconds and a voxel size of 2 µm revealed a dramatic change in porosity accompanying pyrolysis between 390-400°C with the formation of micron-scale heterogeneous pores followed by interconnected fracture networks predominantly along the organic-rich laminations. Combining these techniques provides a powerful tool for quantifying petrophysical properties before, during and after oil shale pyrolysis. Quantitative 2-D, 3-D and 4-D imaging datasets across nm-µm-mm length scales are of great value to better understand, predict and model dynamics of pore structure change and hydrocarbon transport and production during oil shale pyrolysis.Open Acces

Development, Implementation and Pre-clinical Evaluation of Medical Image Computing Tools in Support of Computer-aided Diagnosis: Respiratory, Orthopedic and Cardiac Applications

Over the last decade, image processing tools have become crucial components of all clinical and research efforts involving medical imaging and associated applications. The imaging data available to the radiologists continue to increase their workload, raising the need for efficient identification and visualization of the required image data necessary for clinical assessment. Computer-aided diagnosis (CAD) in medical imaging has evolved in response to the need for techniques that can assist the radiologists to increase throughput while reducing human error and bias without compromising the outcome of the screening, diagnosis or disease assessment. More intelligent, but simple, consistent and less time-consuming methods will become more widespread, reducing user variability, while also revealing information in a more clear, visual way. Several routine image processing approaches, including localization, segmentation, registration, and fusion, are critical for enhancing and enabling the development of CAD techniques. However, changes in clinical workflow require significant adjustments and re-training and, despite the efforts of the academic research community to develop state-of-the-art algorithms and high-performance techniques, their footprint often hampers their clinical use. Currently, the main challenge seems to not be the lack of tools and techniques for medical image processing, analysis, and computing, but rather the lack of clinically feasible solutions that leverage the already developed and existing tools and techniques, as well as a demonstration of the potential clinical impact of such tools. Recently, more and more efforts have been dedicated to devising new algorithms for localization, segmentation or registration, while their potential and much intended clinical use and their actual utility is dwarfed by the scientific, algorithmic and developmental novelty that only result in incremental improvements over already algorithms. In this thesis, we propose and demonstrate the implementation and evaluation of several different methodological guidelines that ensure the development of image processing tools --- localization, segmentation and registration --- and illustrate their use across several medical imaging modalities --- X-ray, computed tomography, ultrasound and magnetic resonance imaging --- and several clinical applications: Lung CT image registration in support for assessment of pulmonary nodule growth rate and disease progression from thoracic CT images. Automated reconstruction of standing X-ray panoramas from multi-sector X-ray images for assessment of long limb mechanical axis and knee misalignment. Left and right ventricle localization, segmentation, reconstruction, ejection fraction measurement from cine cardiac MRI or multi-plane trans-esophageal ultrasound images for cardiac function assessment. When devising and evaluating our developed tools, we use clinical patient data to illustrate the inherent clinical challenges associated with highly variable imaging data that need to be addressed before potential pre-clinical validation and implementation. In an effort to provide plausible solutions to the selected applications, the proposed methodological guidelines ensure the development of image processing tools that help achieve sufficiently reliable solutions that not only have the potential to address the clinical needs, but are sufficiently streamlined to be potentially translated into eventual clinical tools provided proper implementation. G1: Reducing the number of degrees of freedom (DOF) of the designed tool, with a plausible example being avoiding the use of inefficient non-rigid image registration methods. This guideline addresses the risk of artificial deformation during registration and it clearly aims at reducing complexity and the number of degrees of freedom. G2: The use of shape-based features to most efficiently represent the image content, either by using edges instead of or in addition to intensities and motion, where useful. Edges capture the most useful information in the image and can be used to identify the most important image features. As a result, this guideline ensures a more robust performance when key image information is missing. G3: Efficient method of implementation. This guideline focuses on efficiency in terms of the minimum number of steps required and avoiding the recalculation of terms that only need to be calculated once in an iterative process. An efficient implementation leads to reduced computational effort and improved performance. G4: Commence the workflow by establishing an optimized initialization and gradually converge toward the final acceptable result. This guideline aims to ensure reasonable outcomes in consistent ways and it avoids convergence to local minima, while gradually ensuring convergence to the global minimum solution. These guidelines lead to the development of interactive, semi-automated or fully-automated approaches that still enable the clinicians to perform final refinements, while they reduce the overall inter- and intra-observer variability, reduce ambiguity, increase accuracy and precision, and have the potential to yield mechanisms that will aid with providing an overall more consistent diagnosis in a timely fashion

Helical Optical Projection Tomography

A new technique termed Helical Optical Projection Tomography (hOPT) has been developed with the aim to overcome some of the limitations of current 3D optical imaging techniques. hOPT is based on Optical Projection Tomography (OPT) with the major difference that there is a translation of the sample in the vertical direction during the image acquisition process, requiring a new approach to image reconstruction. Contrary to OPT, hOPT makes possible to obtain 3D-optical images of intact long samples without imposing limits on the sample length. This has been tested using hOPT to image long murine tissue samples such as spinal cords and large intestines. Moreover, 3D-reconstructed images of the colon of DSS-treated mice, a model for Inflammatory Bowel Disease, allowed the identification of the structural alterations. Finally, the geometry of the hOPT device facilitates the addition of a Selective Plane Illumination Microscopy (SPIM) arm, providing the possibility of delivering high resolution images of selected areas together with complete volumetric informationThis work was partially supported by EC FP7 collaborative grant FMT-XCT and the Bill and Melinda Gates foundation. A.A. wishes to acknowledge support from Marie Curie IEF-2010-275137. J.R. wishes to acknowledge support from EC FP7 IMI project PREDICT-TB, and the EC FP7 CIG grant HIGH-THROUGHPUT TOMO. D.D., S.Z. and J.T. acknowledge support from the National Basic Research Program of China (973 Program) under Grant 2011CB707700, the Fellowship for Young International Scientists of the Chinese Academy of Sciences under Grant 2010Y2GA03, the National Natural Science Foundation of China under Grant 81101084 and Instrument Developing Project of the Chinese Academy of Sciences under Grant No. YZ201164Publicad

High-throughput phenotyping technology for corn ears

The phenotype of any organism, or as in this case, plants, includes traits or characteristics that can be measured using a technical procedure. Phenotyping is an important activity in plant breeding, since it gives breeders an observable representation of the plant’s genetic code, which is called the genotype. The word phenotype originates from the Greek word “phainein” which means “to show” and the word “typos” which means “type”. Ideally, the development of phenotyping technologies should be in lockstep with genotyping technologies, but unfortunately it is not; currently there exists a major discrepancy between the technological sophistication of genotyping versus phenotyping, and the gap is getting wider. Whereas genotyping has become a high-throughput low-cost standardized procedure, phenotyping still comprises ample manual measurements which are time consuming, tedious, and error prone. The project as conducted here aims at alleviating this problem; To aid breeders, a method was devised that allows for high-throughput phenotyping of corn ears, based on an existing imaging arrangement that produces frontal views of the ears. This thesis describes the development of machine vision algorithms that measure overall ear parameters such as ear length, ear diameter, and cap percentage (the proportion of the ear that features kernels versus the barren area). The main image processing functions used here were segmentation, skewness correction, morphological operation and image registration. To obtain a kernel count, an “ear map” was constructed using both a morphological operation and a feature matching operation. The main challenge for the morphological operation was to accurately select only kernel rows that are frontally exposed in each single image. This issue is addressed in this project by developing an algorithm of shadow recognition. The main challenge for the feature-matching operation was to detect and match image feature points. This issue was addressed by applying the algorithms of Harris’s Conner detection and SIFT descriptor. Once the ear map is created, many other morphological kernel parameters (area, location, circumference, to name a few) can be determined. Remaining challenges in this research are pointed out, including sample choice, apparatus modification and algorithm improvement. Suggestions and recommendations for future work are also provided

Aspects of automation in the shoe industry

The shoe manufacturing industry has undergone a revolution during the last 50 years, due to the introduction of task specific machinery. Great technological strides have been made in the areas of shoe manufacture prior to actual component assembly. Computer systems are now becoming the norm for the design of shoes for today's market place. Technological innovations have also started to be applied in the assembly and construction processes of modern shoes. Computer controlled cutting machines calculate the optimum usage of leather from any given hide, new machines allow decorative stitch patterns to be associated with a given shape and size of component and automatically stitched on to the presented workpiece. However the majority of assembly operations have remained predominantly manual with technology playing a secondary role to the human operator due to complexities either in manipulation, control or sensing. In these machines electronic and mechanical innovations have been used to add new features to often simple machines and in some cases to simplify some of the more complex operations, thus increasing productivity but reducing the required dexterity and knowledge of an operator. Modern preferences in industry are to utilise fully automated machines, that are as operator independent as possible, thus improving quality, consistency and production speed whilst at the same time reducing production costs.Due to the nature of the shoe manufacturing industry and the complex operations that have to be performed in order to construct a shoe, machinery manufacturers who have ventured into this field of automation have generally struggled to gain acceptance from the shoe makers as the machinery is generally complex and slow in operation. This together with the fact that a large proportion of the world's main footwear production is centred in the far east, with their correspondingly low labour costs, has held back the automation of the shoe  manufacturing industry.This thesis examines a selection of operations encountered in the construction of a typical shoe. These include operations for processing single flat component parts as well as more complex three-dimensional operations encountered when lasting and soling a shoe. The aim of the research was to develop an understanding of processes encountered in specific areas within the shoe manufacturing industry in order to identify areas where further advances in automation could be achieved. This understanding has been applied to produce proposals and in some cases hardware, to allow for the development of working systems