3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks

We propose a method for reconstructing 3D shapes from 2D sketches in the form of line drawings. Our method takes as input a single sketch, or multiple sketches, and outputs a dense point cloud representing a 3D reconstruction of the input sketch(es). The point cloud is then converted into a polygon mesh. At the heart of our method lies a deep, encoder-decoder network. The encoder converts the sketch into a compact representation encoding shape information. The decoder converts this representation into depth and normal maps capturing the underlying surface from several output viewpoints. The multi-view maps are then consolidated into a 3D point cloud by solving an optimization problem that fuses depth and normals across all viewpoints. Based on our experiments, compared to other methods, such as volumetric networks, our architecture offers several advantages, including more faithful reconstruction, higher output surface resolution, better preservation of topology and shape structure.Comment: 3DV 2017 (oral

Achieving Information Security by multi-Modal Iris-Retina Biometric Approach Using Improved Mask R-CNN

The need for reliable user recognition (identification/authentication) techniques has grown in response to heightened security concerns and accelerated advances in networking, communication, and mobility. Biometrics, defined as the science of recognizing an individual based on his or her physical or behavioral characteristics, is gaining recognition as a method for determining an individual\u27s identity. Various commercial, civilian, and forensic applications now use biometric systems to establish identity. The purpose of this paper is to design an efficient multimodal biometric system based on iris and retinal features to assure accurate human recognition and improve the accuracy of recognition using deep learning techniques. Deep learning models were tested using retinographies and iris images acquired from the MESSIDOR and CASIA-IrisV1 databases for the same person. The Iris region was segmented from the image using the custom Mask R-CNN method, and the unique blood vessels were segmented from retinal images of the same person using principal curvature. Then, in order to aid precise recognition, they optimally extract significant information from the segmented images of the iris and retina. The suggested model attained 98% accuracy, 98.1% recall, and 98.1% precision. It has been discovered that using a custom Mask R-CNN approach on Iris-Retina images improves efficiency and accuracy in person recognition

GAL: A Stepwise Model for Automated Cloud Shadow Detection in HICO Oceanic Imagery Utilizing Guided Filter, Pixel Assignment, and Geometric Linking

Detection of cloud shadow pixels is an important step in image processing in several remote sensing ocean-color application domains, such as obtaining chlorophyll content. While shadow detection algorithms do exist, the vast majority are for over land which leaves few options for detection over water. The detection of cloud shadow over water in HICO imagery is a unique problem. As its name implies, HICO (Hyperspectral Imager for the Coastal Ocean) imagery is produced for coastal and oceanic regions. Since land based algorithms remove water before processing, these approaches would not be applicable. The only currently published HICO shadow pixel detection algorithm [1] produces good results for predominantly homogeneous regions. It also involves hand-tuning of the parameters, which is not suitable for automation. GAL is a fully automated stepwise model that starts by using satellite imagery and navigational data. The next step is applying the guided filter algorithm proposed by He, Sun, and Tang [2] to these images in order to filter and enhance the images before shadow detection. The third step classifies pixels into water, land, and clouds. The fourth step uses cloud shadow geometry to indicate possible shadow pixels. The final step is to reduce the amount of possible shadow pixels to the most probable shadow pixels. This research combines the past techniques of cloud shadow geometry, edge detection, and thresholding, along with the new techniques of guided image filtering, in such a way that has never been done before. GAL works best with well-defined cloud shadows that contain a large contrast between water and shadow. Water type, coastal or deep ocean, does not affect GAL. Shadows with a large gradient may be under-detected. GAL can be applied to HICO data immediately, with the potential of being applied to all global high resolution ocean-color satellite imagery

Visibility in underwater robotics: Benchmarking and single image dehazing

Dealing with underwater visibility is one of the most important challenges in autonomous underwater robotics. The light transmission in the water medium degrades images making the interpretation of the scene difficult and consequently compromising the whole intervention. This thesis contributes by analysing the impact of the underwater image degradation in commonly used vision algorithms through benchmarking. An online framework for underwater research that makes possible to analyse results under different conditions is presented. Finally, motivated by the results of experimentation with the developed framework, a deep learning solution is proposed capable of dehazing a degraded image in real time restoring the original colors of the image.Una de las dificultades más grandes de la robótica autónoma submarina es lidiar con la falta de visibilidad en imágenes submarinas. La transmisión de la luz en el agua degrada las imágenes dificultando el reconocimiento de objetos y en consecuencia la intervención. Ésta tesis se centra en el análisis del impacto de la degradación de las imágenes submarinas en algoritmos de visión a través de benchmarking, desarrollando un entorno de trabajo en la nube que permite analizar los resultados bajo diferentes condiciones. Teniendo en cuenta los resultados obtenidos con este entorno, se proponen métodos basados en técnicas de aprendizaje profundo para mitigar el impacto de la degradación de las imágenes en tiempo real introduciendo un paso previo que permita recuperar los colores originales

A Low-cost Depth Imaging Mobile Platform for Canola Phenotyping

To meet the high demand for supporting and accelerating progress in the breeding of novel traits, plant scientists and breeders have to measure a large number of plants and their characteristics accurately. A variety of imaging methodologies are being deployed to acquire data for quantitative studies of complex traits. When applied to a large number of plants such as canola plants, however, a complete three-dimensional (3D) model is time-consuming and expensive for high-throughput phenotyping with an enormous amount of data. In some contexts, a full rebuild of entire plants may not be necessary. In recent years, many 3D plan phenotyping techniques with high cost and large-scale facilities have been introduced to extract plant phenotypic traits, but these applications may be affected by limited research budgets and cross environments. This thesis proposed a low-cost depth and high-throughput phenotyping mobile platform to measure canola plant traits in cross environments. Methods included detecting and counting canola branches and seedpods, monitoring canola growth stages, and fusing color images to improve images resolution and achieve higher accuracy. Canola plant traits were examined in both controlled environment and field scenarios. These methodologies were enhanced by different imaging techniques. Results revealed that this phenotyping mobile platform can be used to investigate canola plant traits in cross environments with high accuracy. The results also show that algorithms for counting canola branches and seedpods enable crop researchers to analyze the relationship between canola genotypes and phenotypes and estimate crop yields. In addition to counting algorithms, fusing techniques can be helpful for plant breeders with more comfortable access plant characteristics by improving the definition and resolution of color images. These findings add value to the automation, low-cost depth and high-throughput phenotyping for canola plants. These findings also contribute a novel multi-focus image fusion that exhibits a competitive performance with outperforms some other state-of-the-art methods based on the visual saliency maps and gradient domain fast guided filter. This proposed platform and counting algorithms can be applied to not only canola plants but also other closely related species. The proposed fusing technique can be extended to other fields, such as remote sensing and medical image fusion

An Open Source, Autonomous, Vision-Based Algorithm for Hazard Detection and Avoidance for Celestial Body Landing

Planetary exploration is one of the main goals that humankind has established as a must for space exploration in order to be prepared for colonizing new places and provide scientific data for a better understanding of the formation of our solar system. In order to provide a safe approach, several safety measures must be undertaken to guarantee not only the success of the mission but also the safety of the crew. One of these safety measures is the Autonomous Hazard, Detection, and Avoidance (HDA) sub-system for celestial body landers that will enable different spacecraft to complete solar system exploration. The main objective of the HDA sub-system is to assemble a map of the local terrain during the descent of the spacecraft so that a safe landing site can be marked down. This thesis will be focused on a passive method using a monocular camera as its primary detection sensor due to its form factor and weight, which enables its implementation alongside the proposed HDA algorithm in the Intuitive Machines lunar lander NOVA-C as part of the Commercial Lunar Payload Services technological demonstration in 2021 for the NASA Artemis program to take humans back to the moon. This algorithm is implemented by including two different sources for making decisions, a two-dimensional (2D) vision-based HDA map and a three-dimensional (3D) HDA map obtained through a Structure from Motion process in combination with a plane fitting sequence. These two maps will provide different metrics in order to provide the lander a better probability of performing a safe touchdown. These metrics are processed to optimize a cost function

Artificial Intelligence and Augmented Reality: A Possible Continuum for the Enhancement of Architectural Heritage

Augmented reality and artificial intelligence technologies are increasingly involved in the interpretation, classification, and eventually in our understanding of the built environment. This paper collects three recent on-site research experiences, which developed mobile app prototypes and experimented with these advancements in the relationship between computer vision and architectural artifacts. The first project concerns the vaulted atria of Baroque Turin and integrates a work of analysis and representation with augmented reality applications to visualize interpretative 3D models. The second project focuses on the development of a mobile app to access data on monuments, exploiting deep learning technologies for image recognition. The application field is the Imperial Fora in Rome. The third project is part of the so-called "third mission" of universities, being developed in partnership with a company operating in the electronic telecommunications infrastructure sector. The project explores automatic methods for updating existing building information modeling databases on antenna tower sites. This experience, the last in chronological order, though focused on a modern architectural asset, is meant to reframe and integrate the previous ones, updating the methods and the foreseen developments, and stressing the potential of the combined use of the studied technologies. The joint consideration of the three projects is aimed at reflecting on the processes, which from different techniques for the recognition of spatial features, produce schematic understanding and operational uses of the shape of built heritage

3D/2D Registration of Mapping Catheter Images for Arrhythmia Interventional Assistance

Radiofrequency (RF) catheter ablation has transformed treatment for tachyarrhythmias and has become first-line therapy for some tachycardias. The precise localization of the arrhythmogenic site and the positioning of the RF catheter over that site are problematic: they can impair the efficiency of the procedure and are time consuming (several hours). Electroanatomic mapping technologies are available that enable the display of the cardiac chambers and the relative position of ablation lesions. However, these are expensive and use custom-made catheters. The proposed methodology makes use of standard catheters and inexpensive technology in order to create a 3D volume of the heart chamber affected by the arrhythmia. Further, we propose a novel method that uses a priori 3D information of the mapping catheter in order to estimate the 3D locations of multiple electrodes across single view C-arm images. The monoplane algorithm is tested for feasibility on computer simulations and initial canine data.Comment: International Journal of Computer Science Issues, IJCSI, Volume 4, Issue 2, pp10-19, September 200

Treatment planning study of cyberKnife prostate SBRT (stereotactic body radiation therapy) using CT-based vs MRI-based prostate volumes

This study has been conducted for the purpose of investigating the systematic dose reduction of rectum and neurovascular bundles (NVBs) during treatment planning of the CyberKnifeTM prostate SBRT using CT-Based volumes versus MRI-based volumes. Three prostate cancer patients were Planned for the CyberKnifeTM prostate SBRT and they underwent computed tomography (CT) and magnetic resonance imaging (MRI) preplanning exams. The patients were positioned during both exams using an immobilizing device. A radiation oncologist and a radiologist delineated the prostate gland, intra-prostatic and peri-prostatic structures, and pelvic organs of interest in both CT and MRI images. The CT and MRI images were fused based on fuducial markers to accurately align the prostate. Radiation Therapy Oncology protocol RTOG 0938 was followed to meet the target volume (prostate plus margin) dose coverage requirement, and dose-volume constraints for organs at risk, including rectum, bladder, femoral heads, penile bulb, urethra, skin and NVBs. Radiation dose volume parameters were recorded for both volumes and compared. The preliminary result shows that the CT-based volumes were generally larger than MRI-based volumes of the prostate. Therefore, the CT-based volumes resulted in less accurate treatment planning and dose delivery to radiosensitive structures

Clear Cell Renal Cell Carcinoma: Deep Learning-Based Prediction of Tumor Grade from Contrast-Enhanced CT

Tumor grading is an important prognostic parameter for renal cell carcinoma (RCC). However, current grading schemes require an invasive surgical procedure, putting patients at risks including increased risk of hemorrhage, infection, renal failure, or death. Furthermore, low grade RCC is indolent with low mortality risk and may not require treatment. Therefore, a pre-operative and non-invasive assessment of malignancy grade may be beneficial and facilitate optimal timing of treatment. In recent years, deep learning-based image analysis has gained wide popularity in cancer prognosis and prediction. The goal of this study is to investigate the feasibility and performance of a deep-learning-based model for clear cell RCC grading prediction from contrast-enhance computed tomography (CECT). After institutional review board approval, an institutional pathology database was queried for all renal biopsies between December 2002 and October 2018. All included patients received a CT with a non-contrast and at least one post-contrast series. All patients have Fuhrman grade confirmation from surgical pathology, with CT scan prior to the procedure. Tumors were manually annotated by a radiologist on either the corticomedullary or nephrographic phase CECT. Rectangular regions of interest (ROI) were drawn on each slice throughout the tumor and used as inputs to the Deep CNN ResNet50. A binary label of low grade or high grade was assigned to each patient. Sensitivity, specificity, accuracy, and AUC were calculated based on a five-fold cross-validation. Preliminary results from a small subset of data suggests that a deep learning model can be used to predict clear cell RCC grading based on CT imaging prior to surgical procedures