Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians

Convolutional neural nets (CNNs) have demonstrated remarkable performance in recent history. Such approaches tend to work in a unidirectional bottom-up feed-forward fashion. However, practical experience and biological evidence tells us that feedback plays a crucial role, particularly for detailed spatial understanding tasks. This work explores bidirectional architectures that also reason with top-down feedback: neural units are influenced by both lower and higher-level units. We do so by treating units as rectified latent variables in a quadratic energy function, which can be seen as a hierarchical Rectified Gaussian model (RGs). We show that RGs can be optimized with a quadratic program (QP), that can in turn be optimized with a recurrent neural network (with rectified linear units). This allows RGs to be trained with GPU-optimized gradient descent. From a theoretical perspective, RGs help establish a connection between CNNs and hierarchical probabilistic models. From a practical perspective, RGs are well suited for detailed spatial tasks that can benefit from top-down reasoning. We illustrate them on the challenging task of keypoint localization under occlusions, where local bottom-up evidence may be misleading. We demonstrate state-of-the-art results on challenging benchmarks.Comment: To appear in CVPR 201

Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images

Detection and classification of cell nuclei in histopathology images of cancerous tissue stained with the standard hematoxylin and eosin stain is a challenging task due to cellular heterogeneity. Deep learning approaches have been shown to produce encouraging results on histopathology images in various studies. In this paper, we propose a Spatially Constrained Convolutional Neural Network (SC-CNN) to perform nucleus detection. SC-CNN regresses the likelihood of a pixel being the center of a nucleus, where high probability values are spatially constrained to locate in the vicinity of the center of nuclei. For classification of nuclei, we propose a novel Neighboring Ensemble Predictor (NEP) coupled with CNN to more accurately predict the class label of detected cell nuclei. The proposed approaches for detection and classification do not require segmentation of nuclei. We have evaluated them on a large dataset of colorectal adenocarcinoma images, consisting of more than 20,000 annotated nuclei belonging to four different classes. Our results show that the joint detection and classification of the proposed SC-CNN and NEP produces the highest average F1 score as compared to other recently published approaches. Prospectively, the proposed methods could offer benefit to pathology practice in terms of quantitative analysis of tissue constituents in whole-slide images, and could potentially lead to a better understanding of cancer

Lidar-based Obstacle Detection and Recognition for Autonomous Agricultural Vehicles

Today, agricultural vehicles are available that can drive autonomously and follow exact route plans more precisely than human operators. Combined with advancements in precision agriculture, autonomous agricultural robots can reduce manual labor, improve workflow, and optimize yield. However, as of today, human operators are still required for monitoring the environment and acting upon potential obstacles in front of the vehicle. To eliminate this need, safety must be ensured by accurate and reliable obstacle detection and avoidance systems.In this thesis, lidar-based obstacle detection and recognition in agricultural environments has been investigated. A rotating multi-beam lidar generating 3D point clouds was used for point-wise classification of agricultural scenes, while multi-modal fusion with cameras and radar was used to increase performance and robustness. Two research perception platforms were presented and used for data acquisition. The proposed methods were all evaluated on recorded datasets that represented a wide range of realistic agricultural environments and included both static and dynamic obstacles.For 3D point cloud classification, two methods were proposed for handling density variations during feature extraction. One method outperformed a frequently used generic 3D feature descriptor, whereas the other method showed promising preliminary results using deep learning on 2D range images. For multi-modal fusion, four methods were proposed for combining lidar with color camera, thermal camera, and radar. Gradual improvements in classification accuracy were seen, as spatial, temporal, and multi-modal relationships were introduced in the models. Finally, occupancy grid mapping was used to fuse and map detections globally, and runtime obstacle detection was applied on mapped detections along the vehicle path, thus simulating an actual traversal.The proposed methods serve as a first step towards full autonomy for agricultural vehicles. The study has thus shown that recent advancements in autonomous driving can be transferred to the agricultural domain, when accurate distinctions are made between obstacles and processable vegetation. Future research in the domain has further been facilitated with the release of the multi-modal obstacle dataset, FieldSAFE

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Fast and Robust Automatic Segmentation Methods for MR Images of Injured and Cancerous Tissues

Magnetic Resonance Imaging: MRI) is a key medical imaging technology. Through in vivo soft tissue imaging, MRI allows clinicians and researchers to make diagnoses and evaluations that were previously possible only through biopsy or autopsy. However, analysis of MR images by domain experts can be time-consuming, complex, and subject to bias. The development of automatic segmentation techniques that make use of robust statistical methods allows for fast and unbiased analysis of MR images. In this dissertation, I propose segmentation methods that fall into two classes---(a) segmentation via optimization of a parametric boundary, and: b) segmentation via multistep, spatially constrained intensity classification. These two approaches are applicable in different segmentation scenarios. Parametric boundary segmentation is useful and necessary for segmentation of noisy images where the tissue of interest has predictable shape but poor boundary delineation, as in the case of lung with heavy or diffuse tumor. Spatially constrained intensity classification is appropriate for segmentation of noisy images with moderate contrast between tissue regions, where the areas of interest have unpredictable shapes, as is the case in spinal injury and brain tumor. The proposed automated segmentation techniques address the need for MR image analysis in three specific applications:: 1) preclinical rodent studies of primary and metastatic lung cancer: approach: a)),: 2) preclinical rodent studies of spinal cord lesion: approach: b)), and: 3) postclinical analysis of human brain cancer: approach: b)). In preclinical rodent studies of primary and metastatic lung cancer, respiratory-gated MRI is used to quantitatively measure lung-tumor burden and monitor the time-course progression of individual tumors. I validate a method for measuring tumor burden based upon average lung-image intensity. The method requires accurate lung segmentation; toward this end, I propose an automated lung segmentation method that works for varying tumor burden levels. The method includes development of a novel, two-dimensional parametric model of the mouse lungs and a multifaceted cost function to optimally fit the model parameters to each image. Results demonstrate a strong correlation: 0.93), comparable with that of fully manual expert segmentation, between the automated method\u27s tumor-burden metric and the tumor burden measured by lung weight. In preclinical rodent studies of spinal cord lesion, MRI is used to quantify tissues in control and injured mouse spinal cords. For this application, I propose a novel, multistep, multidimensional approach, utilizing the Classification Expectation Maximization: CEM) algorithm, for automatic segmentation of spinal cord tissues. In contrast to previous methods, my proposed method incorporates prior knowledge of cord geometry and the distinct information contained in the different MR images gathered. Unlike previous approaches, the algorithm is shown to remain accurate for whole spinal cord, white matter, and hemorrhage segmentation, even in the presence of significant injury. The results of the method are shown to be on par with expert manual segmentation. In postclinical analysis of human brain cancer, access to large collections of MRI data enables scientifically rigorous study of cancers like glioblastoma multiforme, the most common form of malignant primary brain tumor. For this application, I propose an efficient and effective automated segmentation method, the Enhanced Classification Expectation Maximization: ECEM) algorithm. The ECEM algorithm is novel in that it introduces spatial information directly into the classical CEM algorithm, which is otherwise spatially unaware, with low additional computational complexity. I compare the ECEM\u27s performance on simulated data to the standard finite Gaussian mixture EM algorithm, which is not spatially aware, and to the hidden-Markov random field EM algorithm, a commonly-used spatially aware automated segmentation method for MR brain images. I also show sample results demonstrating the ECEM algorithm\u27s ability to segment MR images of glioblastoma