A Learning Framework for Morphological Operators using Counter-Harmonic Mean

We present a novel framework for learning morphological operators using counter-harmonic mean. It combines concepts from morphology and convolutional neural networks. A thorough experimental validation analyzes basic morphological operators dilation and erosion, opening and closing, as well as the much more complex top-hat transform, for which we report a real-world application from the steel industry. Using online learning and stochastic gradient descent, our system learns both the structuring element and the composition of operators. It scales well to large datasets and online settings.Comment: Submitted to ISMM'1

Morphological Network: How Far Can We Go with Morphological Neurons?

In recent years, the idea of using morphological operations as networks has received much attention. Mathematical morphology provides very efficient and useful image processing and image analysis tools based on basic operators like dilation and erosion, defined in terms of kernels. Many other morphological operations are built up using the dilation and erosion operations. Although the learning of structuring elements such as dilation or erosion using the backpropagation algorithm is not new, the order and the way these morphological operations are used is not standard. In this paper, we have theoretically analyzed the use of morphological operations for processing 1D feature vectors and shown that this gets extended to the 2D case in a simple manner. Our theoretical results show that a morphological block represents a sum of hinge functions. Hinge functions are used in many places for classification and regression tasks (Breiman (1993)). We have also proved a universal approximation theorem -- a stack of two morphological blocks can approximate any continuous function over arbitrary compact sets. To experimentally validate the efficacy of this network in real-life applications, we have evaluated its performance on satellite image classification datasets since morphological operations are very sensitive to geometrical shapes and structures. We have also shown results on a few tasks like segmentation of blood vessels from fundus images, segmentation of lungs from chest x-ray and image dehazing. The results are encouraging and further establishes the potential of morphological networks.Comment: 35 pages, 19 figures, 7 table

Learning morphological operators for depth completion

Depth images generated by direct projection of LiDAR point clouds on the image plane suffer from a great level of sparsity which is difficult to interpret by classical computer vision algorithms. We propose a method for completing sparse depth images in a semantically accurate manner by training a novel morphological neural network. Our method approximates morphological operations by Contraharmonic Mean Filter layers which are easily trained in a contemporary deep learning framework. An early fusion U-Net architecture then combines dilated depth channels and RGB using multi-scale processing. Using a large scale RGBD dataset we are able to learn the optimal morphological and convolutional filter shapes that produce an accurate and fully sampled depth image at the output. Independent experimental evaluation confirms that our method outperforms classical image restoration techniques as well as current state-of-the-art neural networks. The resulting depth images preserve object boundaries and can easily be used to augment various tasks in intelligent vehicles perception systems

Deep morphological neural networks

Mathematical morphology is a theory and technique applied to collect features like geometric and topological structures in digital images. Determining suitable morphological operations and structuring elements for a give purpose is a cumbersome and time-consuming task. In this paper, morphological neural networks are proposed to address this problem. Serving as a non-linear feature extracting layers in deep learning frameworks, the efficiency of the proposed morphological layer is confirmed analytically and empirically. With a known target, a single-filter morphological layer learns the structuring element correctly, and an adaptive layer can automatically select appropriate morphological operations. For high level applications, the proposed morphological neural networks are tested on several classification datasets which are related to shape or geometric image features, and the experimental results have confirmed the tradeoff between high computational efficiency and high accuracy

Learning Deep Morphological Networks with Neural Architecture Search

Deep Neural Networks (DNNs) are generated by sequentially performing linear and non-linear processes. Using a combination of linear and non-linear procedures is critical for generating a sufficiently deep feature space. The majority of non-linear operators are derivations of activation functions or pooling functions. Mathematical morphology is a branch of mathematics that provides non-linear operators for a variety of image processing problems. We investigate the utility of integrating these operations in an end-to-end deep learning framework in this paper. DNNs are designed to acquire a realistic representation for a particular job. Morphological operators give topological descriptors that convey salient information about the shapes of objects depicted in images. We propose a method based on meta-learning to incorporate morphological operators into DNNs. The learned architecture demonstrates how our novel morphological operations significantly increase DNN performance on various tasks, including picture classification and edge detection.Comment: 19 page

Development of deep learning neural network for ecological and medical images

Deep learning in computer vision and image processing has attracted attentions from various fields including ecology and medical image. Ecologists are interested in finding an effective model structure to classify different species. Tradition deep learning model use a convolutional neural network, such as LeNet, AlexNet, VGG models, residual neural network, and inception models, are first used on classifying bee wing and butterfly datasets. However, insufficient data sample and unbalanced samples in each class have caused a poor accuracy. To make improvement the test accuracy, data augmentation and transfer learning are applied. Recently developed deep learning framework based on mathematical morphology also shows its effective in shape representation, contour detection and image smoothing. The experimental results in the morphological neural network shows this type of deep learning model is also effective in ecology datasets and medical dataset. Compared with CNN, the MNN could achieve a similar or better result in the following datasets. The chest X-ray images are notoriously difficult to analyze for the radiologists due to their noisy nature. The existing models based on convolutional neural networks contain a giant number of parameters and thus require multi-advanced GPUs to deploy. In this research, the morphological neural networks are developed to classify chest X-ray images, including the Pneumonia Dataset and the COVID-19 Dataset. A novel structure, which can self-learn a morphological dilation or erosion, is proposed for determining the most suitable depth of the adaptive layer. Experimental results on the chest X-ray dataset and the COVID-19 dataset show that the proposed model achieves the highest classification rate as comparing against the existing models. More significant improvement is that the proposed model reduces around 97% computational parameters of the existing models. Automatic identification of pneumonia on medical images has attracted intensive studies recently. The model for detecting pneumonia requires both a precise classification model and a localization model. A joint-task joint learning model with shared parameters is proposed to combine the classification model and segmentation model. To accurately classify and localize pneumonia area. Experimental results using the massive dataset of Radiology Society of North America have confirmed the efficiency of showing a test mean interception over union (IoU) of 89.27% and a mean precision of area detection result of 58.45% in segmentation model. Then, two new models are proposed to improve the performance of the original joint-task learning model. Two new modules are developed to improve both classification and segmentation accuracies in the first model. These modules including an image preprocessing module and an attention module. In the second model, a novel design is used to combine both convolutional layers and morphological layers with an attention mechanism. Experimental results performed on the massive dataset of the Radiology Society of North America have confirmed its superiority over other existing methods. The classification test accuracy is improved from 0.89 to 0.95, and the segmentation model achieves an improved mean precision result from 0.58 to 0.78. Finally, two weakly-supervised learning methods: class-saliency map and grad-cam, are used to highlight corresponding pixels or areas which have significant influence on the classification model, such that the refined segmentation can focus on the correct areas with high confidence