Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks

Bilateral filters have wide spread use due to their edge-preserving properties. The common use case is to manually choose a parametric filter type, usually a Gaussian filter. In this paper, we will generalize the parametrization and in particular derive a gradient descent algorithm so the filter parameters can be learned from data. This derivation allows to learn high dimensional linear filters that operate in sparsely populated feature spaces. We build on the permutohedral lattice construction for efficient filtering. The ability to learn more general forms of high-dimensional filters can be used in several diverse applications. First, we demonstrate the use in applications where single filter applications are desired for runtime reasons. Further, we show how this algorithm can be used to learn the pairwise potentials in densely connected conditional random fields and apply these to different image segmentation tasks. Finally, we introduce layers of bilateral filters in CNNs and propose bilateral neural networks for the use of high-dimensional sparse data. This view provides new ways to encode model structure into network architectures. A diverse set of experiments empirically validates the usage of general forms of filters

A superior edge preserving filter with a systematic analysis

A new, adaptive, edge preserving filter for use in image processing is presented. It had superior performance when compared to other filters. Termed the contiguous K-average, it aggregates pixels by examining all pixels contiguous to an existing cluster and adding the pixel closest to the mean of the existing cluster. The process is iterated until K pixels were accumulated. Rather than simply compare the visual results of processing with this operator to other filters, some approaches were developed which allow quantitative evaluation of how well and filter performs. Particular attention is given to the standard deviation of noise within a feature and the stability of imagery under iterative processing. Demonstrations illustrate the performance of several filters to discriminate against noise and retain edges, the effect of filtering as a preprocessing step, and the utility of the contiguous K-average filter when used with remote sensing data

On generalized adaptive neural filter

Linear filters have historically been used in the past as the most useful tools for suppressing noise in signal processing. It has been shown that the optimal filter which minimizes the mean square error (MSE) between the filter output and the desired output is a linear filter provided that the noise is additive white Gaussian noise (AWGN). However, in most signal processing applications, the noise in the channel through which a signal is transmitted is not AWGN; it is not stationary, and it may have unknown characteristics. To overcome the shortcomings of linear filters, nonlinear filters ranging from the median filters to stack filters have been developed. They have been successfully used in a number of applications, such as enhancing the signal-to-noise ratio of the telecommunication receivers, modeling the human vocal tract to synthesize speech in speech processing, and separating out the maternal and fetal electrocardiogram signals to diagnose prenatal ailments. In particular, stack filters have been shown to provide robust noise suppression, and are easily implementable in hardware, but configuring an optimal stack filter remains a challenge. This dissertation takes on this challenge by extending stack filters to a new class of nonlinear adaptive filters called generalized adaptive neural filters (GANFs). The objective of this work is to investigate their performance in terms of the mean absolute error criterion, to evaluate and predict the generalization of various discriminant functions employed for GANFs, and to address issues regarding their applications and implementation. It is shown that GANFs not only extend the class of stack filters, but also have better performance in terms of suppressing non-additive white Gaussian noise. Several results are drawn from the theoretical and experimental work: stack filters can be adaptively configured by neural networks; GANFs encompass a large class of nonlinear sliding-window filters which include stack filters; the mean absolute error (MAE) of the optimal GANF is upper-bounded by that of the optimal stack filter; a suitable class of discriminant functions can be determined before a training scheme is executed; VC dimension (VCdim) theory can be applied to determine the number of training samples; the algorithm presented in configuring GANFs is effective and robust