Learning Multi-Scale Representations for Material Classification

The recent progress in sparse coding and deep learning has made unsupervised feature learning methods a strong competitor to hand-crafted descriptors. In computer vision, success stories of learned features have been predominantly reported for object recognition tasks. In this paper, we investigate if and how feature learning can be used for material recognition. We propose two strategies to incorporate scale information into the learning procedure resulting in a novel multi-scale coding procedure. Our results show that our learned features for material recognition outperform hand-crafted descriptors on the FMD and the KTH-TIPS2 material classification benchmarks

Programmable Spectrometry -- Per-pixel Classification of Materials using Learned Spectral Filters

Many materials have distinct spectral profiles. This facilitates estimation of the material composition of a scene at each pixel by first acquiring its hyperspectral image, and subsequently filtering it using a bank of spectral profiles. This process is inherently wasteful since only a set of linear projections of the acquired measurements contribute to the classification task. We propose a novel programmable camera that is capable of producing images of a scene with an arbitrary spectral filter. We use this camera to optically implement the spectral filtering of the scene's hyperspectral image with the bank of spectral profiles needed to perform per-pixel material classification. This provides gains both in terms of acquisition speed --- since only the relevant measurements are acquired --- and in signal-to-noise ratio --- since we invariably avoid narrowband filters that are light inefficient. Given training data, we use a range of classical and modern techniques including SVMs and neural networks to identify the bank of spectral profiles that facilitate material classification. We verify the method in simulations on standard datasets as well as real data using a lab prototype of the camera

Material Classification of Magnetic Resonance Volume Data

A major unsolved problem in computer graphics is that of making high-quality models. Traditionally, models have consisted of interactively or algorithmically described collections of graphics primitives such as polygons. The process of constructing these models is painstaking and often misses features and behavior that we wish to model. Models extracted from volume data collected from real, physical objects have the potential to show features and behavior that are difficult to capture using these traditional modeling methods. We use vector-valued magnetic resonance volume data in this thesis. The process of extracting models from such data involves four main steps: collecting the sampled volume data; preprocessing it to reduce artifacts from the collection process; classifying materials within the data; and creating either a rigid geometric model that is static, or a flexible, dynamic model that can be simulated. In this thesis we focus on the the first three steps. We present guidelines and techniques for collecting and processing magnetic resonance data to meet the needs of the later steps. Our material classification and model extraction techniques work better when the data values for a given material are constant throughout the dataset, when data values for different materials are different, and when the dataset is free of aliasing artifacts and noise. We present a new material-classification method that operates on vector-valued volume data. The method produces a continuous probability function for each material over the volume of the dataset, and requires no expert interaction to teach it different material classes. It operates by fitting peaks in the histogram of a collected dataset using parameterized gaussian bumps, and by using Bayes' law to calculate material probabilities, with each gaussian bump representing one material. To illustrate the classification method, we apply it to real magnetic resonance data of a human head, a human hand, a banana, and a jade plant. From the classified data, we produce "computationally stained" slices that discriminate among materials better than do the original grey-scale versions. We also generate volume-rendered images of classified datasets clearly showing different anatomical features of various materials. Finally, we extract preliminary static and dynamic geometric models of different tissues

Fuzzy spectral and spatial feature integration for classification of nonferrous materials in hyperspectral data

Hyperspectral data allows the construction of more elaborate models to sample the properties of the nonferrous materials than the standard RGB color representation. In this paper, the nonferrous waste materials are studied as they cannot be sorted by classical procedures due to their color, weight and shape similarities. The experimental results presented in this paper reveal that factors such as the various levels of oxidization of the waste materials and the slight differences in their chemical composition preclude the use of the spectral features in a simplistic manner for robust material classification. To address these problems, the proposed FUSSER (fuzzy spectral and spatial classifier) algorithm detailed in this paper merges the spectral and spatial features to obtain a combined feature vector that is able to better sample the properties of the nonferrous materials than the single pixel spectral features when applied to the construction of multivariate Gaussian distributions. This approach allows the implementation of statistical region merging techniques in order to increase the performance of the classification process. To achieve an efficient implementation, the dimensionality of the hyperspectral data is reduced by constructing bio-inspired spectral fuzzy sets that minimize the amount of redundant information contained in adjacent hyperspectral bands. The experimental results indicate that the proposed algorithm increased the overall classification rate from 44% using RGB data up to 98% when the spectral-spatial features are used for nonferrous material classification

Evaluating Deep Convolutional Neural Networks for Material Classification

Determining the material category of a surface from an image is a demanding task in perception that is drawing increasing attention. Following the recent remarkable results achieved for image classification and object detection utilising Convolutional Neural Networks (CNNs), we empirically study material classification of everyday objects employing these techniques. More specifically, we conduct a rigorous evaluation of how state-of-the art CNN architectures compare on a common ground over widely used material databases. Experimental results on three challenging material databases show that the best performing CNN architectures can achieve up to 94.99% mean average precision when classifying materials