146 research outputs found
Land cover classification using fuzzy rules and aggregation of contextual information through evidence theory
Land cover classification using multispectral satellite image is a very
challenging task with numerous practical applications. We propose a multi-stage
classifier that involves fuzzy rule extraction from the training data and then
generation of a possibilistic label vector for each pixel using the fuzzy rule
base. To exploit the spatial correlation of land cover types we propose four
different information aggregation methods which use the possibilistic class
label of a pixel and those of its eight spatial neighbors for making the final
classification decision. Three of the aggregation methods use Dempster-Shafer
theory of evidence while the remaining one is modeled after the fuzzy k-NN
rule. The proposed methods are tested with two benchmark seven channel
satellite images and the results are found to be quite satisfactory. They are
also compared with a Markov random field (MRF) model-based contextual
classification method and found to perform consistently better.Comment: 14 pages, 2 figure
Feature selection simultaneously preserving both class and cluster structures
When a data set has significant differences in its class and cluster
structure, selecting features aiming only at the discrimination of classes
would lead to poor clustering performance, and similarly, feature selection
aiming only at preserving cluster structures would lead to poor classification
performance. To the best of our knowledge, a feature selection method that
simultaneously considers class discrimination and cluster structure
preservation is not available in the literature. In this paper, we have tried
to bridge this gap by proposing a neural network-based feature selection method
that focuses both on class discrimination and structure preservation in an
integrated manner. In addition to assessing typical classification problems, we
have investigated its effectiveness on band selection in hyperspectral images.
Based on the results of the experiments, we may claim that the proposed
feature/band selection can select a subset of features that is good for both
classification and clustering
Nonlinear Dimensionality Reduction for Data Visualization: An Unsupervised Fuzzy Rule-based Approach
Here, we propose an unsupervised fuzzy rule-based dimensionality reduction
method primarily for data visualization. It considers the following important
issues relevant to dimensionality reduction-based data visualization: (i)
preservation of neighborhood relationships, (ii) handling data on a non-linear
manifold, (iii) the capability of predicting projections for new test data
points, (iv) interpretability of the system, and (v) the ability to reject test
points if required. For this, we use a first-order Takagi-Sugeno type model. We
generate rule antecedents using clusters in the input data. In this context, we
also propose a new variant of the Geodesic c-means clustering algorithm. We
estimate the rule parameters by minimizing an error function that preserves the
inter-point geodesic distances (distances over the manifold) as Euclidean
distances on the projected space. We apply the proposed method on three
synthetic and three real-world data sets and visually compare the results with
four other standard data visualization methods. The obtained results show that
the proposed method behaves desirably and performs better than or comparable to
the methods compared with. The proposed method is found to be robust to the
initial conditions. The predictability of the proposed method for test points
is validated by experiments. We also assess the ability of our method to reject
output points when it should. Then, we extend this concept to provide a general
framework for learning an unsupervised fuzzy model for data projection with
different objective functions. To the best of our knowledge, this is the first
attempt to manifold learning using unsupervised fuzzy modeling
Understanding the classes better with class-specific and rule-specific feature selection, and redundancy control in a fuzzy rule based framework
Recently, several studies have claimed that using class-specific feature
subsets provides certain advantages over using a single feature subset for
representing the data for a classification problem. Unlike traditional feature
selection methods, the class-specific feature selection methods select an
optimal feature subset for each class. Typically class-specific feature
selection (CSFS) methods use one-versus-all split of the data set that leads to
issues such as class imbalance, decision aggregation, and high computational
overhead. We propose a class-specific feature selection method embedded in a
fuzzy rule-based classifier, which is free from the drawbacks associated with
most existing class-specific methods. Additionally, our method can be adapted
to control the level of redundancy in the class-specific feature subsets by
adding a suitable regularizer to the learning objective. Our method results in
class-specific rules involving class-specific subsets. We also propose an
extension where different rules of a particular class are defined by different
feature subsets to model different substructures within the class. The
effectiveness of the proposed method has been validated through experiments on
three synthetic data sets
Group-Feature (Sensor) Selection With Controlled Redundancy Using Neural Networks
In this paper, we present a novel embedded feature selection method based on
a Multi-layer Perceptron (MLP) network and generalize it for group-feature or
sensor selection problems, which can control the level of redundancy among the
selected features or groups. Additionally, we have generalized the group lasso
penalty for feature selection to encompass a mechanism for selecting valuable
group features while simultaneously maintaining a control over redundancy. We
establish the monotonicity and convergence of the proposed algorithm, with a
smoothed version of the penalty terms, under suitable assumptions. Experimental
results on several benchmark datasets demonstrate the promising performance of
the proposed methodology for both feature selection and group feature selection
over some state-of-the-art methods
Two generalizations of Kohonen clustering
The relationship between the sequential hard c-means (SHCM), learning vector quantization (LVQ), and fuzzy c-means (FCM) clustering algorithms is discussed. LVQ and SHCM suffer from several major problems. For example, they depend heavily on initialization. If the initial values of the cluster centers are outside the convex hull of the input data, such algorithms, even if they terminate, may not produce meaningful results in terms of prototypes for cluster representation. This is due in part to the fact that they update only the winning prototype for every input vector. The impact and interaction of these two families with Kohonen's self-organizing feature mapping (SOFM), which is not a clustering method, but which often leads ideas to clustering algorithms is discussed. Then two generalizations of LVQ that are explicitly designed as clustering algorithms are presented; these algorithms are referred to as generalized LVQ = GLVQ; and fuzzy LVQ = FLVQ. Learning rules are derived to optimize an objective function whose goal is to produce 'good clusters'. GLVQ/FLVQ (may) update every node in the clustering net for each input vector. Neither GLVQ nor FLVQ depends upon a choice for the update neighborhood or learning rate distribution - these are taken care of automatically. Segmentation of a gray tone image is used as a typical application of these algorithms to illustrate the performance of GLVQ/FLVQ
Image segmentation using fuzzy LVQ clustering networks
In this note we formulate image segmentation as a clustering problem. Feature vectors extracted from a raw image are clustered into subregions, thereby segmenting the image. A fuzzy generalization of a Kohonen learning vector quantization (LVQ) which integrates the Fuzzy c-Means (FCM) model with the learning rate and updating strategies of the LVQ is used for this task. This network, which segments images in an unsupervised manner, is thus related to the FCM optimization problem. Numerical examples on photographic and magnetic resonance images are given to illustrate this approach to image segmentation
Convolutional Neural Networks Exploiting Attributes of Biological Neurons
In this era of artificial intelligence, deep neural networks like
Convolutional Neural Networks (CNNs) have emerged as front-runners, often
surpassing human capabilities. These deep networks are often perceived as the
panacea for all challenges. Unfortunately, a common downside of these networks
is their ''black-box'' character, which does not necessarily mirror the
operation of biological neural systems. Some even have millions/billions of
learnable (tunable) parameters, and their training demands extensive data and
time.
Here, we integrate the principles of biological neurons in certain layer(s)
of CNNs. Specifically, we explore the use of neuro-science-inspired
computational models of the Lateral Geniculate Nucleus (LGN) and simple cells
of the primary visual cortex. By leveraging such models, we aim to extract
image features to use as input to CNNs, hoping to enhance training efficiency
and achieve better accuracy. We aspire to enable shallow networks with a
Push-Pull Combination of Receptive Fields (PP-CORF) model of simple cells as
the foundation layer of CNNs to enhance their learning process and performance.
To achieve this, we propose a two-tower CNN, one shallow tower and the other as
ResNet 18. Rather than extracting the features blindly, it seeks to mimic how
the brain perceives and extracts features. The proposed system exhibits a
noticeable improvement in the performance (on an average of ) on
CIFAR-10, CIFAR-100, and ImageNet-100 datasets compared to ResNet-18. We also
check the efficiency of only the Push-Pull tower of the network.Comment: 20 pages, 6 figure
Discovering biomarkers from gene expression data for predicting cancer subgroups using neural networks and relational fuzzy clustering
BACKGROUND: The four heterogeneous childhood cancers, neuroblastoma, non-Hodgkin lymphoma, rhabdomyosarcoma, and Ewing sarcoma present a similar histology of small round blue cell tumor (SRBCT) and thus often leads to misdiagnosis. Identification of biomarkers for distinguishing these cancers is a well studied problem. Existing methods typically evaluate each gene separately and do not take into account the nonlinear interaction between genes and the tools that are used to design the diagnostic prediction system. Consequently, more genes are usually identified as necessary for prediction. We propose a general scheme for finding a small set of biomarkers to design a diagnostic system for accurate classification of the cancer subgroups. We use multilayer networks with online gene selection ability and relational fuzzy clustering to identify a small set of biomarkers for accurate classification of the training and blind test cases of a well studied data set. RESULTS: Our method discerned just seven biomarkers that precisely categorized the four subgroups of cancer both in training and blind samples. For the same problem, others suggested 19–94 genes. These seven biomarkers include three novel genes (NAB2, LSP1 and EHD1 – not identified by others) with distinct class-specific signatures and important role in cancer biology, including cellular proliferation, transendothelial migration and trafficking of MHC class antigens. Interestingly, NAB2 is downregulated in other tumors including Non-Hodgkin lymphoma and Neuroblastoma but we observed moderate to high upregulation in a few cases of Ewing sarcoma and Rabhdomyosarcoma, suggesting that NAB2 might be mutated in these tumors. These genes can discover the subgroups correctly with unsupervised learning, can differentiate non-SRBCT samples and they perform equally well with other machine learning tools including support vector machines. These biomarkers lead to four simple human interpretable rules for the diagnostic task. CONCLUSION: Although the proposed method is tested on a SRBCT data set, it is quite general and can be applied to other cancer data sets. Our scheme takes into account the interaction between genes as well as that between genes and the tool and thus is able find a very small set and can discover novel genes. Our findings suggest the possibility of developing specialized microarray chips or use of real-time qPCR assays or antibody based methods such as ELISA and western blot analysis for an easy and low cost diagnosis of the subgroups
- …