41,092 research outputs found
Visual Concepts and Compositional Voting
It is very attractive to formulate vision in terms of pattern theory
\cite{Mumford2010pattern}, where patterns are defined hierarchically by
compositions of elementary building blocks. But applying pattern theory to real
world images is currently less successful than discriminative methods such as
deep networks. Deep networks, however, are black-boxes which are hard to
interpret and can easily be fooled by adding occluding objects. It is natural
to wonder whether by better understanding deep networks we can extract building
blocks which can be used to develop pattern theoretic models. This motivates us
to study the internal representations of a deep network using vehicle images
from the PASCAL3D+ dataset. We use clustering algorithms to study the
population activities of the features and extract a set of visual concepts
which we show are visually tight and correspond to semantic parts of vehicles.
To analyze this we annotate these vehicles by their semantic parts to create a
new dataset, VehicleSemanticParts, and evaluate visual concepts as unsupervised
part detectors. We show that visual concepts perform fairly well but are
outperformed by supervised discriminative methods such as Support Vector
Machines (SVM). We next give a more detailed analysis of visual concepts and
how they relate to semantic parts. Following this, we use the visual concepts
as building blocks for a simple pattern theoretical model, which we call
compositional voting. In this model several visual concepts combine to detect
semantic parts. We show that this approach is significantly better than
discriminative methods like SVM and deep networks trained specifically for
semantic part detection. Finally, we return to studying occlusion by creating
an annotated dataset with occlusion, called VehicleOcclusion, and show that
compositional voting outperforms even deep networks when the amount of
occlusion becomes large.Comment: It is accepted by Annals of Mathematical Sciences and Application
An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams
Existing FNNs are mostly developed under a shallow network configuration
having lower generalization power than those of deep structures. This paper
proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be
automatically extracted from data streams or removed if they play limited role
during their lifespan. The structure of the network can be deepened on demand
by stacking additional layers using a drift detection method which not only
detects the covariate drift, variations of input space, but also accurately
identifies the real drift, dynamic changes of both feature space and target
space. DEVFNN is developed under the stacked generalization principle via the
feature augmentation concept where a recently developed algorithm, namely
gClass, drives the hidden layer. It is equipped by an automatic feature
selection method which controls activation and deactivation of input attributes
to induce varying subsets of input features. A deep network simplification
procedure is put forward using the concept of hidden layer merging to prevent
uncontrollable growth of dimensionality of input space due to the nature of
feature augmentation approach in building a deep network structure. DEVFNN
works in the sample-wise fashion and is compatible for data stream
applications. The efficacy of DEVFNN has been thoroughly evaluated using seven
datasets with non-stationary properties under the prequential test-then-train
protocol. It has been compared with four popular continual learning algorithms
and its shallow counterpart where DEVFNN demonstrates improvement of
classification accuracy. Moreover, it is also shown that the concept drift
detection method is an effective tool to control the depth of network structure
while the hidden layer merging scenario is capable of simplifying the network
complexity of a deep network with negligible compromise of generalization
performance.Comment: This paper has been published in IEEE Transactions on Fuzzy System
Mining and Analyzing the Italian Parliament: Party Structure and Evolution
The roll calls of the Italian Parliament in the XVI legislature are studied
by employing multidimensional scaling, hierarchical clustering, and network
analysis. In order to detect changes in voting behavior, the roll calls have
been divided in seven periods of six months each. All the methods employed
pointed out an increasing fragmentation of the political parties endorsing the
previous government that culminated in its downfall. By using the concept of
modularity at different resolution levels, we identify the community structure
of Parliament and its evolution in each of the considered time periods. The
analysis performed revealed as a valuable tool in detecting trends and drifts
of Parliamentarians. It showed its effectiveness at identifying political
parties and at providing insights on the temporal evolution of groups and their
cohesiveness, without having at disposal any knowledge about political
membership of Representatives.Comment: 27 pages, 14 figure
Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments
The feasibility of deep neural networks (DNNs) to address data stream
problems still requires intensive study because of the static and offline
nature of conventional deep learning approaches. A deep continual learning
algorithm, namely autonomous deep learning (ADL), is proposed in this paper.
Unlike traditional deep learning methods, ADL features a flexible structure
where its network structure can be constructed from scratch with the absence of
an initial network structure via the self-constructing network structure. ADL
specifically addresses catastrophic forgetting by having a different-depth
structure which is capable of achieving a trade-off between plasticity and
stability. Network significance (NS) formula is proposed to drive the hidden
nodes growing and pruning mechanism. Drift detection scenario (DDS) is put
forward to signal distributional changes in data streams which induce the
creation of a new hidden layer. The maximum information compression index
(MICI) method plays an important role as a complexity reduction module
eliminating redundant layers. The efficacy of ADL is numerically validated
under the prequential test-then-train procedure in lifelong environments using
nine popular data stream problems. The numerical results demonstrate that ADL
consistently outperforms recent continual learning methods while characterizing
the automatic construction of network structures
Contextual Information Retrieval based on Algorithmic Information Theory and Statistical Outlier Detection
The main contribution of this paper is to design an Information Retrieval
(IR) technique based on Algorithmic Information Theory (using the Normalized
Compression Distance- NCD), statistical techniques (outliers), and novel
organization of data base structure. The paper shows how they can be integrated
to retrieve information from generic databases using long (text-based) queries.
Two important problems are analyzed in the paper. On the one hand, how to
detect "false positives" when the distance among the documents is very low and
there is actual similarity. On the other hand, we propose a way to structure a
document database which similarities distance estimation depends on the length
of the selected text. Finally, the experimental evaluations that have been
carried out to study previous problems are shown.Comment: Submitted to 2008 IEEE Information Theory Workshop (6 pages, 6
figures
Keep Ballots Secret: On the Futility of Social Learning in Decision Making by Voting
We show that social learning is not useful in a model of team binary decision
making by voting, where each vote carries equal weight. Specifically, we
consider Bayesian binary hypothesis testing where agents have any
conditionally-independent observation distribution and their local decisions
are fused by any L-out-of-N fusion rule. The agents make local decisions
sequentially, with each allowed to use its own private signal and all precedent
local decisions. Though social learning generally occurs in that precedent
local decisions affect an agent's belief, optimal team performance is obtained
when all precedent local decisions are ignored. Thus, social learning is
futile, and secret ballots are optimal. This contrasts with typical studies of
social learning because we include a fusion center rather than concentrating on
the performance of the latest-acting agents
SENATUS: An Approach to Joint Traffic Anomaly Detection and Root Cause Analysis
In this paper, we propose a novel approach, called SENATUS, for joint traffic
anomaly detection and root-cause analysis. Inspired from the concept of a
senate, the key idea of the proposed approach is divided into three stages:
election, voting and decision. At the election stage, a small number of
\nop{traffic flow sets (termed as senator flows)}senator flows are chosen\nop{,
which are used} to represent approximately the total (usually huge) set of
traffic flows. In the voting stage, anomaly detection is applied on the senator
flows and the detected anomalies are correlated to identify the most possible
anomalous time bins. Finally in the decision stage, a machine learning
technique is applied to the senator flows of each anomalous time bin to find
the root cause of the anomalies. We evaluate SENATUS using traffic traces
collected from the Pan European network, GEANT, and compare against another
approach which detects anomalies using lossless compression of traffic
histograms. We show the effectiveness of SENATUS in diagnosing anomaly types:
network scans and DoS/DDoS attacks
- …