2 research outputs found
A Spectral Framework for Anomalous Subgraph Detection
A wide variety of application domains are concerned with data consisting of
entities and their relationships or connections, formally represented as
graphs. Within these diverse application areas, a common problem of interest is
the detection of a subset of entities whose connectivity is anomalous with
respect to the rest of the data. While the detection of such anomalous
subgraphs has received a substantial amount of attention, no
application-agnostic framework exists for analysis of signal detectability in
graph-based data. In this paper, we describe a framework that enables such
analysis using the principal eigenspace of a graph's residuals matrix, commonly
called the modularity matrix in community detection. Leveraging this analytical
tool, we show that the framework has a natural power metric in the spectral
norm of the anomalous subgraph's adjacency matrix (signal power) and of the
background graph's residuals matrix (noise power). We propose several
algorithms based on spectral properties of the residuals matrix, with more
computationally expensive techniques providing greater detection power.
Detection and identification performance are presented for a number of signal
and noise models, including clusters and bipartite foregrounds embedded into
simple random backgrounds as well as graphs with community structure and
realistic degree distributions. The trends observed verify intuition gleaned
from other signal processing areas, such as greater detection power when the
signal is embedded within a less active portion of the background. We
demonstrate the utility of the proposed techniques in detecting small, highly
anomalous subgraphs in real graphs derived from Internet traffic and product
co-purchases.Comment: In submission to the IEEE, 16 pages, 8 figure
Network Detection Theory and Performance
Network detection is an important capability in many areas of applied
research in which data can be represented as a graph of entities and
relationships. Oftentimes the object of interest is a relatively small subgraph
in an enormous, potentially uninteresting background. This aspect characterizes
network detection as a "big data" problem. Graph partitioning and network
discovery have been major research areas over the last ten years, driven by
interest in internet search, cyber security, social networks, and criminal or
terrorist activities. The specific problem of network discovery is addressed as
a special case of graph partitioning in which membership in a small subgraph of
interest must be determined. Algebraic graph theory is used as the basis to
analyze and compare different network detection methods. A new Bayesian network
detection framework is introduced that partitions the graph based on prior
information and direct observations. The new approach, called space-time threat
propagation, is proved to maximize the probability of detection and is
therefore optimum in the Neyman-Pearson sense. This optimality criterion is
compared to spectral community detection approaches which divide the global
graph into subsets or communities with optimal connectivity properties. We also
explore a new generative stochastic model for covert networks and analyze using
receiver operating characteristics the detection performance of both classes of
optimal detection techniques.Comment: Submitted to IEEE Trans. Signal Processin