51,028 research outputs found
Trajectory Clustering and an Application to Airspace Monitoring
This paper presents a framework aimed at monitoring the behavior of aircraft
in a given airspace. Nominal trajectories are determined and learned using data
driven methods. Standard procedures are used by air traffic controllers (ATC)
to guide aircraft, ensure the safety of the airspace, and to maximize the
runway occupancy. Even though standard procedures are used by ATC, the control
of the aircraft remains with the pilots, leading to a large variability in the
flight patterns observed. Two methods to identify typical operations and their
variability from recorded radar tracks are presented. This knowledge base is
then used to monitor the conformance of current operations against operations
previously identified as standard. A tool called AirTrajectoryMiner is
presented, aiming at monitoring the instantaneous health of the airspace, in
real time. The airspace is "healthy" when all aircraft are flying according to
the nominal procedures. A measure of complexity is introduced, measuring the
conformance of current flight to nominal flight patterns. When an aircraft does
not conform, the complexity increases as more attention from ATC is required to
ensure a safe separation between aircraft.Comment: 15 pages, 20 figure
Clustering by compression
We present a new method for clustering based on compression. The method
doesn't use subject-specific features or background knowledge, and works as
follows: First, we determine a universal similarity distance, the normalized
compression distance or NCD, computed from the lengths of compressed data files
(singly and in pairwise concatenation). Second, we apply a hierarchical
clustering method. The NCD is universal in that it is not restricted to a
specific application area, and works across application area boundaries. A
theoretical precursor, the normalized information distance, co-developed by one
of the authors, is provably optimal but uses the non-computable notion of
Kolmogorov complexity. We propose precise notions of similarity metric, normal
compressor, and show that the NCD based on a normal compressor is a similarity
metric that approximates universality. To extract a hierarchy of clusters from
the distance matrix, we determine a dendrogram (binary tree) by a new quartet
method and a fast heuristic to implement it. The method is implemented and
available as public software, and is robust under choice of different
compressors. To substantiate our claims of universality and robustness, we
report evidence of successful application in areas as diverse as genomics,
virology, languages, literature, music, handwritten digits, astronomy, and
combinations of objects from completely different domains, using statistical,
dictionary, and block sorting compressors. In genomics we presented new
evidence for major questions in Mammalian evolution, based on
whole-mitochondrial genomic analysis: the Eutherian orders and the Marsupionta
hypothesis against the Theria hypothesis.Comment: LaTeX, 27 pages, 20 figure
Minimax Structured Normal Means Inference
We provide a unified treatment of a broad class of noisy structure recovery
problems, known as structured normal means problems. In this setting, the goal
is to identify, from a finite collection of Gaussian distributions with
different means, the distribution that produced some observed data. Recent work
has studied several special cases including sparse vectors, biclusters, and
graph-based structures. We establish nearly matching upper and lower bounds on
the minimax probability of error for any structured normal means problem, and
we derive an optimality certificate for the maximum likelihood estimator, which
can be applied to many instantiations. We also consider an experimental design
setting, where we generalize our minimax bounds and derive an algorithm for
computing a design strategy with a certain optimality property. We show that
our results give tight minimax bounds for many structure recovery problems and
consider some consequences for interactive sampling
- …