The Mercury System: Exploiting Truly Fast Hardware in Data Mining

In many data mining applications, the size of the database is not only extremely large, it is also growing rapidly. Even for relatively simple searches, the time required to move the data off magnetic media, cross the system bus into main memory, copy into processor cache, and then execute code to perform a search is prohibitive. We are building a system in which a significant portion of the data mining task (i.e., the portion that examines the bulk of the raw data) is implemented in fast hardware, close to the magnetic media on which it is stored. Furthermore, this hardware can be replicated allowing mining tasks to be performed in parallel, thus providing further speedup for the overall mining application. In this paper, we describe a general framework under which this can be accomplished and provide initial performance results for a set of applications

Information Theoretic Limits on Non-cooperative Airborne Target Recognition by Means of Radar Sensors

The main objective of this research is to demonstrate that information theory, and specifically the concept of mutual information (MI) can be used to predict the maximum target recognition performance for a given radar concept in combination with a given set of targets of interest. This approach also allows for the direct comparison of disparate approaches to designing a radar concept which is capable of target recognition without resorting to choosing specific feature extraction and classification algorithms. The main application area of the study is the recognition of fighter type aircraft using surface based radar systems, although the results are also applicable to airborne radars. Information theoretic concepts are developed mathematically for the analysis of the radar target recognition problem. The various forms of MI required for this application are derived in detail and are tested rigorously against results from digital communication theory. The results are also compared to Shannon’s channel capacity bound, which is the fundamental limit on the amount of information which can be transmitted over a channel. Several sets of simulation based experiments were conducted to demonstrate the insights achievable by applying MI concepts to quantitatively predict the maximum achievable performance of disparate approaches to the radar target recognition problem. Asymptotic computational electromagnetic code was applied to calculate the target’s response to the radar signal for freely available geometrical models of fighter aircraft. The calculated target responses were then used to quantify the amount of information which is transmitted back to the radar about the target as a function of signal to noise ratio (SNR). The information content of the F-14, F-15 and F-16 were evaluated for a 480 MHz bandwidth waveform at 10 GHz as a baseline. Several ultra-wideband (UWB) waveforms, spanning 2-10 GHz, 10- 18 GHz and 2-18 GHz, but which were highly range ambiguous, were evaluated and showed SNR gains of 0.5-2 dB relative to the baseline. The effect of sensing the full polarimetric response of an F-18 and F-35 was evaluated and SNR gains of 5-7 dB over a single linear polarisation were measured. A Boeing 707 scale model (1:25) was measured in the University of Pretoria’s compact range spanning 2-18 GHz and gains of 2 dB were observed between single and dual linear polarisations. This required numerical integration in 8004 dimensions, demonstrating the stability of the MI estimation algorithm in high dimensional signal spaces. The information gained by including the difference channel signal of an X-band monopulse radar for the F-14 data set was approximately 3 dB at 50 km and increased to 4.5 dB at 2 km due to the increased target extent relative to the antenna pattern. This experiment necessitated the use of target profiles which were matched to the range of the target to achieve maximum information transfer. Experiments were conducted to evaluate the loss in information due to envelope processing. For the baseline data set, SNR losses in the region of 7 dB were measured. Linear pre-processing using the fast Fourier transform (FFT) and principal component analysis (PCA), before envelope processing, were compared and the PCA algorithm outperformed the FFT by approximately 1 dB at high MI values. Finally, the expression for multi-target MI was applied in conjunction with Fano’s inequality to predict the probability of incorrectly classifying a target. Probability of error is a critical parameter for a radar user. For the baseline data set, at P(error) = 0.001, maximum losses in the region of 0.6 to 0.9 dB were measured. This result shows that these targets are easily separable in the signal space. This study was only the proverbial “tip of the iceberg” and future research could extend the results and applications of the techniques developed. The types of targets and configurations of the individual targets could be increased and analysed. The analysis should also be extended to describe effects internal to the radar such as phase noise, spurious signals and analogue to digital converters and external effects such as clutter and multipath. The techniques could also be applied to quantify the gains in target recognition performance achievable for multistatic radar, multiple input multiple output (MIMO) radar and more exotic concepts, such as the fusion of data from multiple monostatic microwave radars with multi-receiver multi-band passive bistatic radar (PBR) data