At 2006 AAAI Fall Symposium on Capturing and Using Patterns for Evidence Detection Scoring Hypotheses from Threat Detection Technologies

We describe efficient methods to score structured hypotheses from threat detection technologies that fuse evidence from massive data streams to detect threat phenomena. The strongly object-oriented threat case representation summarizes only key object attributes. Pairing of hypothesized and reference cases exploits a directed acyclic case type graph to minimize case comparisons. Because case pairing is expensive, we expediently avoid it where possible. One global pairing operation suffices to develop: • Count-based metrics (precision, recall, F-value) that generalize the traditional versions to object-oriented versions that accommodate inexact matching over structured hypotheses with weighted attributes; • Area under the object-oriented precision-recall curve; • Cost-based metrics that address timely incremental evidence processing; • Statistical significance of computed scores. Many software parameters support customized experimentation