Classical anomaly detection is principally concerned with point- based anomalies, anomalies that occur at a single data point. In this paper, we present a new mathematical model to express range- based anomalies, anomalies that occur over a range (or period) of time

Gottschlich, Justin E

Lee, Tae J

Metcalf, Eric

Tatbul, Nesime

Zdonik, Stan

English

arXiv

Kosmopolis

University of Pennsylvania 
ScholarlyCommons 
Machine Programming PRECISE (Penn Research in Embedded Computing and Integrated Engineering) 
2018 
Precision and Recall for Range-Based Anomaly Detection 
Tae J. Lee 
Justin E. Gottschlich 
Nesime Tatbul 
Eric Metcalf 
Stan Zdonik 
Follow this and additional works at: https://repository.upenn.edu/cps_machine_programming 
This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cps_machine_programming/8 
For more information, please contact repository@pobox.upenn.edu. 
Precision and Recall for Range-Based Anomaly Detection 
Abstract 
Classical anomaly detection is principally concerned with point- based anomalies, anomalies that occur 
at a single data point. In this paper, we present a new mathematical model to express range- based 
anomalies, anomalies that occur over a range (or period) of time. 
ar
X
iv
:1
80
1.
03
17
5v
3 
 [
cs
.A
I]
  1
1 
Fe
b 
20
18
Precision and Recall for Range-Based Anomaly Detection
Tae Jun Lee*
Microsoft
Justin Gottschlich, Nesime Tatbul
Intel Labs
Eric Metcalf, Stan Zdonik
Brown University
ABSTRACT
Classical anomaly detection is principally concerned with point-
based anomalies, anomalies that occur at a single data point. In
this paper, we present a new mathematical model to express range-
based anomalies, anomalies that occur over a range (or period) of
time.
1 INTRODUCTION
Anomaly detection (AD) seeks to identify atypical events. Anom-
alies tend to be domain or problem specific, and many occur over a
period of time. We refer to such events as range-based anomalies,
as they occur over a range (or period) of time1. Therefore, it is crit-
ical that the accuracy measures for anomalies, and the systems de-
tecting them, capture events that occur over a range of time. Unfor-
tunately, classical metrics for anomaly detection were designed to
handle only fixed-point anomalies [1]. An AD algorithm behaves
much like a pattern recognition and binary classification algorithm:
it recognizes certain patterns in its input and classifies them as ei-
ther normal or anomalous. For this class of algorithms, Recall and
Precision are widely used for evaluating the accuracy of the result.
They are formally defined as in Equations 1 and 2, where TP de-
notes true positives, FP denotes false positives, and FN denotes false
negatives.
Recal l = T P ÷ (T P + F N ) (1)
Precision = T P ÷ (T P + F P ) (2)
While useful for point-based anomalies, classical recall and pre-
cision suffer from their inability to capture, and bias, classifica-
tion correctness for domain-specific time-series anomalies. Because
of this, many time-series AD systems’ accuracy are being misrep-
resented, as point-based recall and precision are used to measure
their effectiveness [9]. Furthermore, the need to accurately identify
time-series anomalies is growing due to the explosion of streaming
and real-time systems [2, 4, 7, 8, 10]. To address this, we redefine
recall and precision to encompass range-based anomalies. Unlike
prior work [2, 6], our mathematical definitions are a superset of the
classical definitions, enabling our system to subsume point-based
anomalies. Moreover, our system is broadly generalizable, provid-
ing specialization functions to control a domain’s bias along a multi-
dimensional axis that is necessary to accommodate the needs of spe-
cific domains.
In this short paper, we present novel formal definitions of recall
and precision for range-based anomaly detection that both subsume
those formerly defined for point-based anomaly detection as well as
being customizable to a rich set of application domains. Empirical
data has been omitted to meet the venue’s compressed format.
*The work was done while a Brown student.
1Range-based anomalies are a specific type of collective anomalies [3]. Moreover,
range-based anomalies are similar, but not identical, to sequence anomalies [11].
SysML’18, February 2018, Stanford, CA, USA
2018.
Table 1: Notation
Notation Description
R set of real anomaly ranges
Ri the i
th real anomaly range
P set of predicted anomaly ranges
Pj the j
th predicted anomaly range
Nr number of real anomaly ranges
Np number of predicted anomaly ranges
α relative weight of existence reward
β relative weight of overlap reward
γ () overlap cardinality function
ω() overlap size function
δ () positional bias function
2 RANGE-BASED RECALL
Classical Recall rewards an AD system when anomalies are success-
fully identified (i.e., TP) and penalizes it when they are not (i.e.,
FN). It is computed by counting the number of anomalous points
successfully predicted and then dividing that number by the total
number of anomalous points. However, it is not sensitive to domains
where a single anomaly can be represented as a range of contiguous
points. In this section, we propose a new way to compute recall for
such range-based anomalies. Table 1 summarizes our notation.
Given a set of real anomaly ranges R = {R1, ..,RNr } and a set of
predicted anomaly ranges P = {P1, .., PNp }, our RecallT (R, P) for-
mulation iterates over the set of all real anomaly ranges (R), comput-
ing a recall score for each real anomaly range (Ri ∈ R) and adding
them up into a total recall score. This total score is then divided by
the total number of real anomalies (Nr ) to obtain an average recall
score for the whole time-series.
Recal lT (R, P ) =
∑Nr
i=1 Recal lT (Ri , P )
Nr
(3)
When computing the recall score RecallT (Ri , P) for a single real
anomaly range Ri , we take the following aspects into account:
• Existence: Identifying an anomaly (even by a single point in
Ri ) may be valuable in some application domains.
• Size: The larger the size of the correctly predicted portion of
Ri , the higher the recall score will likely be.
• Position: In some cases, not only size, but also the relative
position of the correctly predicted portion of Ri may be im-
portant to the application (e.g., early and late biases).
• Cardinality: Detecting Ri with a single predicted anomaly
range Pj ∈ P may be more valuable to an application than
doing so with multiple different ranges in P .
We capture these aspects as a sum of two reward terms weighted
by α and β , respectively, where 0 ≤ α , β ≤ 1 and α + β = 1. α
represents the relative importance of rewarding existence, whereas
β represents the relative importance of rewarding size, position, and
SysML’18, February 2018, Stanford, CA, USA T. J. Lee, J. Gottschlich, N. Tatbul, E. Metcalf, S. Zdonik
function ω (AnomalyRange, OverlapSet, δ )
MyValue← 0
MaxValue← 0
AnomalyLength← length(AnomalyRange)
for i← 1, AnomalyLength do
Bias← δ (i, AnomalyLength)
MaxValue← MaxValue + Bias
if AnomalyRange[i] in OverlapSet then
MyValue← MyValue + Bias
return MyValue/MaxValue
(a) Overlap Size
// Flat positional bias
function δ (i, AnomalyLength)
return 1
// Front-end positional bias
function δ (i, AnomalyLength)
return AnomalyLength - i + 1
// Tail-end positional bias
function δ (i, AnomalyLength)
return i
(b) Positional Bias
Figure 1: Example Functions for ω() and δ ()
cardinality, which all stem from the overlap between Ri and the set
of all predicted anomaly ranges (Pi ∈ P).
Recal lT (Ri , P ) = α × ExistenceReward (Ri , P )
+ β ×Over lapReward (Ri, P ) (4)
If anomaly range Ri is identified (i.e., |Ri ∩ Pj | ≥ 1 across all
Pj ∈ P), then an existence reward of 1 is earned.
ExistenceReward (Ri , P ) =
{
1 , if
∑Np
j=1 |Ri ∩ Pj | ≥ 1
0 , otherwise
(5)
Additionally, an overlap reward, dependent upon three application-
defined functions 0 ≤ γ () ≤ 1, 0 ≤ ω() ≤ 1, and δ () ≥ 1, can
be earned. These functions capture the cardinality (γ ), size (ω), and
position (δ ) of the overlap. The cardinality term serves as a scaling
factor for the rewards earned from size and position of the overlap.
Over lapReward (Ri, P ) = CardinalityF actor (Ri, P )
×
Np∑
j=1
ω(Ri , Ri ∩ Pj, δ ) (6)
The cardinality factor is largest (i.e., 1), when Ri overlaps with at
most one predicted anomaly range (i.e., it is identified by a single
prediction range). Otherwise, it receives a value 0 ≤ γ () ≤ 1 defined
by the application.
CardinalityF actor (Ri, P ) =


1 , if Ri overlaps with
at most one Pj ∈
P
γ (Ri , P ) , otherwise
(7)
The RecallT constants (α and β) and functions (γ (), ω(), and δ ()) are
tunable according to the needs of the application. Next, we illustrate
how they can be customized with examples.
The cardinality factor should generally be inversely proportional
toCard(Ri), i.e., the number of distinct prediction ranges that a real
anomaly range Ri overlaps. For example, γ (Ri , P) can simply be set
to 1/Card(Ri).
Figure 1a provides an example for theω() function for size, which
can be used with many different δ () functions for positional bias as
shown in Figure 1b. If all index positions are equally important,
then the flat bias function should be used. If earlier ones are more
important than later ones (e.g., early cancer detection [5], real-time
apps [2]), then the front-end bias function should be used. Finally,
if later index positions are more important (e.g., delayed response
in robotic defense), then the tail-end bias function should be used.
Our recall formula for range-based anomalies subsumes the clas-
sical one for point-based anomalies (i.e., RecallT ≡ Recall) when:
(i) all Ri ∈ R and Pj ∈ P are represented as single-point ranges
(e.g., range [1, 3] represented as [1, 1], [2, 2], [3, 3]), and
(ii) α = 0, β = 1,γ () = 1, ω() is as in Figure 1a, and δ () returns
flat positional bias as in Figure 1b.
3 RANGE-BASED PRECISION
Classical Precision is computed by counting the number of success-
ful prediction points (i.e., TP) in proportion to the total number of
prediction points (i.e., TP+FP). The key difference between Preci-
sion and Recall is that Precision penalizes FPs. In this section, we
extend classical precision to handle range-based anomalies. Our for-
mulation follows a similar structure as RecallT .
Given a set of real anomaly ranges R = {R1, ..,RNr } and a set of
predicted anomaly ranges P = {P1, .., PNp }, PrecisionT (R, P) iter-
ates over the set of predicted anomaly ranges (P), computing a pre-
cision score for each range (Pi ∈ P) and then sums them. This sum
is then divided by the total number of predicted anomalies (Np ),
averaging the score for the whole time-series.
PrecisionT (R, P ) =
∑Np
i=1 PrecisionT (R, Pi )
Np
(8)
When computing PrecisionT (R, Pi ) for a single predicted anomaly
range Pi , there is no need for an existence reward, because precision
by definition emphasizes prediction quality, and existence by itself
is too low a bar for judging the quality of a prediction. This removes
the need for α and β constants. Therefore:
PrecisionT (R, Pi ) = CardinalityF actor (Pi, R)
×
Nr∑
j=1
ω(Pi, Pi ∩ Rj , δ ) (9)
γ (),ω(), and δ () are customizable as before. Furthermore, PrecisionT
≡ Precision under the same settings as in Section 2 (except α and β
are not needed). Note that, while δ () provides a potential knob for
positional bias, we believe that in many domains a flat bias func-
tion will suffice for PrecisionT , as an FP is typically considered
uniformly bad wherever it appears in a prediction range.
4 CONCLUSION
In this paper, we note that traditional recall and precision were in-
vented for point-based analysis. In range-based anomaly detection,
anomalies are not necessarily single points, but are, in many cases,
ranges. In response, we offered new recall and precision definitions
that take ranges into account.
Acknowledgments. This research has been funded in part by Intel.
Precision and Recall for Range-Based Anomaly Detection SysML’18, February 2018, Stanford, CA, USA
REFERENCES
[1] Charu C. Aggarwal. 2013. Outlier Analysis. Springer.
[2] Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. Unsuper-
vised Real-time Anomaly Detection for Streaming Data. Neurocomputing 262
(2017), 134–147.
[3] Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly Detection:
A Survey. ACM Computing Surveys 41, 3 (2009), 15:1–15:58.
[4] Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. 2016. Robust
Random Cut Forest Based Anomaly Detection on Streams. In International Con-
ference on Machine Learning (ICML). 2712–2721.
[5] Konstantina Kourou, Themis P. Exarchos, Konstantinos P. Exarchos, Michalis V.
Karamouzis, and Dimitrios I. Fotiadis. 2015. Machine Learning Applications in
Cancer Prognosis and Prediction. Computational and Structural Biotechnology
Journal 13 (2015), 8–17.
[6] Alexander Lavin and Subutai Ahmad. 2015. Evaluating Real-Time Anomaly De-
tection Algorithms - The Numenta Anomaly Benchmark. In IEEE International
Conference on Machine Learning and Applications (ICMLA). 38–44.
[7] Tae Jun Lee, Justin Gottschlich, Nesime Tatbul, Eric Metcalf, and Stan Zdonik.
2018. Greenhouse: A Zero-Positive Machine Learning System for Time-Series
Anomaly Detection. https://arxiv.org/abs/1801.03168/. In SysML Conference.
[8] Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. 2015.
Long Short Term Memory Networks for Anomaly Detection in Time Series. In
European Symposium on Artificial Neural Networks, Computational Intelligence
and Machine Learning (ESANN). 89–94.
[9] Nidhi Singh and Craig Olinsky. 2017. Demystifying Numenta Anomaly Bench-
mark. In International Joint Conference on Neural Networks (IJCNN). 1570–
1577.
[10] Twitter. 2015. AnomalyDetection R Package.
https://github.com/twitter/AnomalyDetection/. (2015).
[11] Christina Warrender, Stephanie Forrest, and Barak Pearlmutter. 1999. Detecting
Intrusions using System Calls: Alternative Data Models. In IEEE Symposium on
Security and Privacy. 133–145.


Precision and Recall for Range-Based Anomaly Detection

ScholarlyCommons@Penn

University of Pennsylvania ScholarlyCommons Machine Programming PRECISE (Penn Research in Embedded Computing and Integrated Engineering) 2018 Precision and Recall for Range-Based Anomaly Detection Tae J. Lee Justin E. Gottschlich Nesime Tatbul Eric Metcalf Stan Zdonik Follow this and additional works at: https://repository.upenn.edu/cps_machine_programming This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cps_machine_programming/8 For more information, please contact repository@pobox.upenn.edu. Precision and Recall for Range-Based Anomaly Detection Abstract Classical anomaly detection is principally concerned with point- based anomalies, anomalies that occur at a single data point. In this paper, we present a new mathematical model to express range- based anomalies, anomalies that occur over a range (or period) of time. arXiv:1801.03175v3  [cs.AI]  11 Feb 2018Precision and Recall for Range-Based Anomaly DetectionTae Jun Lee*MicrosoftJustin Gottschlich, Nesime TatbulIntel LabsEric Metcalf, Stan ZdonikBrown UniversityABSTRACTClassical anomaly detection is principally concerned with point-based anomalies, anomalies that occur at a single data point. Inthis paper, we present a new mathematical model to express range-based anomalies, anomalies that occur over a range (or period) oftime.1 INTRODUCTIONAnomaly detection (AD) seeks to identify atypical events. Anom-alies tend to be domain or problem specific, and many occur over aperiod of time. We refer to such events as range-based anomalies,as they occur over a range (or period) of time1. Therefore, it is crit-ical that the accuracy measures for anomalies, and the systems de-tecting them, capture events that occur over a range of time. Unfor-tunately, classical metrics for anomaly detection were designed tohandle only fixed-point anomalies [1]. An AD algorithm behavesmuch like a pattern recognition and binary classification algorithm:it recognizes certain patterns in its input and classifies them as ei-ther normal or anomalous. For this class of algorithms, Recall andPrecision are widely used for evaluating the accuracy of the result.They are formally defined as in Equations 1 and 2, where TP de-notes true positives, FP denotes false positives, and FN denotes falsenegatives.Recal l = T P ÷ (T P + F N ) (1)Precision = T P ÷ (T P + F P ) (2)While useful for point-based anomalies, classical recall and pre-cision suffer from their inability to capture, and bias, classifica-tion correctness for domain-specific time-series anomalies. Becauseof this, many time-series AD systems’ accuracy are being misrep-resented, as point-based recall and precision are used to measuretheir effectiveness [9]. Furthermore, the need to accurately identifytime-series anomalies is growing due to the explosion of streamingand real-time systems [2, 4, 7, 8, 10]. To address this, we redefinerecall and precision to encompass range-based anomalies. Unlikeprior work [2, 6], our mathematical definitions are a superset of theclassical definitions, enabling our system to subsume point-basedanomalies. Moreover, our system is broadly generalizable, provid-ing specialization functions to control a domain’s bias along a multi-dimensional axis that is necessary to accommodate the needs of spe-cific domains.In this short paper, we present novel formal definitions of recalland precision for range-based anomaly detection that both subsumethose formerly defined for point-based anomaly detection as well asbeing customizable to a rich set of application domains. Empiricaldata has been omitted to meet the venue’s compressed format.*The work was done while a Brown student.1Range-based anomalies are a specific type of collective anomalies [3]. Moreover,range-based anomalies are similar, but not identical, to sequence anomalies [11].SysML’18, February 2018, Stanford, CA, USA2018.Table 1: NotationNotation DescriptionR set of real anomaly rangesRi the ith real anomaly rangeP set of predicted anomaly rangesPj the jth predicted anomaly rangeNr number of real anomaly rangesNp number of predicted anomaly rangesα relative weight of existence rewardβ relative weight of overlap rewardγ () overlap cardinality functionω() overlap size functionδ () positional bias function2 RANGE-BASED RECALLClassical Recall rewards an AD system when anomalies are success-fully identified (i.e., TP) and penalizes it when they are not (i.e.,FN). It is computed by counting the number of anomalous pointssuccessfully predicted and then dividing that number by the totalnumber of anomalous points. However, it is not sensitive to domainswhere a single anomaly can be represented as a range of contiguouspoints. In this section, we propose a new way to compute recall forsuch range-based anomalies. Table 1 summarizes our notation.Given a set of real anomaly ranges R = {R1, ..,RNr } and a set ofpredicted anomaly ranges P = {P1, .., PNp }, our RecallT (R, P) for-mulation iterates over the set of all real anomaly ranges (R), comput-ing a recall score for each real anomaly range (Ri ∈ R) and addingthem up into a total recall score. This total score is then divided bythe total number of real anomalies (Nr ) to obtain an average recallscore for the whole time-series.Recal lT (R, P ) =∑Nri=1 Recal lT (Ri , P )Nr(3)When computing the recall score RecallT (Ri , P) for a single realanomaly range Ri , we take the following aspects into account:• Existence: Identifying an anomaly (even by a single point inRi ) may be valuable in some application domains.• Size: The larger the size of the correctly predicted portion ofRi , the higher the recall score will likely be.• Position: In some cases, not only size, but also the relativeposition of the correctly predicted portion of Ri may be im-portant to the application (e.g., early and late biases).• Cardinality: Detecting Ri with a single predicted anomalyrange Pj ∈ P may be more valuable to an application thandoing so with multiple different ranges in P .We capture these aspects as a sum of two reward terms weightedby α and β , respectively, where 0 ≤ α , β ≤ 1 and α + β = 1. αrepresents the relative importance of rewarding existence, whereasβ represents the relative importance of rewarding size, position, andSysML’18, February 2018, Stanford, CA, USA T. J. Lee, J. Gottschlich, N. Tatbul, E. Metcalf, S. Zdonikfunction ω (AnomalyRange, OverlapSet, δ )MyValue← 0MaxValue← 0AnomalyLength← length(AnomalyRange)for i← 1, AnomalyLength doBias← δ (i, AnomalyLength)MaxValue← MaxValue + Biasif AnomalyRange[i] in OverlapSet thenMyValue← MyValue + Biasreturn MyValue/MaxValue(a) Overlap Size// Flat positional biasfunction δ (i, AnomalyLength)return 1// Front-end positional biasfunction δ (i, AnomalyLength)return AnomalyLength - i + 1// Tail-end positional biasfunction δ (i, AnomalyLength)return i(b) Positional BiasFigure 1: Example Functions for ω() and δ ()cardinality, which all stem from the overlap between Ri and the setof all predicted anomaly ranges (Pi ∈ P).Recal lT (Ri , P ) = α × ExistenceReward (Ri , P )+ β ×Over lapReward (Ri, P ) (4)If anomaly range Ri is identified (i.e., |Ri ∩ Pj | ≥ 1 across allPj ∈ P), then an existence reward of 1 is earned.ExistenceReward (Ri , P ) ={1 , if∑Npj=1 |Ri ∩ Pj | ≥ 10 , otherwise(5)Additionally, an overlap reward, dependent upon three application-defined functions 0 ≤ γ () ≤ 1, 0 ≤ ω() ≤ 1, and δ () ≥ 1, canbe earned. These functions capture the cardinality (γ ), size (ω), andposition (δ ) of the overlap. The cardinality term serves as a scalingfactor for the rewards earned from size and position of the overlap.Over lapReward (Ri, P ) = CardinalityF actor (Ri, P )×Np∑j=1ω(Ri , Ri ∩ Pj, δ ) (6)The cardinality factor is largest (i.e., 1), when Ri overlaps with atmost one predicted anomaly range (i.e., it is identified by a singleprediction range). Otherwise, it receives a value 0 ≤ γ () ≤ 1 definedby the application.CardinalityF actor (Ri, P ) =1 , if Ri overlaps withat most one Pj ∈Pγ (Ri , P ) , otherwise(7)The RecallT constants (α and β) and functions (γ (), ω(), and δ ()) aretunable according to the needs of the application. Next, we illustratehow they can be customized with examples.The cardinality factor should generally be inversely proportionaltoCard(Ri), i.e., the number of distinct prediction ranges that a realanomaly range Ri overlaps. For example, γ (Ri , P) can simply be setto 1/Card(Ri).Figure 1a provides an example for theω() function for size, whichcan be used with many different δ () functions for positional bias asshown in Figure 1b. If all index positions are equally important,then the flat bias function should be used. If earlier ones are moreimportant than later ones (e.g., early cancer detection [5], real-timeapps [2]), then the front-end bias function should be used. Finally,if later index positions are more important (e.g., delayed responsein robotic defense), then the tail-end bias function should be used.Our recall formula for range-based anomalies subsumes the clas-sical one for point-based anomalies (i.e., RecallT ≡ Recall) when:(i) all Ri ∈ R and Pj ∈ P are represented as single-point ranges(e.g., range [1, 3] represented as [1, 1], [2, 2], [3, 3]), and(ii) α = 0, β = 1,γ () = 1, ω() is as in Figure 1a, and δ () returnsflat positional bias as in Figure 1b.3 RANGE-BASED PRECISIONClassical Precision is computed by counting the number of success-ful prediction points (i.e., TP) in proportion to the total number ofprediction points (i.e., TP+FP). The key difference between Preci-sion and Recall is that Precision penalizes FPs. In this section, weextend classical precision to handle range-based anomalies. Our for-mulation follows a similar structure as RecallT .Given a set of real anomaly ranges R = {R1, ..,RNr } and a set ofpredicted anomaly ranges P = {P1, .., PNp }, PrecisionT (R, P) iter-ates over the set of predicted anomaly ranges (P), computing a pre-cision score for each range (Pi ∈ P) and then sums them. This sumis then divided by the total number of predicted anomalies (Np ),averaging the score for the whole time-series.PrecisionT (R, P ) =∑Npi=1 PrecisionT (R, Pi )Np(8)When computing PrecisionT (R, Pi ) for a single predicted anomalyrange Pi , there is no need for an existence reward, because precisionby definition emphasizes prediction quality, and existence by itselfis too low a bar for judging the quality of a prediction. This removesthe need for α and β constants. Therefore:PrecisionT (R, Pi ) = CardinalityF actor (Pi, R)×Nr∑j=1ω(Pi, Pi ∩ Rj , δ ) (9)γ (),ω(), and δ () are customizable as before. Furthermore, PrecisionT≡ Precision under the same settings as in Section 2 (except α and βare not needed). Note that, while δ () provides a potential knob forpositional bias, we believe that in many domains a flat bias func-tion will suffice for PrecisionT , as an FP is typically considereduniformly bad wherever it appears in a prediction range.4 CONCLUSIONIn this paper, we note that traditional recall and precision were in-vented for point-based analysis. In range-based anomaly detection,anomalies are not necessarily single points, but are, in many cases,ranges. In response, we offered new recall and precision definitionsthat take ranges into account.Acknowledgments. This research has been funded in part by Intel.Precision and Recall for Range-Based Anomaly Detection SysML’18, February 2018, Stanford, CA, USAREFERENCES[1] Charu C. Aggarwal. 2013. Outlier Analysis. Springer.[2] Subutai Ahmad, Alexander Lavin, Scott Purdy, and Zuha Agha. 2017. Unsuper-vised Real-time Anomaly Detection for Streaming Data. Neurocomputing 262(2017), 134–147.[3] Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly Detection:A Survey. ACM Computing Surveys 41, 3 (2009), 15:1–15:58.[4] Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. 2016. RobustRandom Cut Forest Based Anomaly Detection on Streams. In International Con-ference on Machine Learning (ICML). 2712–2721.[5] Konstantina Kourou, Themis P. Exarchos, Konstantinos P. Exarchos, Michalis V.Karamouzis, and Dimitrios I. Fotiadis. 2015. Machine Learning Applications inCancer Prognosis and Prediction. Computational and Structural BiotechnologyJournal 13 (2015), 8–17.[6] Alexander Lavin and Subutai Ahmad. 2015. Evaluating Real-Time Anomaly De-tection Algorithms - The Numenta Anomaly Benchmark. In IEEE InternationalConference on Machine Learning and Applications (ICMLA). 38–44.[7] Tae Jun Lee, Justin Gottschlich, Nesime Tatbul, Eric Metcalf, and Stan Zdonik.2018. Greenhouse: A Zero-Positive Machine Learning System for Time-SeriesAnomaly Detection. https://arxiv.org/abs/1801.03168/. In SysML Conference.[8] Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. 2015.Long Short Term Memory Networks for Anomaly Detection in Time Series. InEuropean Symposium on Artificial Neural Networks, Computational Intelligenceand Machine Learning (ESANN). 89–94.[9] Nidhi Singh and Craig Olinsky. 2017. Demystifying Numenta Anomaly Bench-mark. In International Joint Conference on Neural Networks (IJCNN). 1570–1577.[10] Twitter. 2015. AnomalyDetection R Package.https://github.com/twitter/AnomalyDetection/. (2015).[11] Christina Warrender, Stephanie Forrest, and Barak Pearlmutter. 1999. DetectingIntrusions using System Calls: Alternative Data Models. In IEEE Symposium onSecurity and Privacy. 133–145.

Precision and Recall for Range-Based Anomaly Detection

Abstract

Similar works

Full text

Available Versions

Kosmopolis

ScholarlyCommons@Penn