Drift Detection Using Uncertainty Distribution Divergence

Concept drift is believed to be prevalent inmost data gathered from naturally occurring processes andthus warrants research by the machine learning community.There are a myriad of approaches to concept drift handlingwhich have been shown to handle concept drift with varyingdegrees of success. However, most approaches make the keyassumption that the labelled data will be available at nolabelling cost shortly after classification, an assumption whichis often violated. The high labelling cost in many domainsprovides a strong motivation to reduce the number of labelledinstances required to handle concept drift. Explicit detectionapproaches that do not require labelled instances to detectconcept drift show great promise for achieving this. Ourapproach Confidence Distribution Batch Detection (CDBD)provides a signal correlated to changes in concept without usinglabelled data. We also show how this signal combined with atrigger and a rebuild policy can maintain classifier accuracywhile using a limited amount of labelled data

Drift Detection using Uncertainty Distribution Divergence

Data generated from naturally occurring processes tends to be non-stationary. For example, seasonal and gradual changes in climate data and sudden changes in financial data. In machine learning the degradation in classifier performance due to such changes in the data is known as concept drift and there are many approaches to detecting and handling it. Most approaches to detecting concept drift, however, make the assumption that true classes for test examples will be available at no cost shortly after classification and base the detection of concept drift on measures relying on these labels. The high labelling cost in many domains provides a strong motivation to reduce the number of labelled instances required to detect and handle concept drift. Triggered detection approaches that do not require labelled instances to detect concept drift show great promise for achieving this. In this paper we present Confidence Distribution Batch Detection (CDBD), an approach that provides a signal correlated to changes in concept without using labelled data. This signal combined with a trigger and a rebuild policy can maintain classifier accuracy which, in most cases, matches the accuracy achieved using classification error based detection techniques but using only a limited amount of labelled data

Drift Detection Using Uncertainty Distribution Divergence

Concept drift is believed to be prevalent inmost data gathered from naturally occurring processes andthus warrants research by the machine learning community.There are a myriad of approaches to concept drift handlingwhich have been shown to handle concept drift with varyingdegrees of success. However, most approaches make the keyassumption that the labelled data will be available at nolabelling cost shortly after classification, an assumption whichis often violated. The high labelling cost in many domainsprovides a strong motivation to reduce the number of labelledinstances required to handle concept drift. Explicit detectionapproaches that do not require labelled instances to detectconcept drift show great promise for achieving this. Ourapproach Confidence Distribution Batch Detection (CDBD)provides a signal correlated to changes in concept without usinglabelled data. We also show how this signal combined with atrigger and a rebuild policy can maintain classifier accuracywhile using a limited amount of labelled data

Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels

One important assumption underlying common classification models is the stationarity of the data. However, in real-world streaming applications, the data concept indicated by the joint distribution of feature and label is not stationary but drifting over time. Concept drift detection aims to detect such drifts and adapt the model so as to mitigate any deterioration in the model's predictive performance. Unfortunately, most existing concept drift detection methods rely on a strong and over-optimistic condition that the true labels are available immediately for all already classified instances. In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary. Two methods, namely Hierarchical Hypothesis Testing with Classification Uncertainty (HHT-CU) and Hierarchical Hypothesis Testing with Attribute-wise "Goodness-of-fit" (HHT-AG), are proposed respectively under the novel framework. In experiments with benchmark datasets, our methods demonstrate overwhelming advantages over state-of-the-art unsupervised drift detectors. More importantly, our methods even outperform DDM (the widely used supervised drift detector) when we use significantly fewer labels.Comment: Published as a conference paper at IJCAI 201

ILC Beam Energy Measurement by means of Laser Compton Backscattering

A novel, non-invasive method of measuring the beam energy at the International Linear Collider is proposed. Laser light collides head-on with beam particles and either the energy of the Compton scattered electrons near the kinematic end-point is measured or the positions of the Compton backscattered $\gamma$-rays, the edge electrons and the unscattered beam particles are recorded. A compact layout for the Compton spectrometer is suggested. It consists of a bending magnet and position sensitive detectors operating in a large radiation environment. Several options for high spatial resolution detectors are discussed. Simulation studies support the use of an infrared or green laser and quartz fiber detectors to monitor the backscattered photons and edge electrons. Employing a cavity monitor, the beam particle position downstream of the magnet can be recorded with submicrometer precision. Such a scheme provides a feasible and promising method to access the incident beam energy with precisions of $10^{-4}$ or better on a bunch-to-bunch basis while the electron and positron beams are in collision.Comment: 47 pages, 26 figures, version as accepted by Nucl. Instr. Meth. A after improvement