DriftDetectionMetrics

class capymoa.drift.eval_detector.DriftDetectionMetrics

Bases: object

Metrics for evaluating drift detection performance.

__init__(

fp: int,

tp: int,

fn: int,

precision: float,

recall: float,

episode_recall: float,

f1: float,

mdt: float,

ndt: float,

far: float,

ar: float,

n_episodes: int,

n_alarms: int,

) → None

ar: float

Alarm rate per EvaluateDriftDetector.rate_period instances

episode_recall: float

Recall score for drift episodes (in this case, a correct prediction of a drift episode is counted as 1 true positive).

f1: float

F1 score (harmonic mean of precision and recall).

far: float

False alarm rate per EvaluateDriftDetector.rate_period instances

fn: int

False negatives (missed drifts).

fp: int

False positives (incorrect detections).

mdt: float

Mean time to detect successful detections.

n_alarms: int

Total number of alarms raised

n_episodes: int

Total number of drift episodes

ndt: float

mdt divided by EvaluateDriftDetector.max_delay, so that detection delay is expressed as a fraction of the delay that was deemed acceptable.

For abrupt drifts this runs from 0 (detected the instant the drift began) to 1 (detected just as the drift became obvious anyway). It can fall outside that range in two cases: with gradual drifts, because the delay is measured from the drift start while detections stay valid until end + max_delay; and with EvaluateDriftDetector.max_early_detection above zero, which admits detections before the drift and so negative delays.

Unlike mdt, which is a count of instances, ndt is comparable across streams that were evaluated with different values of max_delay.

precision: float

Precision score (tp / (tp + fp)).

recall: float

Recall score (tp / (tp + fn)).

tp: int

True positives (correct detections).