EvaluateDetector

class capymoa.drift.eval_detector.EvaluateDetector

Bases: object

Evaluate Drift Detector

Evaluate drift detection Methods based on known drift locations

References: - Cerqueira, Vitor, Heitor Murilo Gomes, and Albert Bifet. “Unsupervised concept drift detection using a student–teacher approach.” Discovery Science: 23rd International Conference, DS 2020, Thessaloniki, Greece, October 19–21, 2020 - Bifet, A.: Classifier concept drift detection and the illusion of progress. In: International Conference on Artificial Intelligence and Soft Computing. pp. 715–725. Springer (2017)

Example usages:

>>> import numpy as np
>>> from capymoa.drift.detectors import ADWIN
>>> from capymoa.drift.eval_detector import EvaluateDetector
>>>
>>> detector = ADWIN(delta=0.001)
>>>
>>> data_stream = np.random.randint(2, size=2000)
>>> for i in range(999, 2000):
...     data_stream[i] = np.random.randint(4, high=8)
>>>
>>> for i in range(2000):
...     detector.add_element(data_stream[i])
>>>
>>> trues = np.array([1000])
>>> preds = detector.detection_index
>>>
>>> eval = EvaluateDetector(max_delay=200)
>>> print(eval.calc_performance(preds, trues))
mean_time_to_detect           24.0
missed_detection_ratio         0.0
mean_time_btw_false_alarms     NaN
no_alarms_per_episode          0.0
dtype: float64

__init__(max_delay: int)

Parameters:

max_delay – Maximum number of instances to wait for a detection [after which the drift becomes obvious and the detector is considered to have missed the change]

calc_performance(

preds: List[int] | ndarray[int],

trues: List[int] | ndarray[int],

) → Series

Parameters:

• preds – (array): detection location (index) values

• trues – (array): actual location (index) of drift. For drifts based on an interval (e.g gradual drifts), the current approach is to define it using the starting location and the max_delay parameter

Returns:

pd.Series with a performance summary

update_metrics()

static calc_false_alarms(

preds: List[int] | ndarray[int],

true: int,

) → Tuple[float, int]

Parameters:

• preds – detection points

• true – actual drift point

Returns:

tuple, (float: mean time between false alarms, int: no. false alarms)

static calc_detection_delay(

preds: List[int] | ndarray[int],

true: int,

) → Tuple[int, int]

Parameters:

• preds – detection points

• true – actual drift point

Returns:

detection delay (number of instances), and a flag for whether drift is detected