EvaluateDetector#

class capymoa.drift.eval_detector.EvaluateDetector[source]#

Bases: object

Evaluate Drift Detector

Evaluate drift detection Methods based on known drift locations

References: - Cerqueira, Vitor, Heitor Murilo Gomes, and Albert Bifet. “Unsupervised concept drift detection using a student–teacher approach.” Discovery Science: 23rd International Conference, DS 2020, Thessaloniki, Greece, October 19–21, 2020 - Bifet, A.: Classifier concept drift detection and the illusion of progress. In: International Conference on Artificial Intelligence and Soft Computing. pp. 715–725. Springer (2017)

Example usages:

>>> import numpy as np
>>> from capymoa.drift.detectors import ADWIN
>>> from capymoa.drift.eval_detector import EvaluateDetector
>>>
>>> detector = ADWIN(delta=0.001)
>>>
>>> data_stream = np.random.randint(2, size=2000)
>>> for i in range(999, 2000):
...     data_stream[i] = np.random.randint(4, high=8)
>>>
>>> for i in range(2000):
...     detector.add_element(data_stream[i])
>>>
>>> trues = np.array([1000])
>>> preds = detector.detection_index
>>>
>>> eval = EvaluateDetector(max_delay=200)
>>> print(eval.calc_performance(preds, trues))
mean_time_to_detect           24.0
missed_detection_ratio         0.0
mean_time_btw_false_alarms     NaN
no_alarms_per_episode          0.0
dtype: float64
__init__(max_delay: int)[source]#
Parameters:

max_delay – Maximum number of instances to wait for a detection [after which the drift becomes obvious and the detector is considered to have missed the change]

calc_performance(
preds: List[int] | ndarray[int],
trues: List[int] | ndarray[int],
) Series[source]#
Parameters:
  • preds – (array): detection location (index) values

  • trues – (array): actual location (index) of drift. For drifts based on an interval (e.g gradual drifts), the current approach is to define it using the starting location and the max_delay parameter

Returns:

pd.Series with a performance summary

update_metrics()[source]#
static calc_false_alarms(
preds: List[int] | ndarray[int],
true: int,
) Tuple[float, int][source]#
Parameters:
  • preds – detection points

  • true – actual drift point

Returns:

tuple, (float: mean time between false alarms, int: no. false alarms)

static calc_detection_delay(
preds: List[int] | ndarray[int],
true: int,
) Tuple[int, int][source]#
Parameters:
  • preds – detection points

  • true – actual drift point

Returns:

detection delay (number of instances), and a flag for whether drift is detected