# `evaluation`

Evaluate online continual learning in classification tasks.

## Modules

| [`events`](capymoa.ocl.evaluation.events.md#module-capymoa.ocl.evaluation.events)   | Event definitions for OCL evaluation loops.   |
|------------------------------------------------------------------------------------------------|-----------------------------------------------|

## Classes

| [`OCLMetrics`](capymoa.ocl.evaluation.OCLMetrics.md#capymoa.ocl.evaluation.OCLMetrics)   | A collection of metrics evaluating an online continual learner.   |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------|

## Functions

| [`ocl_train_eval_loop`](#capymoa.ocl.evaluation.ocl_train_eval_loop)   | Run the OCL training loop with periodic continual evaluation.   |
|------------------------------------------------------------------------|-----------------------------------------------------------------|

### capymoa.ocl.evaluation.ocl_train_eval_loop(learner: [Classifier](capymoa.base.Classifier.md#capymoa.base.Classifier), train_streams: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[[DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader)[[tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor), [Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor)]]], test_streams: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[[DataLoader](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader)[[tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor), [Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor)]]], continual_evaluations: [int](https://docs.python.org/3/builtins/functions.html#int) = 1, progress_bar: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False, eval_window_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 1000, epochs: [int](https://docs.python.org/3/builtins/functions.html#int) = 1, dispatcher: [Dispatcher](capymoa.ocl.events.Dispatcher.md#capymoa.ocl.events.Dispatcher) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [OCLMetrics](capymoa.ocl.evaluation.OCLMetrics.md#capymoa.ocl.evaluation.OCLMetrics)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/ocl/evaluation/_loop.py#L93)

Run the OCL training loop with periodic continual evaluation.

* **Parameters:**
  * **learner** – The classifier to train and evaluate.
  * **train_streams** – Sequence of task-wise training data loaders.
  * **test_streams** – Sequence of task-wise test data loaders. Must have the same
    number of tasks as train_streams.
  * **continual_evaluations** – Number of evaluation passes performed during each
    training task, defaults to 1.
  * **progress_bar** – Whether to enable a progress-bar, defaults to False.
  * **eval_window_size** – Window size used by the default metrics handler to compute
    rolling metrics, defaults to 1000.
  * **epochs** – Number of epochs to train each task stream, defaults to 1.
  * **dispatcher** – Optional event dispatcher. If None, a new dispatcher is created.
* **Returns:**
  Aggregated OCL metrics collected by the default metrics handler.
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If train/test task counts differ, `continual_evaluations
    < 1`, or a train stream has fewer batches than requested evaluations.
  * [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError) – If learner is not a classifier.