# Evaluating regressors in CapyMOA

This notebook further explores **high-level evaluation functions** as applied to **regressors**.

* **High-level evaluation functions**
  * We use `prequential_evaluation()` and `prequential_evaluation_multiple_learners()`, the same functions introduced for classification (see notebooks/classifier/evaluation.py), and show how they apply to regression with only minor differences.
  * We also show how to plot **predictions vs. ground truth** over time, which is particularly useful for regression tasks.

---

*More information about CapyMOA can be found at* https://www.capymoa.org.

**last update on 28/11/2025**

## Regression

* We introduce a simple example using regression just to show how similar it is to assess regressors using the **high-level evaluation functions**.
* In the example below, we just use `prequential_evaluation()` but it would work with `cumulative_evaluation()` and `windowed_evaluation()` as well.
* One difference between classification and regression evaluation in CapyMOA is that the evaluators are different. Instead of `ClassificationEvaluator` and `ClassificationWindowedEvaluator` functions use `RegressionEvaluator` and `RegressionWindowedEvaluator`.

```ipython3
from capymoa.datasets import Fried
from capymoa.evaluation import prequential_evaluation
from capymoa.evaluation.visualization import plot_windowed_results
from capymoa.regressor import AdaptiveRandomForestRegressor, KNNRegressor

stream = Fried()
kNN_learner = KNNRegressor(schema=stream.get_schema(), k=5)
ARF_learner = AdaptiveRandomForestRegressor(
    schema=stream.get_schema(), ensemble_size=10
)

kNN_results = prequential_evaluation(
    stream=stream, learner=kNN_learner, window_size=5000
)
ARF_results = prequential_evaluation(
    stream=stream, learner=ARF_learner, window_size=5000
)

print(
    f"{kNN_results['learner']} [cumulative] RMSE = {kNN_results['cumulative'].rmse()} and \
    {ARF_results['learner']}  [cumulative] RMSE = {ARF_results['cumulative'].rmse()}"
)

plot_windowed_results(kNN_results, ARF_results, metric="rmse")
```

```myst-ansi
kNNRegressor [cumulative] RMSE = 2.7229994765160916 and     AdaptiveRandomForestRegressor  [cumulative] RMSE = 3.312901313632365
```

### Evaluating a single stream using multiple learners

* `prequential_evaluation_multiple_learners` also works for multiple regressors; the example below shows how it can be used.

```ipython3
from capymoa.evaluation import prequential_evaluation_multiple_learners

# Define the learners + an alias (dictionary key)
learners = {
    "kNNReg_k5": KNNRegressor(schema=stream.get_schema(), k=5),
    "kNNReg_k2": KNNRegressor(schema=stream.get_schema(), k=2),
    "kNNReg_k5_median": KNNRegressor(schema=stream.get_schema(), CLI="-k 5 -m"),
    "ARFReg_s5": AdaptiveRandomForestRegressor(
        schema=stream.get_schema(), ensemble_size=5
    ),
}

results = prequential_evaluation_multiple_learners(stream, learners)

print("Cumulative results for each learner:")
for learner_id in learners:
    if learner_id in results:
        cumulative = results[learner_id]["cumulative"]
        print(
            f"{learner_id}, RMSE: {cumulative.rmse():.2f}, adjusted R2: {cumulative.adjusted_r2():.2f}"
        )

# Tip: invoking metrics_header() from an evaluator will show us all the metrics available,
# e.g. results['kNNReg_k5']['cumulative'].metrics_header()
plot_windowed_results(
    results["kNNReg_k5"],
    results["kNNReg_k2"],
    results["kNNReg_k5_median"],
    results["ARFReg_s5"],
    metric="rmse",
)

plot_windowed_results(
    results["kNNReg_k5"],
    results["kNNReg_k2"],
    results["kNNReg_k5_median"],
    results["ARFReg_s5"],
    metric="adjusted_r2",
)
```

```myst-ansi
Cumulative results for each learner:
kNNReg_k5, RMSE: 2.72, adjusted R2: 0.70
kNNReg_k2, RMSE: 3.08, adjusted R2: 0.62
kNNReg_k5_median, RMSE: 2.94, adjusted R2: 0.65
ARFReg_s5, RMSE: 3.77, adjusted R2: 0.43
```

### Plotting predictions vs. ground truth over time

* In regression it is sometimes desirable to plot **predictions vs. ground truth** to observe what is happening with the stream. If we create a custom loop and use the evaluators directly, it is trivial to store the ground truth and predictions, and then proceed to plot them. However, to make people’s life easier the `plot_predictions_vs_ground_truth` function can be used.
* For massive streams with millions of instances, it can be unbearable to plot all at once, thus we can specify a `plot_interval` (that we want to investigate) to `plot_predictions_vs_ground_truth`. By default, the plot function will attempt to plot everything, i.e. with `plot_interval=None`, which is seldom a good idea.

```ipython3
from capymoa.datasets import Fried
from capymoa.evaluation import prequential_evaluation
from capymoa.evaluation.visualization import plot_predictions_vs_ground_truth
from capymoa.regressor import AdaptiveRandomForestRegressor, KNNRegressor

stream = Fried()
kNN_learner = KNNRegressor(schema=stream.get_schema(), k=5)
ARF_learner = AdaptiveRandomForestRegressor(
    schema=stream.get_schema(), ensemble_size=10
)

# When we specify store_predictions and store_y, the results will also include all the predictions and all the ground truth y.
# It is useful for debugging and outputting the predictions elsewhere.
kNN_results = prequential_evaluation(
    stream=stream,
    learner=kNN_learner,
    window_size=5000,
    store_predictions=True,
    store_y=True,
)
# We don't need to store the ground-truth for every experiment, since it is always the same for the same stream.
ARF_results = prequential_evaluation(
    stream=stream, learner=ARF_learner, window_size=5000, store_predictions=True
)


# Plot only 200 predictions (see plot_interval).
plot_predictions_vs_ground_truth(
    kNN_results,
    ARF_results,
    ground_truth=kNN_results["ground_truth_y"],
    plot_interval=(0, 200),
)
```