# `ShrubsRegressor`

### *class* capymoa.regressor.ShrubsRegressor[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/regressor/_shrubs_regressor.py#L15)

Bases: `_ShrubEnsembles`, [`Regressor`](capymoa.base.Regressor.md#capymoa.base.Regressor)

ShrubsRegressor

This class implements the ShrubEnsembles algorithm for regression, which is
an ensemble classifier that continuously adds regression trees to the ensemble by training them over a sliding window while pruning unnecessary trees away using proximal (stochastic) gradient descent, hence allowing for adaptation to concept drift. For regression, the MSE loss is minimized.

Reference:

[Shrub Ensembles for Online Classification
Sebastian Buschjäger, Sibylle Hess, and Katharina Morik
In Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), Jan 2022.](https://aaai.org/papers/06123-shrub-ensembles-for-online-classification/)

Example usage:

```pycon
>>> from capymoa.datasets import Fried
>>> from capymoa.regressor import ShrubsRegressor
>>> from capymoa.evaluation import prequential_evaluation
>>> stream = Fried()
>>> schema = stream.get_schema()
>>> learner = ShrubsRegressor(schema)
>>> results = prequential_evaluation(stream, learner, max_instances=1000)
>>> results["cumulative"].rmse()
5.21...
```

#### \_\_init_\_(schema: [Schema](capymoa.stream.Schema.md#capymoa.stream.Schema), step_size: [float](https://docs.python.org/3/builtins/functions.html#float) | [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['adaptive'] = 'adaptive', ensemble_regularizer: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['hard-L0', 'L0', 'L1', 'none'] = 'hard-L0', l_ensemble_reg: [float](https://docs.python.org/3/builtins/functions.html#float) = 32, l_l2_reg: [float](https://docs.python.org/3/builtins/functions.html#float) = 0, l_tree_reg: [float](https://docs.python.org/3/builtins/functions.html#float) = 0, normalize_weights: [bool](https://docs.python.org/3/builtins/functions.html#bool) = True, burnin_steps: [int](https://docs.python.org/3/builtins/functions.html#int) = 5, update_leaves: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False, batch_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 32, sk_dt: [DecisionTreeRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html#sklearn.tree.DecisionTreeRegressor) | [None](https://docs.python.org/3/builtins/constants.html#None) = None)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/regressor/_shrubs_regressor.py#L42)

Initializes the ShrubEnsemble regressor with the given parameters.

* **Parameters:**
  * **step_size** – The step size (i.e. learning rate of SGD) for updating
    the model. Can be a float or “adaptive”. Adaptive reduces the step
    size with more estimators, i.e. sets it to `1.0 / (n_estimators +
    1.0)`
  * **ensemble_regularizer** – 

    The regularizer for the weights of the
    ensemble. Supported values are:
    * `hard-L0`: L0 regularization via the prox-operator.
    * `L0`: L0 regularization via projection.
    * `L1`: L1 regularization via projection.
    * `none`: No regularization.

    Projection can be viewed as a softer regularization that drives the
    weights of each member towards 0, whereas `hard-l0` limits the
    number of trees in the entire ensemble.
  * **l_ensemble_reg** – 

    The regularization strength. Depending on the
    value of `ensemble_regularizer`, this parameter has different
    meanings:
    * `hard-L0`: then this parameter represent the total number of
      trees in the ensembles.
    * `L0` or `L1`: then this parameter is the regularization
      strength. In these cases the number of trees grow over time and
      only trees that do not contribute to the ensemble will be
      removed.
    * `none`: then this parameter is ignored.
  * **l_l2_reg** – The L2 regularization strength of the weights of each
    tree.
  * **l_tree_reg** – The regularization parameter for individual trees.
    Must be greater than or equal to 0. `l_tree_reg` controls the
    number of (overly) large trees in the ensemble by punishing the
    weights of each tree. Formally, the number of nodes of each tree is
    used as an additional regularizer.
  * **normalize_weights** – Whether to normalize the weights of the
    ensemble, i.e. the weight sum to 1.
  * **burnin_steps** – The number of burn-in steps before updating the
    model, i.e. the number of SGD steps to be take per each call of
    train
  * **update_leaves** – Whether to update the leaves of the trees as well
    using SGD.
  * **batch_size** – The batch size for training each individual tree.
    Internally, a sliding window is stored. Must be greater than or
    equal to 1.
  * **sk_dt** – Base object which is used to clone any new decision trees
    from. Note, that if you set random_state to an integer the exact
    same clone is used for any DT object

#### *classmethod* from_params(schema: [Any](https://docs.python.org/3/library/typing.html#typing.Any) = None, params: [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)] | [None](https://docs.python.org/3/builtins/constants.html#None) = None, random_seed: [int](https://docs.python.org/3/builtins/functions.html#int) = 1) → [Any](https://docs.python.org/3/library/typing.html#typing.Any)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/base/_learner_params.py#L170)

Construct an instance from parameters produced by `get_params`.

#### get_params() → [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)][[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/base/_learner_params.py#L163)

Return the hyper-parameters captured from the constructor.

#### predict(instance)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/regressor/_shrubs_regressor.py#L142)

#### train(instance)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/classifier/_shrubs_ensemble.py#L191)