Regressors#

Regressors implement the capymoa.base.Regressor interface.

class capymoa.regressor.SOKNLBT[source]#

Bases: MOARegressor

The base tree for Self-Optimising K Nearest Leaves as distribed by Sun. at el.

__init__(
schema: Schema,
subspace_size_size: int = 2,
split_criterion: SplitCriterion | str = 'VarianceReductionSplitCriterion',
grace_period: int = 200,
split_confidence: float = 1e-07,
tie_threshold: float = 0.05,
page_hinckley_alpha: float = 0.005,
page_hinckley_threshold: int = 50,
alternate_tree_fading_factor: float = 0.995,
alternate_tree_t_min: int = 150,
alternate_tree_time: int = 1500,
learning_ratio: float = 0.02,
learning_ratio_decay_factor: float = 0.001,
learning_ratio_const: bool = False,
random_seed: int | None = None,
) None[source]#

Construct SelfOptimisingBaseTree.

Parameters:
  • subspace_size_size – Number of features per subset for each node split. Negative values = #features - k

  • split_criterion – Split criterion to use.

  • grace_period – Number of instances a leaf should observe between split attempts.

  • split_confidence – Allowed error in split decision, values close to 0 will take long to decide.

  • tie_threshold – Threshold below which a split will be forced to break ties.

  • page_hinckley_alpha – Alpha value to use in the Page Hinckley change detection tests.

  • page_hinckley_threshold – Threshold value used in the Page Hinckley change detection tests.

  • alternate_tree_fading_factor – Fading factor used to decide if an alternate tree should replace an original.

  • alternate_tree_t_min – Tmin value used to decide if an alternate tree should replace an original.

  • alternate_tree_time – The number of instances used to decide if an alternate tree should be discarded.

  • learning_ratio – Learning ratio to used for training the Perceptrons in the leaves.

  • learning_ratio_decay_factor – Learning rate decay factor (not used when learning rate is constant).

  • learning_ratio_const – Keep learning rate constant instead of decaying.

class capymoa.regressor.SOKNL[source]#

Bases: MOARegressor

__init__(
schema=None,
CLI=None,
random_seed=1,
tree_learner=None,
ensemble_size=100,
max_features=0.6,
lambda_param=6.0,
drift_detection_method=None,
warning_detection_method=None,
disable_drift_detection=False,
disable_background_learner=False,
disable_self_optimising=False,
k_value=10,
)[source]#
class capymoa.regressor.ORTO[source]#

Bases: MOARegressor

Implementation of the ORTO tree as described by Ikonomovska et al.

__init__(
schema: Schema,
max_trees: int = 10,
max_option_level: int = 10,
option_decay_factor: float = 0.9,
option_fading_factor: float = 0.9995,
split_criterion: SplitCriterion | str = 'VarianceReductionSplitCriterion',
grace_period: int = 200,
split_confidence: float = 1e-07,
tie_threshold: float = 0.05,
page_hinckley_alpha: float = 0.005,
page_hinckley_threshold: int = 50,
alternate_tree_fading_factor: float = 0.995,
alternate_tree_t_min: int = 150,
alternate_tree_time: int = 1500,
regression_tree: bool = False,
learning_ratio: float = 0.02,
learning_ratio_decay_factor: float = 0.001,
learning_ratio_const: bool = False,
random_seed: int | None = None,
) None[source]#

Construct ORTO.

Parameters:
  • max_trees – The maximum number of trees contained in the option tree.

  • max_option_level – The maximal depth at which option nodes can be created.

  • option_decay_factor – The option decay factor that determines how many options can be selected at a given level.

  • option_fading_factor – The fading factor used for comparing subtrees of an option node.

  • split_criterion – Split criterion to use.

  • grace_period – Number of instances a leaf should observe between split attempts.

  • split_confidence – Allowed error in split decision, values close to 0 will take long to decide.

  • tie_threshold – Threshold below which a split will be forced to break ties.

  • page_hinckley_alpha – Alpha value to use in the Page Hinckley change detection tests.

  • page_hinckley_threshold – Threshold value used in the Page Hinckley change detection tests.

  • alternate_tree_fading_factor – Fading factor used to decide if an alternate tree should replace an original.

  • alternate_tree_t_min – Tmin value used to decide if an alternate tree should replace an original.

  • alternate_tree_time – The number of instances used to decide if an alternate tree should be discarded.

  • regression_tree – Build a regression tree instead of a model tree.

  • learning_ratio – Learning ratio to used for training the Perceptrons in the leaves.

  • learning_ratio_decay_factor – Learning rate decay factor (not used when learning rate is constant).

  • learning_ratio_const – Keep learning rate constant instead of decaying.

class capymoa.regressor.KNNRegressor[source]#

Bases: MOARegressor

The default number of neighbors (k) is set to 3 instead of 10 (as in MOA)

__init__(
schema=None,
CLI=None,
random_seed=1,
k=3,
median=False,
window_size=1000,
)[source]#
class capymoa.regressor.FIMTDD[source]#

Bases: MOARegressor

Implementation of the FIMT-DD tree as described by Ikonomovska et al.

__init__(
schema: Schema,
split_criterion: SplitCriterion | str = 'VarianceReductionSplitCriterion',
grace_period: int = 200,
split_confidence: float = 1e-07,
tie_threshold: float = 0.05,
page_hinckley_alpha: float = 0.005,
page_hinckley_threshold: int = 50,
alternate_tree_fading_factor: float = 0.995,
alternate_tree_t_min: int = 150,
alternate_tree_time: int = 1500,
regression_tree: bool = False,
learning_ratio: float = 0.02,
learning_ratio_decay_factor: float = 0.001,
learning_ratio_const: bool = False,
random_seed: int | None = None,
) None[source]#

Construct FIMTDD.

Parameters:
  • split_criterion – Split criterion to use.

  • grace_period – Number of instances a leaf should observe between split attempts.

  • split_confidence – Allowed error in split decision, values close to 0 will take long to decide.

  • tie_threshold – Threshold below which a split will be forced to break ties.

  • page_hinckley_alpha – Alpha value to use in the Page Hinckley change detection tests.

  • page_hinckley_threshold – Threshold value used in the Page Hinckley change detection tests.

  • alternate_tree_fading_factor – Fading factor used to decide if an alternate tree should replace an original.

  • alternate_tree_t_min – Tmin value used to decide if an alternate tree should replace an original.

  • alternate_tree_time – The number of instances used to decide if an alternate tree should be discarded.

  • regression_tree – Build a regression tree instead of a model tree.

  • learning_ratio – Learning ratio to used for training the Perceptrons in the leaves.

  • learning_ratio_decay_factor – Learning rate decay factor (not used when learning rate is constant).

  • learning_ratio_const – Keep learning rate constant instead of decaying.

class capymoa.regressor.ARFFIMTDD[source]#

Bases: MOARegressor

Modified Fast Incremental Model Tree with Drift Detection for basic learner for ARF-Regas described by Ikonomovska et al.

__init__(
schema: Schema,
subspace_size_size: int = 2,
split_criterion: SplitCriterion | str = 'VarianceReductionSplitCriterion',
grace_period: int = 200,
split_confidence: float = 1e-07,
tie_threshold: float = 0.05,
page_hinckley_alpha: float = 0.005,
page_hinckley_threshold: int = 50,
alternate_tree_fading_factor: float = 0.995,
alternate_tree_t_min: int = 150,
alternate_tree_time: int = 1500,
learning_ratio: float = 0.02,
learning_ratio_decay_factor: float = 0.001,
learning_ratio_const: bool = False,
random_seed: int | None = None,
) None[source]#

Construct ARFFIMTDD.

Parameters:
  • subspace_size_size – Number of features per subset for each node split. Negative values = #features - k

  • split_criterion – Split criterion to use.

  • grace_period – Number of instances a leaf should observe between split attempts.

  • split_confidence – Allowed error in split decision, values close to 0 will take long to decide.

  • tie_threshold – Threshold below which a split will be forced to break ties.

  • page_hinckley_alpha – Alpha value to use in the Page Hinckley change detection tests.

  • page_hinckley_threshold – Threshold value used in the Page Hinckley change detection tests.

  • alternate_tree_fading_factor – Fading factor used to decide if an alternate tree should replace an original.

  • alternate_tree_t_min – Tmin value used to decide if an alternate tree should replace an original.

  • alternate_tree_time – The number of instances used to decide if an alternate tree should be discarded.

  • learning_ratio – Learning ratio to used for training the Perceptrons in the leaves.

  • learning_ratio_decay_factor – Learning rate decay factor (not used when learning rate is constant).

  • learning_ratio_const – Keep learning rate constant instead of decaying.

class capymoa.regressor.AdaptiveRandomForestRegressor[source]#

Bases: MOARegressor

Adaptive Random Forest Regressor

This class implements the Adaptive Random Forest (ARF) algorithm, which is an ensemble regressor capable of adapting to concept drift.

ARF is implemented in MOA (Massive Online Analysis) and provides several parameters for customization.

See also capymoa.classifier.AdaptiveRandomForestClassifier See capymoa.base.MOARegressor for train and predict.

Reference:

Adaptive random forests for data stream regression. Heitor Murilo Gomes, J. P. Barddal, L. E. B. Ferreira, A. Bifet. ESANN, pp. 267-272, 2018.

Example usage:

>>> from capymoa.datasets import Fried
>>> from capymoa.regressor import AdaptiveRandomForestRegressor
>>> from capymoa.evaluation import prequential_evaluation
>>> stream = Fried()
>>> schema = stream.get_schema()
>>> learner = AdaptiveRandomForestRegressor(schema)
>>> results = prequential_evaluation(stream, learner, max_instances=1000)
>>> results["cumulative"].RMSE()
3.659072011685404
__init__(
schema=None,
CLI=None,
random_seed=1,
tree_learner=None,
ensemble_size=100,
max_features=0.6,
lambda_param=6.0,
drift_detection_method=None,
warning_detection_method=None,
disable_drift_detection=False,
disable_background_learner=False,
)[source]#

Construct an Adaptive Random Forest Regressor

Parameters:
  • schema – The schema of the stream. If not provided, it will be inferred from the data.

  • CLI – Command Line Interface (CLI) options for configuring the ARF algorithm. If not provided, default options will be used.

  • random_seed – Seed for the random number generator.

  • tree_learner – The tree learner to use. If not provided, a default Hoeffding Tree is used.

  • ensemble_size – The number of trees in the ensemble.

  • max_features – The maximum number of features to consider when splitting a node. If provided as a float between 0.0 and 1.0, it represents the percentage of features to consider. If provided as an integer, it specifies the exact number of features to consider. If provided as the string “sqrt”, it indicates that the square root of the total number of features. If not provided, the default value is 60%.

  • lambda_param – The lambda parameter that controls the Poisson distribution for the online bagging simulation.

  • drift_detection_method – The method used for drift detection.

  • warning_detection_method – The method used for warning detection.

  • disable_drift_detection – Whether to disable drift detection.

  • disable_background_learner – Whether to disable background learning.

class capymoa.regressor.PassiveAggressiveRegressor[source]#

Bases: SKRegressor

Streaming Passive Aggressive regressor

This wraps sklearn.linear_model.PassiveAggressiveRegressor for ease of use in the streaming context. Some options are missing because they are not relevant in the streaming context.

Online Passive-Aggressive Algorithms K. Crammer, O. Dekel, J. Keshat, S. Shalev-Shwartz, Y. Singer - JMLR (2006)

>>> from capymoa.datasets import Fried
>>> from capymoa.regressor import PassiveAggressiveRegressor
>>> from capymoa.evaluation import prequential_evaluation
>>> stream = Fried()
>>> schema = stream.get_schema()
>>> learner = PassiveAggressiveRegressor(schema)
>>> results = prequential_evaluation(stream, learner, max_instances=1000)
>>> results["cumulative"].RMSE()
3.7...
sklearner: PassiveAggressiveRegressor#

The underlying scikit-learn object. See: sklearn.linear_model.PassiveAggressiveRegressor

__init__(
schema: Schema,
max_step_size: float = 1.0,
fit_intercept: bool = True,
loss: str = 'epsilon_insensitive',
average: bool = False,
random_seed=1,
)[source]#

Construct a passive aggressive regressor.

Parameters:
  • schema – Stream schema

  • max_step_size – Maximum step size (regularization).

  • fit_intercept – Whether the intercept should be estimated or not. If False, the data is assumed to be already centered.

  • loss

    The loss function to be used:

    • "epsilon_insensitive": equivalent to PA-I in the reference paper.

    • "squared_epsilon_insensitive": equivalent to PA-II in the reference paper.

  • average – When set to True, computes the averaged SGD weights and stores the result in the sklearner.coef_ attribute. If set to an int greater than 1, averaging will begin once the total number of samples seen reaches average. So average=10 will begin averaging after seeing 10 samples.

  • random_seed – Seed for the random number generator.

class capymoa.regressor.SGDRegressor[source]#

Bases: SKRegressor

Streaming stochastic gradient descent regressor.

This wraps sklearn.linear_model.SGDRegressor for ease of use in the streaming context. Some options are missing because they are not relevant in the streaming context. Furthermore, the learning rate is constant.

>>> from capymoa.datasets import Fried
>>> from capymoa.regressor import PassiveAggressiveRegressor
>>> from capymoa.evaluation import prequential_evaluation
>>> stream = Fried()
>>> schema = stream.get_schema()
>>> learner = SGDRegressor(schema)
>>> results = prequential_evaluation(stream, learner, max_instances=1000)
>>> results["cumulative"].RMSE()
4.625...
sklearner: SGDRegressor#

The underlying scikit-learn object

__init__(
schema: Schema,
loss: Literal['squared_error', 'huber', 'epsilon_insensitive', 'squared_epsilon_insensitive'] = 'squared_error',
penalty: Literal['l2', 'l1', 'elasticnet'] | None = 'l2',
alpha: float = 0.0001,
l1_ratio: float = 0.15,
fit_intercept: bool = True,
epsilon: float = 0.1,
learning_rate: str = 'invscaling',
eta0: float = 0.01,
random_seed: int | None = None,
)[source]#

Construct stochastic gradient descent Regressor.

Parameters:
  • schema – Describes the datastream’s structure.

  • loss – The loss function to be used.

  • penalty – The penalty (aka regularization term) to be used.

  • alpha – Constant that multiplies the regularization term.

  • l1_ratio – The Elastic Net mixing parameter, with 0 <= l1_ratio <= 1. l1_ratio=0 corresponds to L2 penalty, l1_ratio=1 to L1. Only used if penalty is ‘elasticnet’. Values must be in the range [0.0, 1.0].

  • fit_intercept – Whether the intercept (bias) should be estimated or not. If False, the data is assumed to be already centered.

  • epsilon – Epsilon in the epsilon-insensitive loss functions; only if loss is ‘huber’, ‘epsilon_insensitive’, or ‘squared_epsilon_insensitive’. For ‘huber’, determines the threshold at which it becomes less important to get the prediction exactly right. For epsilon-insensitive, any differences between the current prediction and the correct label are ignored if they are less than this threshold.

  • learning_rate – The size of the gradient step.

  • eta0 – The initial learning rate for the ‘constant’, ‘invscaling’ or ‘adaptive’ schedules. The default value is 0.0 as eta0 is not used by the default schedule ‘optimal’.

  • random_seed – Seed for reproducibility.