StochasticGradientTree

class capymoa.regressor.StochasticGradientTree

Bases: MOARegressor

Stochastic Gradient Tree regressor.

Stochastic Gradient Tree (SGT) [1] is an incremental decision tree that learns using stochastic gradient information as its source of supervision, rather than a heuristic such as information gain. Instead of using soft splits or rebuilding a new tree for every update, as prior gradient-based tree learners did in the batch setting, SGT accumulates per-node gradient and Hessian statistics online and uses them to make hard splitting decisions incrementally. Because splitting is driven only by the loss function’s gradients and Hessians, the same algorithm can be applied to classification, regression, or multi-instance learning simply by changing the loss function.

>>> from capymoa.regressor import StochasticGradientTree
>>> from capymoa.datasets import Fried
>>> from capymoa.evaluation import prequential_evaluation
>>>
>>> stream = Fried()
>>> learner = StochasticGradientTree(stream.get_schema())
>>> results = prequential_evaluation(stream, learner, max_instances=1000)
>>> round(results["cumulative"].rmse(), 2)
15.49

[1]

Gouk, Henry, Bernhard Pfahringer, and Eibe Frank. “Stochastic Gradient Trees.” Proceedings of The 11th Asian Conference on Machine Learning (ACML 2019). PMLR 101, 2019, pp. 1094-1109.

__init__(

schema: Schema,

grace_period: int = 200,

lambda_: float = 0.1,

warm_start: int = 1000,

confidence: float = 1e-06,

split_test: Literal['TTest'] = 'TTest',

disable_resplits: bool = False,

) → None

Construct StochasticGradientTree regressor.

Parameters:

• grace_period – The number of instances a leaf should observe between split attempts.

• lambda – Regularization parameter lambda.

• warm_start – Number of instances to use for fitting the discretizers.

• confidence – The level of confidence required that a split candidate is an improvement before the split is actually performed.

• split_test –

  Which type of hypothesis test to use for determining when to split.

  • TTest: Use a t-Test for checking statistical significance.

• disable_resplits – Disable node resplitting.

cli_help()

classmethod from_params(

schema: Any = None,

params: dict[str, Any] | None = None,

random_seed: int = 1,

) → Any

Construct an instance from parameters produced by get_params.

get_params() → dict[str, Any]

Return the hyper-parameters captured from the constructor.

predict(instance)

train(instance)