StochasticGradientTree

class capymoa.classifier.StochasticGradientTree

Bases: MOAClassifier

Stochastic Gradient Tree classifier.

Stochastic Gradient Tree (SGT) [1] is an incremental decision tree that learns using stochastic gradient information as its source of supervision, rather than a heuristic such as information gain. Instead of using soft splits or rebuilding a new tree for every update, as prior gradient-based tree learners did in the batch setting, SGT accumulates per-node gradient and Hessian statistics online and uses them to make hard splitting decisions incrementally. Because splitting is driven only by the loss function’s gradients and Hessians, the same algorithm can be applied to classification, regression, or multi-instance learning simply by changing the loss function.

>>> from capymoa.classifier import StochasticGradientTree
>>> from capymoa.datasets import ElectricityTiny
>>> from capymoa.evaluation import prequential_evaluation
>>>
>>> stream = ElectricityTiny()
>>> classifier = StochasticGradientTree(stream.get_schema())
>>> results = prequential_evaluation(stream, classifier, max_instances=1000)
>>> print(f"{results['cumulative'].accuracy():.1f}")
50.6

[1]

Gouk, Henry, Bernhard Pfahringer, and Eibe Frank. “Stochastic Gradient Trees.” Proceedings of The 11th Asian Conference on Machine Learning (ACML 2019). PMLR 101, 2019, pp. 1094-1109.

__init__(

schema: Schema,

grace_period: int = 200,

lambda_: float = 0.1,

warm_start: int = 1000,

confidence: float = 1e-06,

split_test: Literal['TTest'] = 'TTest',

disable_resplits: bool = False,

) → None

Construct StochasticGradientTree classifier.

Parameters:

• grace_period – The number of instances a leaf should observe between split attempts.

• lambda – Regularization parameter lambda.

• warm_start – Number of instances to use for fitting the discretizers.

• confidence – The level of confidence required that a split candidate is an improvement before the split is actually performed.

• split_test –

  Which type of hypothesis test to use for determining when to split.

  • TTest: Use a t-Test for checking statistical significance.

• disable_resplits – Disable node resplitting.

cli_help()

classmethod from_params(

schema: Any = None,

params: dict[str, Any] | None = None,

random_seed: int = 1,

) → Any

Construct an instance from parameters produced by get_params.

get_params() → dict[str, Any]

Return the hyper-parameters captured from the constructor.

predict(instance: Instance) → int | None

Predict the label of an instance.

The base implementation calls predict_proba() and returns the label with the highest probability.

Parameters:

instance – The instance to predict the label for.

Returns:

The predicted label or None if the classifier is unable to make a prediction.

predict_proba(

instance,

) → ndarray[tuple[Any, ...], dtype[float64]] | None

Return probability estimates for each label.

Parameters:

instance – The instance to estimate the probabilities for.

Returns:

An array of probabilities for each label or None if the classifier is unable to make a prediction.

train(instance)

Train the classifier with a labeled instance.

Parameters:

instance – The labeled instance to train the classifier with.

random_seed: int

The random seed for reproducibility.

When implementing a classifier ensure random number generators are seeded.

schema: Schema

The schema representing the instances.