# `functional`

A collection of functional utilities for OCL.

## Functions

| [`hinton_distillation_loss`](#capymoa.ocl.util.functional.hinton_distillation_loss)   | Hinton's distillation loss <sup>[1](#f1)</sup> .   |
|-----------------------------------------------------------------------------|----------------------------------------------------|

### capymoa.ocl.util.functional.hinton_distillation_loss(teacher_logits: [Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor), student_logits: [Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor), temperature: [float](https://docs.python.org/3/builtins/functions.html#float) = 1.0) → [Tensor](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor)[[source]](https://github.com/adaptive-machine-learning/CapyMOA/blob/3e255b1/src/capymoa/ocl/util/functional.py#L7)

Hinton’s distillation loss <sup>[1](#f1)</sup> .

$$
L_{KD} = T^2 KL(softmax(z_t / T), softmax(z_s / T))

$$

where $T$ is the temperature, $z_s$ are the student logits, and
$z_t$ are the teacher logits.

* <a id='f1'>**[1]**</a> Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531 [Cs, Stat]. http://arxiv.org/abs/1503.02531
* **Parameters:**
  * **teacher_logits** – Teacher logits of shape `(batch_size, num_classes)`.
  * **student_logits** – Student logits of shape `(batch_size, num_classes)`.
  * **temperature** – Temperature for distillation. Higher values produce softer
    probability distributions.
* **Returns:**
  The distillation loss as a scalar tensor.