vtl.nn.loss

Differentiable objectives consume a prediction autograd.Variable[T] and a target vtl.Tensor[T], returning a scalar-like variable for backpropagation. This module currently provides:

huber_loss and smooth_l1_loss are separate objectives: Huber scales its linear region by delta, while Smooth L1 divides the quadratic region by beta and has unit slope outside it.

Constructor signatures and reduction behavior are defined in the corresponding source files. In particular, binary cross entropy's from_logits setting and the expected representation for multiclass losses must match the model output.

cross_entropy_loss_with_options supports optional class weights, ignore_index, label_smoothing in [0, 1], and none, mean, or sum reductions. Use cross_entropy_loss_indices for Tensor[int] targets; it accepts a scalar target with logits shaped [C] or targets shaped [N, ...] with logits shaped [N, C, ...]. The generic cross_entropy_loss also accepts same-type integer-valued class-index targets and one-hot/soft targets matching the logits. For mean, weighted class-index targets divide by the sum of their non-ignored class weights; probability targets divide by the number of samples.

nll_loss(weight) uses mean reduction by default. Use nll_loss_with_options to select sum or per-sample reduction, or ignore a class-index target, while keeping the weight tensor optional:

objective := loss.nll_loss_with_options[f32](class_weights,
    ignore_index: -100,
    reduction: 'sum',
)

Use nll_loss_indices when class indices are stored in Tensor[int]; the regular nll_loss accepts a target tensor with the same element type as input. Both accept unbatched log-probabilities with shape [C] and a scalar class-index target, or batched log-probabilities with shape [N, C, ...] and class-index targets with shape [N, ...].

NLL none reduction returns one loss per target position (a scalar for an unbatched input). mean reduction divides by the total target weight when class weights are provided, and otherwise averages over non-ignored class-index targets. Matrix targets are an extra VTL extension; their weighted mean divides by the number of target positions.

import vtl.nn.loss

objective := loss.mse_loss[f32]()
loss_value := objective.loss(prediction, target)!
loss_value.backprop()!

See the neural-network tutorial for a training loop and loss tests for concrete input conventions.

fn bce_loss[T](config BCELossConfig) types.Loss[T]

bce_loss creates a new BCELoss instance.

fn cross_entropy_loss[T]() types.Loss[T]

cross_entropy_loss exposes this operation as part of the public API.

fn cross_entropy_loss_gate[T](input &vtl.Tensor[T], target &vtl.Tensor[T]) &CrossEntropyLossGate[T]

cross_entropy_loss_gate exposes this operation as part of the public API.

fn cross_entropy_loss_gate_with_options[T](input &vtl.Tensor[T], target &vtl.Tensor[T], weight &vtl.Tensor[T], ignore_index int, reduction string, label_smoothing f64) &CrossEntropyLossGate[T]

cross_entropy_loss_gate_with_options creates the configured backward gate.

fn cross_entropy_loss_indices[T](weight &vtl.Tensor[T], options CrossEntropyLossOptions) CrossEntropyIndexLoss[T]

cross_entropy_loss_indices creates an objective that accepts Tensor[int] targets.

fn cross_entropy_loss_with_options[T](weight &vtl.Tensor[T], options CrossEntropyLossOptions) types.Loss[T]

cross_entropy_loss_with_options creates a weighted cross-entropy objective.

fn focal_loss[T](config FocalLossConfig) types.Loss[T]

fn hinge_loss[T]() types.Loss[T]

fn huber_loss[T](config HuberLossConfig) types.Loss[T]

huber_loss creates a new HuberLoss instance.

fn kl_div_loss[T]() types.Loss[T]

kl_div_loss creates a new KLDivLoss instance with reduction = 'mean'.

fn l1_loss[T]() types.Loss[T]

fn mse_loss[T]() types.Loss[T]

mse_loss creates a new MSELoss instance.

fn nll_loss[T](weight &vtl.Tensor[T]) types.Loss[T]

nll_loss creates a new NLLLoss instance.

weight — optional per-class weight tensor; pass unsafe { nil } for uniform weights.

fn nll_loss_indices[T](weight &vtl.Tensor[T], options NLLLossOptions) NLLIndexLoss[T]

nll_loss_indices creates an NLL loss that accepts integer class-index targets.

fn nll_loss_with_options[T](weight &vtl.Tensor[T], options NLLLossOptions) types.Loss[T]

nll_loss_with_options creates an NLLLoss with a configurable ignore index and scalar (mean or sum) or per-sample (none) reduction. ignore_index applies to class-index targets.

fn sigmoid_cross_entropy_loss[T]() types.Loss[T]

sigmoid_cross_entropy_loss exposes this operation as part of the public API.

fn smooth_l1_loss[T](config SmoothL1LossConfig) types.Loss[T]

smooth_l1_loss creates a SmoothL1Loss with the configured reduction.

fn softmax_cross_entropy_loss[T]() types.Loss[T]

softmax_cross_entropy_loss exposes this operation as part of the public API.

fn (l &BCELoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (l &CrossEntropyIndexLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[int]) !&autograd.Variable[T]

loss computes CrossEntropyLoss from integer class-index targets.

fn (g &CrossEntropyLossGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &CrossEntropyLossGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (l &CrossEntropyLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (l &FocalLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

fn (_ &HingeLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

fn (l &HuberLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (_ &KLDivLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (_ &L1Loss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

fn (_ &MSELoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (l &NLLIndexLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[int]) !&autograd.Variable[T]

loss computes NLL from an integer class-index target tensor.

fn (l &NLLLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (_ &SigmoidCrossEntropyLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

fn (l &SmoothL1Loss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss computes the configured Smooth L1 reduction and attaches its gradient.

fn (_ &SoftmaxCrossEntropyLoss[T]) loss(input &autograd.Variable[T], target &vtl.Tensor[T]) !&autograd.Variable[T]

loss exposes this operation as part of the public API.

struct BCELoss[T] {
	from_logits bool
}

BCELoss computes Binary Cross-Entropy loss per element and returns the mean.

Formula (when from_logits = false): L = -mean(y·log(p) + (1-y)·log(1-p))

When from_logits = true (default), the stable logits form is used directly without materializing sigmoid probabilities.

Shape: input and target are both [batch, ...]; target values are in [0, 1].

import vtl.nn.loss
l := loss.bce_loss[f64]()          // from_logits: true
out := l.loss(logits, labels)!

struct BCELossConfig {
	from_logits bool = true // apply sigmoid to input (recommended for numerical stability)
}

BCELossConfig defines a public data structure for this module.

struct CrossEntropyIndexLoss[T] {
	inner &CrossEntropyLoss[T] = unsafe { nil }
}

CrossEntropyIndexLoss adapts integer class-index targets to CrossEntropyLoss.

struct CrossEntropyLoss[T] {
pub:
	weight          &vtl.Tensor[T] = unsafe { nil }
	ignore_index    int            = -100
	reduction       string         = 'mean' // 'mean' | 'sum' | 'none'
	label_smoothing f64
}

CrossEntropyLoss combines LogSoftmax + NLLLoss for unbatched, batched, and spatial logits. Reduction, class weights, and ignored indices match PyTorch.

struct CrossEntropyLossGate[T] {
pub:
	input           &vtl.Tensor[T] = unsafe { nil }
	target          &vtl.Tensor[T] = unsafe { nil }
	weight          &vtl.Tensor[T] = unsafe { nil }
	ignore_index    int            = -100
	reduction       string         = 'mean'
	label_smoothing f64
}

CrossEntropyLossGate defines a public data structure for this module.

struct CrossEntropyLossOptions {
pub:
	ignore_index    int    = -100
	reduction       string = 'mean'
	label_smoothing f64
}

CrossEntropyLossOptions configures ignored class indices, reduction, and smoothing.

struct FocalLoss[T] {
	config FocalLossConfig
}

FocalLoss computes mean binary focal loss.

struct FocalLossConfig {
pub:
	alpha       f64  = 0.25
	gamma       f64  = 2.0
	from_logits bool = true
}

FocalLossConfig configures binary focal loss; alpha weights positive examples and gamma focuses hard examples.

struct HingeLoss[T] {}

HingeLoss computes mean binary hinge loss for labels in {-1, +1}.

struct HuberLoss[T] {
	delta T
}

HuberLoss combines quadratic and linear error, controlled by delta.

Formula: L = mean(0.5·(x-y)²) if |x-y| ≤ delta L = mean(delta·(|x-y| - 0.5·delta)) if |x-y| > delta

Behaves like MSE for small errors and like MAE for large errors, making it more robust to outliers than pure MSE.

import vtl.nn.loss
l := loss.huber_loss[f64](delta: 1.0)
out := l.loss(prediction, target)!

struct HuberLossConfig {
	delta f64 = 1.0
}

HuberLossConfig defines a public data structure for this module.

struct KLDivLoss[T] {
pub:
	reduction string = 'mean'
}

KLDivLoss computes the Kullback-Leibler divergence loss.

Formula: D_KL(P ‖ Q) = sum(P · log(P / Q))

Input: [batch, n_classes] — Q log-probabilities (e.g. log-softmax output) Target: [batch, n_classes] — P probabilities (non-negative, sum to 1 per sample)

Fields:- reduction — 'mean' | 'sum' | 'none' (default: 'mean')

import vtl.nn.loss
l := loss.kl_div_loss[f64]()
out := l.loss(log_q, p_target)!

struct L1Loss[T] {}

L1Loss computes mean absolute error.

struct MSELoss[T] {}

MSELoss computes the Mean Squared Error between the model output and the target.

Formula: L = mean((input - target)²)

Typically used for regression tasks.

import vtl.nn.loss
l := loss.mse_loss[f64]()
out  := l.loss(prediction, target)!

struct NLLIndexLoss[T] {
	inner &NLLLoss[T] = unsafe { nil }
}

NLLIndexLoss adapts integer class-index targets to the generic NLL loss API.

struct NLLLoss[T] {
pub:
	weight       &vtl.Tensor[T] = unsafe { nil }
	ignore_index int            = -1
	reduction    string         = 'mean'
}

NLLLoss computes the Negative Log Likelihood loss.

Intended to be used after a log-softmax layer for multi-class classification.

Input shape: [n_classes] or [batch, n_classes, ...] — log-probabilities (log-softmax output). Unbatched class-index targets are scalar; batched targets omit the class axis or match the input shape for one-hot/soft labels.

Fields:- weight — optional per-class weight tensor (nil = uniform weights)

• ignore_index — class index to ignore in class-index targets (default: -1 = none)
• reduction — 'none' | 'mean' | 'sum' (default: 'mean')

import vtl.nn.loss
l := loss.nll_loss[f64](unsafe { nil })
log_probs := log_softmax_output  // shape [batch, n_classes]
out := l.loss(log_probs, one_hot_target)!

struct NLLLossOptions {
pub:
	ignore_index int    = -1
	reduction    string = 'mean'
}

NLLLossOptions configures ignored class indices and scalar reduction.

struct SigmoidCrossEntropyLoss[T] {}

SigmoidCrossEntropyLoss

struct SmoothL1Loss[T] {
	beta      T
	reduction string
}

SmoothL1Loss computes PyTorch-style Smooth L1 with configurable reduction.

For error d, the loss is 0.5*d*d/beta when |d| < beta, and |d| - 0.5*beta otherwise. A beta of zero selects absolute error. This is distinct from Huber loss, whose outer linear region is scaled by delta.

import vtl.nn.loss
objective := loss.smooth_l1_loss[f32](beta: 0.5)
result := objective.loss(prediction, target)!

struct SmoothL1LossConfig {
	beta      f64    = 1.0
	reduction string = 'mean'
}

SmoothL1LossConfig configures the transition point between quadratic and linear loss regions. Beta must be finite and non-negative; zero selects absolute error. The default beta is 1.

struct SoftmaxCrossEntropyLoss[T] {}

SoftmaxCrossEntropyLoss defines a public data structure for this module.