vtl.nn.layers

Layers transform autograd variables and expose trainable parameters to optimizers. Implementations on main include:

The common low-level constructor returns a types.Layer[T]:

gru_layer(ctx, input_size, hidden_size) returns this interface and starts each sequence from a zero hidden state. Use new_gru_layer when you need the typed GRULayer[T] API: forward_with_state(input, h0) returns both the full sequence and final hidden state, with autograd support for either output and the initial state. The typed layer also provides forward_batch_first and forward_batch_first_with_state for [batch, sequence, features] input; its hidden states keep shape [batch, hidden_size].

lstm_layer uses batch-first [batch, sequence, features] inputs and returns the output at every timestep. It supports stacked layers and full CPU BPTT for input and parameters, with gate matrices ordered input, forget, cell, output. Use new_lstm_layer and forward_with_state(input, h0, c0) to provide initial states and receive final hidden and cell states shaped [num_layers, batch, hidden_size]; autograd propagates gradients through each of the three outputs and back to the initial states.

import vtl.autograd
import vtl.nn.layers

ctx := autograd.ctx[f32]()
layer := layers.linear_layer[f32](ctx, 4, 2)

Shapes, parameters, and supported input ranks are layer-specific; follow each constructor's source comments and tests. Sequential composition is described in the models reference. CUDA and Vulkan paths are conditional and experimental; see the device notes.

GroupNorm

GroupNorm splits channels into num_groups and normalizes each group independently for every batch item, including trailing spatial dimensions. Its input layout is channel-first ([batch, channels, ...]). The constructor's input_shape excludes batch; with affine enabled (the default), it learns one scale and bias per channel.

import vtl.autograd
import vtl.nn.layers

ctx := autograd.ctx[f32]()
norm := layers.group_norm_layer[f32](ctx, [8, 16, 16], 4, layers.GroupNormConfig{})

num_groups must be positive and divide the channel count. GroupNorm uses current input statistics in both training and evaluation, so it has no running statistics state.

RMSNorm

RMSNorm divides each trailing normalized_shape block by its root mean square, then applies a learnable per-element weight when elementwise_affine is true. Unlike LayerNorm, it does not subtract the mean or learn a bias. If eps is omitted, VTL uses the machine epsilon for f32 or f64 computations, matching PyTorch's default for these dtypes. See the PyTorch RMSNorm reference for the formula and public API.

import vtl.autograd
import vtl.nn.layers

ctx := autograd.ctx[f32]()
norm := layers.rms_norm_layer[f32](ctx, [128], layers.RMSNormConfig{})

fn attention_gate[T](input &vtl.Tensor[T], w_q &vtl.Tensor[T], w_k &vtl.Tensor[T], w_v &vtl.Tensor[T], w_o &vtl.Tensor[T], num_heads int, head_dim int) &AttentionGate[T]

attention_gate exposes this operation as part of the public API.

fn avgpool2d_gate[T](input &vtl.Tensor[T], kernel []int, padding []int, stride []int) &AvgPool2DGate[T]

avgpool2d_gate exposes this operation as part of the public API.

fn avgpool2d_layer[T](ctx &autograd.Context[T], input_shape []int, kernel []int, padding []int, stride []int) types.Layer[T]

avgpool2d_layer creates an AveragePool2DLayer.

fn batchnorm1d_gate[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], mean &vtl.Tensor[T], var_ &vtl.Tensor[T], eps f64) &BatchNorm1DGate[T]

batchnorm1d_gate exposes this operation as part of the public API.

fn batchnorm1d_gate_with_mode[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], mean &vtl.Tensor[T], var_ &vtl.Tensor[T], eps f64, training bool) &BatchNorm1DGate[T]

batchnorm1d_gate_with_mode creates the backward gate for train or eval mode.

fn batchnorm1d_layer[T](ctx &autograd.Context[T], num_features int, config BatchNorm1DConfig) types.Layer[T]

batchnorm1d_layer exposes this operation as part of the public API.

fn batchnorm2d_gate_with_mode[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], mean &vtl.Tensor[T], var_ &vtl.Tensor[T], eps f64, training bool) &BatchNorm2DGate[T]

batchnorm2d_gate_with_mode creates the backward gate for train or eval mode.

fn batchnorm2d_layer[T](ctx &autograd.Context[T], num_features int, config BatchNorm2DConfig) types.Layer[T]

batchnorm2d_layer creates a BatchNorm2D layer for num_features channels.

fn batchnorm3d_gate_with_mode[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], mean &vtl.Tensor[T], var_ &vtl.Tensor[T], eps f64, training bool) &BatchNorm3DGate[T]

batchnorm3d_gate_with_mode creates the backward gate for train or eval mode.

fn batchnorm3d_layer[T](ctx &autograd.Context[T], num_features int, config BatchNorm3DConfig) types.Layer[T]

batchnorm3d_layer creates a BatchNorm3D layer for num_features channels.

fn conv1d_layer[T](ctx &autograd.Context[T], in_channels int, out_channels int,
	kernel_size int, config Conv1DConfig, input_length int) types.Layer[T]

conv1d_layer creates a 1D convolution layer. Pass input_length as -1 when the input length is dynamic; output_shape then reports -1 for its length.

fn conv2d_gate[T](input &vtl.Tensor[T], weight &vtl.Tensor[T], bias &vtl.Tensor[T], kernel_size []int, config Conv2DConfig) &Conv2DGate[T]

conv2d_gate exposes this operation as part of the public API.

fn conv2d_layer[T](ctx &autograd.Context[T], in_ch int, out_ch int, kernel_size []int, config Conv2DConfig, input_shape []int) types.Layer[T]

conv2d_layer creates a Conv2DLayer.

fn cuda_linear_enabled() bool

cuda_linear_enabled exposes this operation as part of the public API.

fn cuda_tests_enabled() bool

cuda_tests_enabled exposes this operation as part of the public API.

fn dropout_layer[T](ctx &autograd.Context[T], output_shape []int, data DropoutLayerConfig) types.Layer[T]

dropout_layer exposes this operation as part of the public API.

fn elu_layer[T](ctx &autograd.Context[T], output_shape []int, data EluLayerConfig) types.Layer[T]

elu_layer exposes this operation as part of the public API.

fn embedding_gate[T](input &vtl.Tensor[T], weight &vtl.Tensor[T]) &EmbeddingGate[T]

embedding_gate exposes this operation as part of the public API.

fn embedding_layer[T](ctx &autograd.Context[T], vocab_size int, embedding_dim int) types.Layer[T]

embedding_layer creates an EmbeddingLayer.

fn flatten_layer[T](ctx &autograd.Context[T], shape []int) types.Layer[T]

flatten_layer exposes this operation as part of the public API.

fn gelu_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

gelu_layer exposes this operation as part of the public API.

fn global_avgpool2d_gate[T](input &vtl.Tensor[T]) &GlobalAvgPool2DGate[T]

global_avgpool2d_gate exposes this operation as part of the public API.

fn global_avgpool2d_layer[T](ctx &autograd.Context[T]) types.Layer[T]

global_avgpool2d_layer creates a GlobalAvgPool2DLayer.

fn group_norm_gate[T](input &vtl.Tensor[T], num_groups int, gamma &vtl.Tensor[T], beta &vtl.Tensor[T], eps f64) &GroupNormGate[T]

group_norm_gate creates a backward gate for GroupNorm.

fn group_norm_layer[T](ctx &autograd.Context[T], input_shape []int, num_groups int, config GroupNormConfig) types.Layer[T]

group_norm_layer creates a GroupNorm layer. input_shape excludes batch and uses channel-first order: [channels, ...spatial dimensions].

fn gru_layer[T](ctx &autograd.Context[T], input_size int, hidden_size int) types.Layer[T]

gru_layer creates a GRU layer with reset, update, and candidate gates.

fn hardswish_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

hardswish_forward_f32 uses Vulkan when opted in, then falls back to CPU.

fn hardswish_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

hardswish_forward_f32_try exposes the Vulkan HardSwish kernel when available.

fn hardswish_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

fn input_layer[T](ctx &autograd.Context[T], shape []int) types.Layer[T]

input_layer exposes this operation as part of the public API.

fn layer_norm_layer[T](ctx &autograd.Context[T], normalized_shape []int, config LayerNormConfig) types.Layer[T]

layer_norm_layer creates a LayerNormLayer. layer_norm_layer creates a LayerNormLayer.

fn layernorm_gate[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], eps f64) &LayerNormGate[T]

layernorm_gate exposes this operation as part of the public API.

fn layernorm_gate_with_shape[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], eps f64, normalized_shape []int) &LayerNormGate[T]

layernorm_gate_with_shape creates a backward gate for trailing dimensions.

fn leaky_relu_layer[T](ctx &autograd.Context[T], output_shape []int, data LeakyReluLayerConfig) types.Layer[T]

leaky_relu_layer exposes this operation as part of the public API.

fn linear_bind_output_gpu(_result voidptr)

linear_bind_output_gpu exposes this operation as part of the public API.

fn linear_forward_cuda_f64(x &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64]) !&vtl.Tensor[f64]

linear_forward_cuda_f64 stub when not built with -d cuda.

fn linear_forward_f32(input &vtl.Tensor[f32], weights &vtl.Tensor[f32], bias &vtl.Tensor[f32]) !&vtl.Tensor[f32]

linear_forward_f32 uses Vulkan GEMM when VTL_USE_VULKAN=1 and -d vulkan, else CPU.

fn linear_forward_f32_use_vulkan() bool

linear_forward_f32_use_vulkan exposes this operation as part of the public API.

fn linear_forward_f64(input &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64],
	input_gpu voidptr, session voidptr) !&vtl.Tensor[f64]

linear_forward_f64 may use CUDA when enabled at runtime (VTL_USE_CUDA=1, -d cuda). session is an opaque pointer to autograd_cuda.DeviceSession when CUDA is enabled.

fn linear_forward_f64_session(input &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64],
	_input_gpu voidptr, session voidptr) !&vtl.Tensor[f64]

linear_forward_f64_session is a no-op without -d cuda.

fn linear_forward_f64_use_cuda() bool

linear_forward_f64_use_cuda exposes this operation as part of the public API.

fn linear_forward_vulkan[T](x &vtl.Tensor[T], weights &vtl.Tensor[T], bias &vtl.Tensor[T]) !&vtl.Tensor[T]

linear_forward_vulkan returns error if Vulkan is not enabled

fn linear_forward_vulkan_f32(x &vtl.Tensor[f32], weights &vtl.Tensor[f32], bias &vtl.Tensor[f32]) !&vtl.Tensor[f32]

linear_forward_vulkan_f32 stub when not built with -d vulkan.

fn linear_layer[T](ctx &autograd.Context[T], input_dim int, output_dim int) types.Layer[T]

linear_layer creates a LinearLayer.

fn linear_take_gpu_input(_input voidptr) voidptr

linear_take_gpu_input is a no-op without -d cuda (keeps f32 nn builds free of f64 GPU take).

fn lstm_layer[T](ctx &autograd.Context[T], input_size int, hidden_size int, num_layers int) types.Layer[T]

lstm_layer creates an LSTMLayer.

fn maxpool2d_layer[T](ctx &autograd.Context[T], input_shape []int, kernel []int, padding []int, stride []int) types.Layer[T]

maxpool2d_layer exposes this operation as part of the public API.

fn mish_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

mish_layer exposes this operation as part of the public API.

fn multihead_attention_layer[T](ctx &autograd.Context[T], embed_dim int, num_heads int) types.Layer[T]

multihead_attention_layer creates a MultiHeadAttentionLayer.

fn new_gru_layer[T](ctx &autograd.Context[T], input_size int, hidden_size int) &GRULayer[T]

new_gru_layer creates a GRU layer value that exposes forward_with_state. Use gru_layer when composing the layer inside Sequential.

fn new_lstm_layer[T](ctx &autograd.Context[T], input_size int, hidden_size int, num_layers int) &LSTMLayer[T]

new_lstm_layer creates an LSTM layer value with explicit state support. Use lstm_layer when composing the layer inside Sequential.

fn positional_encoding_layer[T](ctx &autograd.Context[T], embed_dim int, max_len int) !types.Layer[T]

positional_encoding_layer creates a PositionalEncodingLayer.

fn relu_forward_cuda(x &vtl.Tensor[f64]) !&vtl.Tensor[f64]

relu_forward_cuda returns error if CUDA is not enabled

fn relu_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

relu_forward_f32 uses Vulkan in-place ReLU when opted in (-d vulkan).

fn relu_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

relu_forward_f32_try exposes this operation as part of the public API.

fn relu_forward_vulkan[T](x &vtl.Tensor[T]) !&vtl.Tensor[T]

relu_forward_vulkan returns error if Vulkan is not enabled

fn relu_forward_vulkan_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

relu_forward_vulkan_f32 exposes this operation as part of the public API.

fn relu_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

relu_layer exposes this operation as part of the public API.

fn rms_norm_gate[T](input &vtl.Tensor[T], weight &vtl.Tensor[T], normalized_shape []int, eps f64) &RMSNormGate[T]

rms_norm_gate creates an RMSNorm backward gate.

fn rms_norm_layer[T](ctx &autograd.Context[T], normalized_shape []int, config RMSNormConfig) types.Layer[T]

rms_norm_layer creates an RMSNorm layer. normalized_shape must match the trailing dimensions of each input handled by the layer.

fn selu_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

selu_forward_f32 uses Vulkan when opted in, then falls back to CPU.

fn selu_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

selu_forward_f32_try exposes the Vulkan SELU kernel when available.

fn selu_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

fn sigmoid_forward_cuda(x &vtl.Tensor[f64]) !&vtl.Tensor[f64]

sigmoid_forward_cuda returns error if CUDA is not enabled

fn sigmoid_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

sigmoid_forward_f32 uses Vulkan in-place sigmoid when opted in.

fn sigmoid_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

sigmoid_forward_f32_try exposes this operation as part of the public API.

fn sigmoid_forward_vulkan[T](x &vtl.Tensor[T]) !&vtl.Tensor[T]

sigmoid_forward_vulkan returns error if Vulkan is not enabled

fn sigmoid_forward_vulkan_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

sigmoid_forward_vulkan_f32 exposes this operation as part of the public API.

fn sigmoid_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

sigmoid_layer exposes this operation as part of the public API.

fn softmax_layer[T](ctx &autograd.Context[T], config SoftmaxLayerConfig) types.Layer[T]

softmax_layer exposes this operation as part of the public API.

fn softplus_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

softplus_forward_f32 uses Vulkan when opted in, then falls back to CPU.

fn softplus_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

softplus_forward_f32_try exposes the Vulkan Softplus kernel when available.

fn softplus_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

fn swish_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

swish_layer exposes this operation as part of the public API.

fn tanh_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

tanh_layer exposes this operation as part of the public API.

fn vulkan_linear_enabled() bool

vulkan_linear_enabled exposes this operation as part of the public API.

fn vulkan_tests_enabled() bool

vulkan_tests_enabled exposes this operation as part of the public API.

fn (g &AttentionGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AttentionGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &AvgPool2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AvgPool2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (g &BatchNorm1DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &BatchNorm1DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) forward_with_mode(input &autograd.Variable[T], training bool) !&autograd.Variable[T]

forward_with_mode uses batch statistics and updates running statistics in training mode, and uses the stored running statistics in evaluation mode.

fn (g &BatchNorm2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward computes gradients w.r.t. input, gamma and beta.

fn (g &BatchNorm2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache registers the gate on the result variable.

fn (layer &BatchNorm2DLayer[T]) output_shape() []int

output_shape returns the per-channel feature count.

fn (layer &BatchNorm2DLayer[T]) variables() []&autograd.Variable[T]

variables returns the affine parameters, or none when affine is disabled.

fn (layer &BatchNorm2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward runs the layer; the mode follows the input's gradient requirement, mirroring BatchNorm1D. Prefer forward_with_mode through Sequential, which propagates train()/eval().

fn (layer &BatchNorm2DLayer[T]) forward_with_mode(input &autograd.Variable[T], training bool) !&autograd.Variable[T]

forward_with_mode uses batch statistics and updates running statistics in training mode, and uses the stored running statistics in evaluation mode.

fn (g &BatchNorm3DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward computes gradients w.r.t. input, gamma and beta.

fn (g &BatchNorm3DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache registers the gate on the result variable.

fn (layer &BatchNorm3DLayer[T]) output_shape() []int

output_shape returns the per-channel feature count.

fn (layer &BatchNorm3DLayer[T]) variables() []&autograd.Variable[T]

variables returns the affine parameters, or none when affine is disabled.

fn (layer &BatchNorm3DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward runs the layer; the mode follows the input's gradient requirement, mirroring BatchNorm1D. Prefer forward_with_mode through Sequential, which propagates train()/eval().

fn (layer &BatchNorm3DLayer[T]) forward_with_mode(input &autograd.Variable[T], training bool) !&autograd.Variable[T]

forward_with_mode uses batch statistics and updates running statistics in training mode, and uses the stored running statistics in evaluation mode.

fn (l &Conv1DLayer[T]) output_shape() []int

output_shape returns [out_channels, output_length] excluding batch.

fn (l &Conv1DLayer[T]) variables() []&autograd.Variable[T]

variables returns the trainable weight and bias variables.

fn (l &Conv1DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward computes Conv1D and registers its reverse-mode derivative.

fn (g &Conv2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &Conv2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &DropoutLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &DropoutLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &DropoutLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &DropoutLayer[T]) forward_with_mode(input &autograd.Variable[T], training bool) !&autograd.Variable[T]

forward_with_mode applies inverted dropout during training and is an identity operation during evaluation.

fn (layer &EluLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &EluLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &EluLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &EmbeddingGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &EmbeddingGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &FlattenLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &FlattenLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &FlattenLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &GELULayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &GELULayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &GELULayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (l &GRULayer[T]) forward_with_state(input &autograd.Variable[T], h0 &autograd.Variable[T]) !(&autograd.Variable[T], &autograd.Variable[T])

forward_with_state runs the GRU from an explicit initial hidden state and returns both the per-timestep sequence and final hidden state. Both outputs participate in autograd, including gradients propagated back into h0.

fn (l &GRULayer[T]) forward_batch_first(input &autograd.Variable[T]) !&autograd.Variable[T]

forward_batch_first accepts and returns [batch, sequence, features] tensors, matching the common batch-first recurrent-layer convention. The recurrent kernel still operates on [sequence, batch, features]; transpose_op keeps gradients connected to the caller's layout.

fn (l &GRULayer[T]) forward_batch_first_with_state(input &autograd.Variable[T],
	h0 &autograd.Variable[T]) !(&autograd.Variable[T], &autograd.Variable[T])

forward_batch_first_with_state is the batch-first counterpart of forward_with_state. h0 and the final hidden state keep shape [batch, hidden].

fn (g &GlobalAvgPool2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &GlobalAvgPool2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (gate &GroupNormGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward returns input and present affine parameter gradients.

fn (gate &GroupNormGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache registers this gate with its input and trainable parameters.

fn (layer &GroupNormLayer[T]) output_shape() []int

output_shape returns the feature shape, excluding batch.

fn (layer &GroupNormLayer[T]) variables() []&autograd.Variable[T]

variables returns affine trainable parameters, if enabled.

fn (layer &GroupNormLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward applies GroupNorm and records its backward operation when needed.

fn (layer &HardSwishLayer[T]) output_shape() []int

fn (_ &HardSwishLayer[T]) variables() []&autograd.Variable[T]

fn (layer &HardSwishLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

fn (layer &InputLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &InputLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &InputLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (l &LSTMLayer[T]) forward_with_state(input &autograd.Variable[T],
	hidden0 &autograd.Variable[T], cell0 &autograd.Variable[T]) !(&autograd.Variable[T], &autograd.Variable[T], &autograd.Variable[T])

forward_with_state accepts initial hidden and cell states with shape [num_layers, batch, hidden_size], then returns the output sequence and both final states in the same stacked shape.

fn (g &LayerNormGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &LayerNormGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &LeakyReluLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &LeakyReluLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LeakyReluLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MishLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &MishLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MishLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (gate &RMSNormGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward returns input and optional per-element weight gradients.

fn (gate &RMSNormGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache registers the RMSNorm gate with its input and optional weight.

fn (layer &RMSNormLayer[T]) output_shape() []int

output_shape returns the normalized feature shape.

fn (layer &RMSNormLayer[T]) variables() []&autograd.Variable[T]

variables returns the learnable weight, when elementwise_affine is enabled.

fn (layer &RMSNormLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward normalizes trailing dimensions and records autograd when requested.

fn (layer &ReLULayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &ReLULayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &ReLULayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SeluLayer[T]) output_shape() []int

fn (_ &SeluLayer[T]) variables() []&autograd.Variable[T]

fn (layer &SeluLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

fn (layer &SigmoidLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &SigmoidLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SigmoidLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SoftplusLayer[T]) output_shape() []int

fn (_ &SoftplusLayer[T]) variables() []&autograd.Variable[T]

fn (layer &SoftplusLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

fn (layer &SwishLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &SwishLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SwishLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &TanhLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &TanhLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &TanhLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

struct AttentionGate[T] {
	input        &vtl.Tensor[T] = unsafe { nil }
	w_q          &vtl.Tensor[T] = unsafe { nil }
	w_k          &vtl.Tensor[T] = unsafe { nil }
	w_v          &vtl.Tensor[T] = unsafe { nil }
	w_o          &vtl.Tensor[T] = unsafe { nil }
	q            &vtl.Tensor[T] = unsafe { nil }
	k            &vtl.Tensor[T] = unsafe { nil }
	v            &vtl.Tensor[T] = unsafe { nil }
	attn_weights &vtl.Tensor[T] = unsafe { nil }
	merged       &vtl.Tensor[T] = unsafe { nil }
	num_heads    int
	head_dim     int
}

AttentionGate defines a public data structure for this module.

struct AveragePool2DLayer[T] {
	kernel      []int
	padding     []int
	stride      []int
	input_shape []int
}

AveragePool2DLayer applies 2D average pooling over a 4D input.

Input: [batch, channels, H, W] Output: [batch, channels, out_H, out_W]

Config options:- kernel — pool window size in (H, W) (default: determined by input_shape)

• padding — zero-border padding before pooling (default: [0,0])
• stride — pool stride in (H, W) (default: same as kernel)

struct AvgPool2DGate[T] {
	input   &vtl.Tensor[T] = unsafe { nil }
	kernel  []int
	padding []int
	stride  []int
}

AvgPool2DGate defines a public data structure for this module.

struct BatchNorm1DConfig {
pub:
	eps      f64  = 1e-5
	momentum f64  = 0.1
	affine   bool = true
}

BatchNorm1DConfig defines a public data structure for this module.

struct BatchNorm1DGate[T] {
	input    &vtl.Tensor[T] = unsafe { nil }
	gamma    &vtl.Tensor[T] = unsafe { nil }
	beta     &vtl.Tensor[T] = unsafe { nil }
	mean     &vtl.Tensor[T] = unsafe { nil }
	var_     &vtl.Tensor[T] = unsafe { nil }
	eps      f64
	training bool
}

BatchNorm1DGate defines a public data structure for this module.

struct BatchNorm1DLayer[T] {
	eps      f64
	momentum f64
pub mut:
	gamma               &autograd.Variable[T] = unsafe { nil }
	beta                &autograd.Variable[T] = unsafe { nil }
	running_mean        &vtl.Tensor[T]        = unsafe { nil }
	running_var         &vtl.Tensor[T]        = unsafe { nil }
	num_batches_tracked int
}

BatchNorm1DLayer defines a public data structure for this module.

struct BatchNorm2DConfig {
pub:
	eps      f64  = 1e-5
	momentum f64  = 0.1
	affine   bool = true
}

BatchNorm2DConfig configures a BatchNorm2D layer.

struct BatchNorm2DGate[T] {
	input    &vtl.Tensor[T] = unsafe { nil }
	gamma    &vtl.Tensor[T] = unsafe { nil }
	beta     &vtl.Tensor[T] = unsafe { nil }
	mean     &vtl.Tensor[T] = unsafe { nil }
	var_     &vtl.Tensor[T] = unsafe { nil }
	eps      f64
	training bool
}

BatchNorm2DGate is the autograd gate for BatchNorm2D.

struct BatchNorm2DLayer[T] {
	eps      f64
	momentum f64
	affine   bool
pub mut:
	gamma               &autograd.Variable[T] = unsafe { nil }
	beta                &autograd.Variable[T] = unsafe { nil }
	running_mean        &vtl.Tensor[T]        = unsafe { nil }
	running_var         &vtl.Tensor[T]        = unsafe { nil }
	num_batches_tracked int
}

BatchNorm2DLayer is the BatchNorm2D layer state.

struct BatchNorm3DConfig {
pub:
	eps      f64  = 1e-5
	momentum f64  = 0.1
	affine   bool = true
}

BatchNorm3DConfig configures a BatchNorm3D layer.

struct BatchNorm3DGate[T] {
	input    &vtl.Tensor[T] = unsafe { nil }
	gamma    &vtl.Tensor[T] = unsafe { nil }
	beta     &vtl.Tensor[T] = unsafe { nil }
	mean     &vtl.Tensor[T] = unsafe { nil }
	var_     &vtl.Tensor[T] = unsafe { nil }
	eps      f64
	training bool
}

BatchNorm3DGate is the autograd gate for BatchNorm3D.

struct BatchNorm3DLayer[T] {
	eps      f64
	momentum f64
	affine   bool
pub mut:
	gamma               &autograd.Variable[T] = unsafe { nil }
	beta                &autograd.Variable[T] = unsafe { nil }
	running_mean        &vtl.Tensor[T]        = unsafe { nil }
	running_var         &vtl.Tensor[T]        = unsafe { nil }
	num_batches_tracked int
}

BatchNorm3DLayer is the BatchNorm3D layer state.

struct Conv1DConfig {
pub:
	stride   int = 1
	padding  int
	dilation int = 1
	groups   int = 1
}

Conv1DConfig configures stride, padding, dilation, and channel groups.

struct Conv1DLayer[T] {
pub:
	in_channels  int
	out_channels int
	kernel_size  int
	input_length int
	config       Conv1DConfig
pub mut:
	weight &autograd.Variable[T] = unsafe { nil }
	bias   &autograd.Variable[T] = unsafe { nil }
}

Conv1DLayer applies grouped cross-correlation to [batch, channels, length].

struct Conv2DConfig {
pub:
	padding  []int = [0, 0]
	stride   []int = [1, 1]
	dilation []int = [1, 1]
	groups   int   = 1
}

Conv2DConfig defines a public data structure for this module.

struct Conv2DGate[T] {
	input       &vtl.Tensor[T] = unsafe { nil }
	weight      &vtl.Tensor[T] = unsafe { nil }
	bias        &vtl.Tensor[T] = unsafe { nil }
	kernel_size []int
	config      Conv2DConfig
}

Conv2DGate defines a public data structure for this module.

struct Conv2DLayer[T] {
pub:
	in_channels  int
	out_channels int
	kernel_size  []int
	config       Conv2DConfig
	// [C, H, W] from the previous layer when known (for output_shape / flatten).
	input_shape []int
pub mut:
	weight &autograd.Variable[T] = unsafe { nil }
	bias   &autograd.Variable[T] = unsafe { nil }
}

Conv2DLayer applies a 2D convolution over a 4D input tensor.

Input: [batch, in_channels, H, W] Output: [batch, out_channels, out_H, out_W]

Config options (via Conv2DConfig):- padding — zero-padding added to input borders (default: [0,0])

• stride — sampling stride in H and W dimensions (default: [1,1])
• dilation — spacing between kernel elements (default: [1,1])
• groups — split input channels into groups groups (default: 1)