nn.layers #

linear_forward_f32_use_vulkan exposes this operation as part of the public API.

fn attention_gate[T](input &vtl.Tensor[T], w_q &vtl.Tensor[T], w_k &vtl.Tensor[T], w_v &vtl.Tensor[T], w_o &vtl.Tensor[T], num_heads int, head_dim int) &AttentionGate[T]

attention_gate exposes this operation as part of the public API.

fn avgpool2d_gate[T](input &vtl.Tensor[T], kernel []int, padding []int, stride []int) &AvgPool2DGate[T]

avgpool2d_gate exposes this operation as part of the public API.

fn avgpool2d_layer[T](ctx &autograd.Context[T], input_shape []int, kernel []int, padding []int, stride []int) types.Layer[T]

avgpool2d_layer creates an AveragePool2DLayer.

fn batchnorm1d_gate[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], mean &vtl.Tensor[T], var_ &vtl.Tensor[T], eps f64) &BatchNorm1DGate[T]

batchnorm1d_gate exposes this operation as part of the public API.

fn batchnorm1d_layer[T](ctx &autograd.Context[T], num_features int, config BatchNorm1DConfig) types.Layer[T]

batchnorm1d_layer exposes this operation as part of the public API.

fn conv2d_gate[T](input &vtl.Tensor[T], weight &vtl.Tensor[T], bias &vtl.Tensor[T], kernel_size []int, config Conv2DConfig) &Conv2DGate[T]

conv2d_gate exposes this operation as part of the public API.

fn conv2d_layer[T](ctx &autograd.Context[T], in_ch int, out_ch int, kernel_size []int, config Conv2DConfig, input_shape []int) types.Layer[T]

conv2d_layer creates a Conv2DLayer.

fn cuda_linear_enabled() bool

cuda_linear_enabled exposes this operation as part of the public API.

fn cuda_tests_enabled() bool

cuda_tests_enabled exposes this operation as part of the public API.

fn dropout_layer[T](ctx &autograd.Context[T], output_shape []int, data DropoutLayerConfig) types.Layer[T]

dropout_layer exposes this operation as part of the public API.

fn elu_layer[T](ctx &autograd.Context[T], output_shape []int, data EluLayerConfig) types.Layer[T]

elu_layer exposes this operation as part of the public API.

fn embedding_gate[T](input &vtl.Tensor[T], weight &vtl.Tensor[T]) &EmbeddingGate[T]

embedding_gate exposes this operation as part of the public API.

fn embedding_layer[T](ctx &autograd.Context[T], vocab_size int, embedding_dim int) types.Layer[T]

embedding_layer creates an EmbeddingLayer.

fn flatten_layer[T](ctx &autograd.Context[T], shape []int) types.Layer[T]

flatten_layer exposes this operation as part of the public API.

fn gelu_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

gelu_layer exposes this operation as part of the public API.

fn global_avgpool2d_gate[T](input &vtl.Tensor[T]) &GlobalAvgPool2DGate[T]

global_avgpool2d_gate exposes this operation as part of the public API.

fn global_avgpool2d_layer[T](ctx &autograd.Context[T]) types.Layer[T]

global_avgpool2d_layer creates a GlobalAvgPool2DLayer.

fn input_layer[T](ctx &autograd.Context[T], shape []int) types.Layer[T]

input_layer exposes this operation as part of the public API.

fn layer_norm_layer[T](ctx &autograd.Context[T], normalized_shape []int, config LayerNormConfig) types.Layer[T]

layer_norm_layer creates a LayerNormLayer. layer_norm_layer creates a LayerNormLayer.

fn layernorm_gate[T](input &vtl.Tensor[T], gamma &vtl.Tensor[T], beta &vtl.Tensor[T], eps f64) &LayerNormGate[T]

layernorm_gate exposes this operation as part of the public API.

fn leaky_relu_layer[T](ctx &autograd.Context[T], output_shape []int, data LeakyReluLayerConfig) types.Layer[T]

leaky_relu_layer exposes this operation as part of the public API.

fn linear_bind_output_gpu(_result voidptr)

linear_bind_output_gpu exposes this operation as part of the public API.

fn linear_forward_cuda_f64(x &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64]) !&vtl.Tensor[f64]

linear_forward_cuda_f64 stub when not built with -d cuda.

fn linear_forward_f32(input &vtl.Tensor[f32], weights &vtl.Tensor[f32], bias &vtl.Tensor[f32]) !&vtl.Tensor[f32]

linear_forward_f32 uses Vulkan GEMM when VTL_USE_VULKAN=1 and -d vulkan, else CPU.

fn linear_forward_f32_use_vulkan() bool

fn linear_forward_f64(input &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64],
	input_gpu voidptr, session voidptr) !&vtl.Tensor[f64]

linear_forward_f64 may use CUDA when enabled at runtime (VTL_USE_CUDA=1, -d cuda). session is an opaque pointer to autograd_cuda.DeviceSession when CUDA is enabled.

fn linear_forward_f64_session(input &vtl.Tensor[f64], weights &vtl.Tensor[f64], bias &vtl.Tensor[f64],
	_input_gpu voidptr, session voidptr) !&vtl.Tensor[f64]

linear_forward_f64_session is a no-op without -d cuda.

fn linear_forward_f64_use_cuda() bool

fn linear_forward_vulkan[T](x &Tensor[T], weights &Tensor[T], bias &Tensor[T]) !&Tensor[T]

linear_forward_vulkan returns error if Vulkan is not enabled

fn linear_forward_vulkan_f32(x &vtl.Tensor[f32], weights &vtl.Tensor[f32], bias &vtl.Tensor[f32]) !&vtl.Tensor[f32]

linear_forward_vulkan_f32 stub when not built with -d vulkan.

fn linear_layer[T](ctx &autograd.Context[T], input_dim int, output_dim int) types.Layer[T]

linear_layer creates a LinearLayer.

fn linear_take_gpu_input(_input voidptr) voidptr

linear_take_gpu_input is a no-op without -d cuda (keeps f32 nn builds free of f64 GPU take).

fn lstm_layer[T](ctx &autograd.Context[T], input_size int, hidden_size int, num_layers int) types.Layer[T]

lstm_layer creates an LSTMLayer.

fn maxpool2d_layer[T](ctx &autograd.Context[T], input_shape []int, kernel []int, padding []int, stride []int) types.Layer[T]

maxpool2d_layer exposes this operation as part of the public API.

fn mish_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

mish_layer exposes this operation as part of the public API.

fn multihead_attention_layer[T](ctx &autograd.Context[T], embed_dim int, num_heads int) types.Layer[T]

multihead_attention_layer creates a MultiHeadAttentionLayer.

fn positional_encoding_layer[T](ctx &autograd.Context[T], embed_dim int, max_len int) !types.Layer[T]

positional_encoding_layer creates a PositionalEncodingLayer.

fn relu_forward_cuda(x &vtl.Tensor[f64]) !&vtl.Tensor[f64]

relu_forward_cuda returns error if CUDA is not enabled

fn relu_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

relu_forward_f32 uses Vulkan in-place ReLU when opted in (-d vulkan).

fn relu_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

relu_forward_f32_try exposes this operation as part of the public API.

fn relu_forward_vulkan[T](x &Tensor[T]) !&Tensor[T]

relu_forward_vulkan returns error if Vulkan is not enabled

fn relu_forward_vulkan_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

relu_forward_vulkan_f32 exposes this operation as part of the public API.

fn relu_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

relu_layer exposes this operation as part of the public API.

fn sigmoid_forward_cuda(x &vtl.Tensor[f64]) !&vtl.Tensor[f64]

sigmoid_forward_cuda returns error if CUDA is not enabled

fn sigmoid_forward_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

sigmoid_forward_f32 uses Vulkan in-place sigmoid when opted in.

fn sigmoid_forward_f32_try(x &vtl.Tensor[f32]) ?&vtl.Tensor[f32]

sigmoid_forward_f32_try exposes this operation as part of the public API.

fn sigmoid_forward_vulkan[T](x &Tensor[T]) !&Tensor[T]

sigmoid_forward_vulkan returns error if Vulkan is not enabled

fn sigmoid_forward_vulkan_f32(x &vtl.Tensor[f32]) !&vtl.Tensor[f32]

sigmoid_forward_vulkan_f32 exposes this operation as part of the public API.

fn sigmoid_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

sigmoid_layer exposes this operation as part of the public API.

fn softmax_layer[T](ctx &autograd.Context[T], config SoftmaxLayerConfig) types.Layer[T]

softmax_layer exposes this operation as part of the public API.

fn swish_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

swish_layer exposes this operation as part of the public API.

fn tanh_layer[T](ctx &autograd.Context[T], output_shape []int) types.Layer[T]

tanh_layer exposes this operation as part of the public API.

fn vulkan_linear_enabled() bool

vulkan_linear_enabled exposes this operation as part of the public API.

fn vulkan_tests_enabled() bool

vulkan_tests_enabled exposes this operation as part of the public API.

fn (g &AttentionGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AttentionGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &AveragePool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &AvgPool2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AvgPool2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (g &BatchNorm1DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &BatchNorm1DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &BatchNorm1DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &Conv2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &Conv2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &Conv2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &DropoutLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &DropoutLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &DropoutLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &EluLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &EluLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &EluLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &EmbeddingGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &EmbeddingGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &EmbeddingLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &FlattenLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &FlattenLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &FlattenLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &GELULayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &GELULayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &GELULayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &GlobalAvgPool2DGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &GlobalAvgPool2DGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &GlobalAvgPool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &InputLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &InputLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &InputLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (g &LayerNormGate[T]) backward(payload &autograd.Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &LayerNormGate[T]) cache(mut result autograd.Variable[T], args ...autograd.CacheParam) !

cache exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LayerNormLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &LeakyReluLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &LeakyReluLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LeakyReluLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &LinearLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MaxPool2DLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MishLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &MishLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MishLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &MultiHeadAttentionLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &PositionalEncodingLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &ReLULayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &ReLULayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &ReLULayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SigmoidLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &SigmoidLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SigmoidLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SoftmaxLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &SwishLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &SwishLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &SwishLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

fn (layer &TanhLayer[T]) output_shape() []int

output_shape exposes this operation as part of the public API.

fn (_ &TanhLayer[T]) variables() []&autograd.Variable[T]

variables exposes this operation as part of the public API.

fn (layer &TanhLayer[T]) forward(input &autograd.Variable[T]) !&autograd.Variable[T]

forward exposes this operation as part of the public API.

struct AttentionGate[T] {
	input     &vtl.Tensor[T] = unsafe { nil }
	w_q       &vtl.Tensor[T] = unsafe { nil }
	w_k       &vtl.Tensor[T] = unsafe { nil }
	w_v       &vtl.Tensor[T] = unsafe { nil }
	w_o       &vtl.Tensor[T] = unsafe { nil }
	num_heads int
	head_dim  int
}

AttentionGate defines a public data structure for this module.

struct AveragePool2DLayer[T] {
	kernel      []int
	padding     []int
	stride      []int
	input_shape []int
}

AveragePool2DLayer applies 2D average pooling over a 4D input.

Input: [batch, channels, H, W] Output: [batch, channels, out_H, out_W]

Config options:- kernel — pool window size in (H, W) (default: determined by input_shape)

• padding — zero-border padding before pooling (default: [0,0])
• stride — pool stride in (H, W) (default: same as kernel)

struct AvgPool2DGate[T] {
	input   &vtl.Tensor[T] = unsafe { nil }
	kernel  []int
	padding []int
	stride  []int
}

AvgPool2DGate defines a public data structure for this module.

struct BatchNorm1DConfig {
pub:
	eps      f64  = 1e-5
	momentum f64  = 0.1
	affine   bool = true
}

BatchNorm1DConfig defines a public data structure for this module.

struct BatchNorm1DGate[T] {
	input &vtl.Tensor[T] = unsafe { nil }
	gamma &vtl.Tensor[T] = unsafe { nil }
	beta  &vtl.Tensor[T] = unsafe { nil }
	mean  &vtl.Tensor[T] = unsafe { nil }
	var_  &vtl.Tensor[T] = unsafe { nil }
	eps   f64
}

BatchNorm1DGate defines a public data structure for this module.

struct BatchNorm1DLayer[T] {
	eps      f64
	momentum f64
pub mut:
	gamma               &autograd.Variable[T] = unsafe { nil }
	beta                &autograd.Variable[T] = unsafe { nil }
	running_mean        &vtl.Tensor[T]        = unsafe { nil }
	running_var         &vtl.Tensor[T]        = unsafe { nil }
	num_batches_tracked int
}

BatchNorm1DLayer defines a public data structure for this module.

struct Conv2DConfig {
pub:
	padding  []int = [0, 0]
	stride   []int = [1, 1]
	dilation []int = [1, 1]
	groups   int   = 1
}

Conv2DConfig defines a public data structure for this module.

struct Conv2DGate[T] {
	input       &vtl.Tensor[T] = unsafe { nil }
	weight      &vtl.Tensor[T] = unsafe { nil }
	bias        &vtl.Tensor[T] = unsafe { nil }
	kernel_size []int
	config      Conv2DConfig
}

Conv2DGate defines a public data structure for this module.

struct Conv2DLayer[T] {
pub:
	in_channels  int
	out_channels int
	kernel_size  []int
	config       Conv2DConfig
	// [C, H, W] from the previous layer when known (for output_shape / flatten).
	input_shape []int
pub mut:
	weight &autograd.Variable[T] = unsafe { nil }
	bias   &autograd.Variable[T] = unsafe { nil }
}

Conv2DLayer applies a 2D convolution over a 4D input tensor.

Input: [batch, in_channels, H, W] Output: [batch, out_channels, out_H, out_W]

Config options (via Conv2DConfig):- padding — zero-padding added to input borders (default: [0,0])

• stride — sampling stride in H and W dimensions (default: [1,1])
• dilation — spacing between kernel elements (default: [1,1])
• groups — split input channels into groups groups (default: 1)

struct DropoutLayer[T] {
	output_shape []int
	prob         f64
}

DropoutLayer is a dropout layer.

struct DropoutLayerConfig {
	prob f64 = 0.5
}

DropoutLayerConfig defines a public data structure for this module.

struct EluLayer[T] {
	output_shape []int
	alpha        f64
}

EluLayer is an activation layer that applies the element-wise function f(x) = x > 0 ? x : alpha * (exp(x) - 1)

struct EluLayerConfig {
	alpha f64 = 0.01
}

EluLayerConfig defines a public data structure for this module.

struct EmbeddingGate[T] {
	input  &vtl.Tensor[T] = unsafe { nil }
	weight &vtl.Tensor[T] = unsafe { nil }
}

EmbeddingGate defines a public data structure for this module.

struct EmbeddingLayer[T] {
	vocab_size    int
	embedding_dim int
pub mut:
	weight &autograd.Variable[T] = unsafe { nil }
}

EmbeddingLayer maps integer token indices to dense embedding vectors.

Input: [batch, seq_len] — integer indices in [0, vocab_size) Output: [batch, seq_len, embedding_dim]

Weight shape: [vocab_size, embedding_dim]

struct FlattenLayer[T] {
	shape []int
}

FlattenLayer is a layer

struct GELULayer[T] {
	output_shape []int
}

GELULayer applies the Gaussian Error Linear Unit (GELU) activation. GELU(x) = 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3))) Approximation used: 0.7978845608 * (x + 0.044715 * x^3) (lower-cost tanh approximation)

struct GlobalAvgPool2DGate[T] {
	input &vtl.Tensor[T] = unsafe { nil }
}

GlobalAvgPool2DGate defines a public data structure for this module.

struct GlobalAvgPool2DLayer[T] {}

GlobalAvgPool2DLayer applies global average pooling over spatial dimensions.

Input: [batch, channels, H, W] Output: [batch, channels, 1, 1] — one value per channel per sample

struct InputLayer[T] {
	shape []int
}

InputLayer is a layer that takes a single input tensor and returns the same tensor.

This layer is used as the first layer in a model.

struct LSTMLayer[T] {
pub mut:
	w_ih &autograd.Variable[T]
	w_hh &autograd.Variable[T]
	b_ih &autograd.Variable[T]
	b_hh &autograd.Variable[T]
pub:
	ctx         &autograd.Context[T]
	hidden_size int
	num_layers  int
}

LSTMLayer implements a Long Short-Term Memory layer.

Input: [batch, seq_len, input_size] Output: [batch, seq_len, hidden_size] (final hidden state per timestep)

Contains learnable weights w_ih, w_hh, b_ih, b_hh for the input-to-hidden and hidden-to-hidden transformations of the three gates (forget, input, cell).

struct LayerNormConfig {
pub:
	eps    f64  = 1e-5
	affine bool = true
}

LayerNormConfig defines a public data structure for this module.

struct LayerNormGate[T] {
	input &vtl.Tensor[T] = unsafe { nil }
	gamma &vtl.Tensor[T] = unsafe { nil }
	beta  &vtl.Tensor[T] = unsafe { nil }
	eps   f64
}

LayerNormGate defines a public data structure for this module.

struct LayerNormLayer[T] {
pub:
	normalized_shape []int
	eps              f64
pub mut:
	gamma &autograd.Variable[T] = unsafe { nil }
	beta  &autograd.Variable[T] = unsafe { nil }
}

LayerNormLayer defines a public data structure for this module.

struct LeakyReluLayer[T] {
	output_shape []int
	slope        f64
}

LeakyReluLayer is an activation layer that applies the leaky relu function to the input.

struct LeakyReluLayerConfig {
	slope f64 = 0.01
}

LeakyReluLayerConfig defines a public data structure for this module.

struct LinearLayer[T] {
pub:
	weights &autograd.Variable[T] = unsafe { nil }
	bias    &autograd.Variable[T] = unsafe { nil }
}

LinearLayer applies a linear transformation: y = x·Wᵀ + b

Input: [..., in_features] Output: [..., out_features]

Weights shape: [out_features, in_features] Bias shape: [1, out_features]

struct MaxPool2DLayer[T] {
	input_shape []int
	kernel      []int
	padding     []int
	stride      []int
}

MaxPool2DLayer is a layer that implements the maxpooling operation.

struct MishLayer[T] {
	output_shape []int
}

MishLayer applies the Mish activation: x * tanh(softplus(x)). Mish(x) = x * tanh(softplus(x)) where softplus(x) = log(1 + exp(x))

struct MultiHeadAttentionLayer[T] {
pub:
	embed_dim int
	num_heads int
	head_dim  int
pub mut:
	w_q &autograd.Variable[T] = unsafe { nil }
	w_k &autograd.Variable[T] = unsafe { nil }
	w_v &autograd.Variable[T] = unsafe { nil }
	w_o &autograd.Variable[T] = unsafe { nil }
}

MultiHeadAttentionLayer implements scaled dot-product multi-head attention.

Input: [batch, seq_len, embed_dim] Output: [batch, seq_len, embed_dim]

Computes attention across num_heads heads and projects back to embed_dim.

Config options (via constructor parameters):- embed_dim — model dimension (must be divisible by num_heads)

• num_heads — number of parallel attention heads

struct PositionalEncodingLayer[T] {
	max_len   int
	embed_dim int
pub mut:
	pe &vtl.Tensor[T] = unsafe { nil }
}

PositionalEncodingLayer adds fixed sinusoidal positional encodings to an embedding.

Does not contain learnable parameters. Encodings follow the original Transformer formulation (Attention is All You Need, §3.5).

Input: [batch, seq_len, embed_dim] Output: [batch, seq_len, embed_dim] — input + positional encoding

struct ReLULayer[T] {
	output_shape []int
}

ReLULayer is a layer that applies the rectified linear unit function element-wise.

struct SigmoidLayer[T] {
	output_shape []int
}

SigmoidLayer is a layer that applies the sigmoid function to its input.

struct SoftmaxLayer[T] {
	dim int
}

SoftmaxLayer applies softmax activation over the last dimension of the input. input shape: [..., n_classes] → output shape: [..., n_classes] Softmax output sums to 1 along the last dimension.

struct SoftmaxLayerConfig {
	dim int = -1 // dimension to apply softmax over; -1 means last dimension
}

SoftmaxLayerConfig defines a public data structure for this module.

struct SwishLayer[T] {
	output_shape []int
}

SwishLayer applies the Swish activation: x * sigmoid(x). Swish(x) = x * sigmoid(beta * x) — here we use beta=1 (standard Swish).

struct TanhLayer[T] {
	output_shape []int
}

TanhLayer is a layer that applies the tanh activation function to its input.

struct VulkanParams {}

VulkanParams defines a public data structure for this module.