VTL Autograd

vtl.autograd tracks tensor operations and propagates gradients through the recorded computation graph. Create related variables from the same context, build the forward computation, and call backprop() on its result.

import vtl
import vtl.autograd

context := autograd.ctx[f64]()
x := context.variable(vtl.from_1d([2.0])!)
mut y := x.multiply(x)!
y.backprop()!
println(x.grad) // [4.0]

Checking gradients numerically

autograd.grad_check compares the analytical gradient with central finite differences. It is useful when implementing a gate or debugging a model. The context must have no pending graph before calling it. The callback must build a fresh output from the supplied input each time it runs.

import vtl
import vtl.autograd

fn square(input &autograd.Variable[f64]) !&autograd.Variable[f64] {
    return input.multiply(input)!
}

context := autograd.ctx[f64]()
mut x := context.variable(vtl.from_1d([2.0])!)
matches := autograd.grad_check[f64](mut x, square, 1e-5, 1e-6)!
assert matches
assert x.grad.get_nth(0) == 4.0

The callback may return a tensor of any non-zero size. The checker compares the gradient of the sum of its output elements, matching backprop()'s all-ones initial gradient. eps must be positive. tolerance is a non-negative relative tolerance with a unit absolute scale near zero. The input value is restored after checking; its gradient is replaced by the analytical gradient.

Call the V compiler from the parent of the vtl clone:

systemd-run --user --scope --quiet --property=MemoryMax=768M --property=MemorySwapMax=0 --setenv=VJOBS=2 \
    --working-directory="$HOME/.vmodules" -- v test ./vtl/autograd_tests

backprop() seeds the output gradient with ones and accumulates gradients in its ancestors. It consumes the recorded graph, so run a fresh forward pass for each subsequent backward pass. Set requires_grad: false when creating a variable that should not receive gradients. See the autograd tutorial and backprop example. The module direction and planned work are summarized in the VTL roadmap.

The graph records operations that have an autograd gate; it does not promise to track every tensor API operation. Check the relevant operation and tests before depending on its gradient behavior. GPU autograd paths are backend-specific and experimental; see device memory notes.

Tracked operations

Variable methods record gates for elementwise add, subtract, multiply, divide, floating-point remainder, and power; exponential, logarithm, sine, cosine, tangent, absolute value, square root, hyperbolic tangent, and clamp; matrix multiplication; sum and mean reductions over the whole tensor, one axis, or multiple axes; reshape, permutation transpose, and concatenation. See variable_ops.v for method signatures and the gates reference for the backward-rule adapters.

Multiply and divide gates fuse both input-gradient calculations into one pass for same-shaped, row-major contiguous CPU tensors. Strided and broadcasted inputs use the general tensor-operation path. Division uses the derivative d(a / b) / db = -a / b²; see the gate backward benchmark for a focused comparison.

The implementation is grouped by rule in gates_basic.v (arithmetic), gates_pow.v, gates_exp.v, gates_trig.v, gates_unary.v, gates_blas.v, and gates_reduction.v. This list describes registered autograd rules, not every operation available on Tensor.

CUDA-specific variables and helpers are conditional builds. Accelerator coverage depends on the operation and backend; a GPU tensor does not imply that every forward operation or backward rule stays device-resident. See DEVICE_MEMORY.md for the current caveats.

fn abs_gate[T](a &Variable[T]) &AbsGate[T]

abs_gate exposes this operation as part of the public API.

fn add_gate[T]() &AddGate[T]

add_gate exposes this operation as part of the public API.

fn clamp_gate[T](min_val T, max_val T, a &vtl.Tensor[T]) &ClampGate[T]

clamp_gate exposes this operation as part of the public API.

fn concat_gate[T](axis int, splits []int) &ConcatGate[T]

concat_gate exposes this operation as part of the public API.

fn concatenate[T](variables []&Variable[T], data vtl.AxisData) !&Variable[T]

concatenate joins variables along an existing axis and routes gradients back to each input. All variables must have compatible shapes and share a context.

fn cos_gate[T](a &Variable[T]) &CosGate[T]

cos_gate exposes this operation as part of the public API.

fn ctx[T]() &Context[T]

Contexts can only be initialized as empty, and a generic type must be provided

fn cumprod_gate[T](input &vtl.Tensor[T], axis int) &CumprodGate[T]

cumprod_gate creates a cumulative-product gradient gate.

fn cumsum_gate[T](shape []int, axis int) &CumsumGate[T]

cumsum_gate creates a cumulative-sum gradient gate.

fn divide_gate[T](a &Variable[T], b &Variable[T]) &DivideGate[T]

divide_gate exposes this operation as part of the public API.

fn exp_gate[T](a &Variable[T]) &ExpGate[T]

exp_gate exposes this operation as part of the public API.

fn flip_gate[T](axes []int) &FlipGate[T]

flip_gate creates a gradient gate for the selected axes.

fn gate_backward[T](gate Gate[T], payload &Payload[T]) ![]&vtl.Tensor[T]

gate_backward is kept for backwards compatibility but now delegates directly through the Gate[T] interface instead of a manual match.

fn grad_check[T](mut input &Variable[T], forward fn (&Variable[T]) !&Variable[T], eps f64, tolerance f64) !bool

grad_check compares the analytical gradient with central finite differences for every element in input. The forward function must build a fresh result from input on each call. For non-scalar results, the checked objective is the sum of all output elements, matching backprop's all-ones initial gradient.

The input's gradient is replaced by the analytical gradient produced by this check. The input value is restored before the function returns.

fn log_gate[T](a &Variable[T]) &LogGate[T]

log_gate exposes this operation as part of the public API.

fn matmul_gate[T](a &Variable[T], b &Variable[T]) &MatMulGate[T]

matmul_gate exposes this operation as part of the public API.

fn mean_gate[T](shape []int, axis int, num_elems int) &MeanGate[T]

mean_gate exposes this operation as part of the public API.

fn multiply_gate[T](a &Variable[T], b &Variable[T]) &MultiplyGate[T]

multiply_gate exposes this operation as part of the public API.

fn nan_to_num_gate[T](finite_mask &vtl.Tensor[bool]) &NanToNumGate[T]

nan_to_num_gate exposes the saved finite-value mask to the autograd API.

fn node[T](gate voidptr, backward BackwardFn, parents []&Variable[T], payload &Payload[T], name string) &Node[T]

node

fn pad_gate[T](shape []int, pad_width [][]int, mode vtl.PadMode) &PadGate[T]

pad_gate creates backward metadata for a padded tensor.

fn payload[T](variable &Variable[T]) &Payload[T]

payload exposes this operation as part of the public API.

fn pow_gate[T](a &Variable[T], b &Variable[T]) &PowGate[T]

pow_gate exposes this operation as part of the public API.

fn register[T](name string, gate voidptr, backward BackwardFn, result &Variable[T], parents []&Variable[T]) !

register exposes this operation as part of the public API.

fn remainder_gate[T](a &Variable[T], b &Variable[T]) &RemainderGate[T]

remainder_gate creates a backward rule for floating-point remainder.

fn repeat_gate[T](shape []int, repeats int, axis int, flatten bool) &RepeatGate[T]

repeat_gate creates backward metadata for repeat or repeat_axis.

fn reshape_gate[T](orig_shape []int) &ReshapeGate[T]

reshape_gate exposes this operation as part of the public API.

fn roll_gate[T](input &vtl.Tensor[T], shift int, axes []int) !&RollGate[T]

roll_gate creates the inverse shift for a flattened or axis-wise roll.

fn roll_shifts_gate[T](input &vtl.Tensor[T], shifts []int, axes []int) !&RollGate[T]

roll_shifts_gate creates inverse shifts for paired axes and shift values.

fn rot90_gate[T](input &vtl.Tensor[T], k int, axes []int) !&Rot90Gate[T]

rot90_gate validates and normalizes the axes using the input tensor shape.

fn sin_gate[T](a &Variable[T]) &SinGate[T]

sin_gate exposes this operation as part of the public API.

fn sqrt_gate[T](a &Variable[T]) &SqrtGate[T]

sqrt_gate exposes this operation as part of the public API.

fn stack[T](variables []&Variable[T], data vtl.AxisData) !&Variable[T]

stack inserts a new axis and joins variables along it. Its backward pass removes that axis through each input's reshape gate.

fn subtract_gate[T]() &SubtractGate[T]

subtract_gate exposes this operation as part of the public API.

fn sum_gate[T](shape []int, axis int) &SumGate[T]

sum_gate exposes this operation as part of the public API.

fn tan_gate[T](a &Variable[T]) &TanGate[T]

tan_gate exposes this operation as part of the public API.

fn tanh_gate[T](cache &vtl.Tensor[T]) &TanhGate[T]

tanh_gate exposes this operation as part of the public API.

fn tensor_ptrs_to_voidptrs[T](tensors []&vtl.Tensor[T]) []voidptr

tensor_ptrs_to_voidptrs converts typed gradient tensor pointers at dispatch boundaries. The typed gate implementations remain the source of truth.

fn tile_gate[T](shape []int, reps []int) &TileGate[T]

tile_gate creates backward metadata for a tiled tensor.

fn transpose_gate[T](perm []int) &TransposeGate[T]

transpose_gate exposes this operation as part of the public API.

fn variable[T](context &Context[T], value &vtl.Tensor[T], data VariableData) &Variable[T]

variable

fn variable_gpu_activation_ptr[T](mut v Variable[T]) &voidptr

variable_gpu_activation_ptr exposes this operation as part of the public API.

fn variable_release_gpu_activation[T](mut v Variable[T])

variable_release_gpu_activation releases a stored GPU tensor handle (f64 CUDA).

fn variable_set_gpu_activation[T](mut v Variable[T], ptr voidptr)

variable_set_gpu_activation stores a GPU tensor handle (f64 CUDA).

fn variable_take_gpu_activation_input[T](mut v Variable[T]) voidptr

variable_take_gpu_activation_input moves the pending GPU activation out (f64 CUDA builds only).

interface CacheParam {}

CacheParam defines a public behavior contract for this module.

interface Gate[T] {
	backward(payload &Payload[T]) ![]&vtl.Tensor[T]
}

Gate is a generic interface for autograd operations. Any struct that implements backward for a given T satisfies Gate[T]. This allows both core math gates (AddGate, MatMulGate, …) and higher-level nn gates (LinearGate, ReLUGate, …) to participate in backpropagation without circular imports.

fn (g &AbsGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AbsGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &AddGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AddGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

type BackwardFn = fn (gate voidptr, payload voidptr) ![]voidptr

BackwardFn dispatches a stored gate without relying on a generic interface inside Node[T]. Both parameters and return values stay opaque at the graph storage boundary to avoid V generic interface specialization drift between Payload[f32] and Payload[f64].

fn (g &ClampGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ClampGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &ConcatGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ConcatGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (ctx &Context[T]) len() int

len exposes this operation as part of the public API.

fn (mut ctx Context[T]) push(node &Node[T])

push exposes this operation as part of the public API.

fn (ctx &Context[T]) last() !&Node[T]

last exposes this operation as part of the public API.

fn (mut ctx Context[T]) pop() !&Node[T]

pop exposes this operation as part of the public API.

fn (ctx &Context[T]) variable(value &vtl.Tensor[T], data ContextVariableData) &Variable[T]

variable exposes this operation as part of the public API.

fn (ctx &Context[T]) str() string

str exposes this operation as part of the public API.

fn (g &CosGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &CosGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &CumprodGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward computes product gradients without division, so zero inputs are handled correctly.

fn (g &CumprodGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the cumulative-product backward rule on its input variable.

fn (g &CumsumGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward computes the reverse cumulative sum of the output gradient.

fn (g &CumsumGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the cumulative-sum backward rule on its input variable.

fn (g &DivideGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &DivideGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &ExpGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ExpGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &FlipGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward flips the output gradient over the same axes as the forward pass.

fn (g &FlipGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for flip backpropagation.

fn (g &LogGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &LogGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MatMulGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MatMulGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MeanGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MeanGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MultiplyGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MultiplyGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &NanToNumGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward zeros gradients for values replaced by nan_to_num.

fn (g &NanToNumGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input with the graph for nan_to_num backward.

fn (g &PadGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward accumulates gradients according to the forward padding mode.

fn (g &PadGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for padding backpropagation.

fn (g &PowGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &PowGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &RemainderGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward returns da = grad and db = -grad * floor(a / b), reducing broadcast dimensions back to each original operand shape.

fn (g &RemainderGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the operands needed by the remainder backward rule.

fn (g &RepeatGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward groups duplicate positions and adds their gradients.

fn (g &RepeatGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for repeat backpropagation.

fn (g &ReshapeGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ReshapeGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &RollGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward rolls the output gradient by the inverse of the forward shift.

fn (g &RollGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for roll backpropagation.

fn (g &Rot90Gate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward applies the inverse rotation to the output gradient.

fn (g &Rot90Gate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for rot90 backpropagation.

fn (g &SinGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SinGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SqrtGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SqrtGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SubtractGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SubtractGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SumGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SumGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TanGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TanGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TanhGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TanhGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TileGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward interleaves repeat/source dimensions, sums the repeat dimensions, then restores the original shape.

fn (g &TileGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache registers the input variable for tile backpropagation.

fn (g &TransposeGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TransposeGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (v &Variable[T]) slice(idx ...[]int) !&Variable[T]

slice exposes this operation as part of the public API.

fn (v &Variable[T]) slice_hilo(idx1 []int, idx2 []int) !&Variable[T]

slice_hilo exposes this operation as part of the public API.

fn (v &Variable[T]) is_grad_needed() bool

is_grad_needed exposes this operation as part of the public API.

fn (v &Variable[T]) str() string

str exposes this operation as part of the public API.

fn (mut v Variable[T]) backprop() !

backprop Back propagates an operation along a computational graph. This operation will destroy the operational graph, populating the gradients for all variables that are predecessors of the Variable this is called on. Even if this is called on the first node in a graph, it will destroy all descendents of this variable stored by the Context

fn (v &Variable[T]) take(indices []int, axis int) !&Variable[T]

take gathers values along an axis and tracks the inverse scatter in backprop. Negative axes and indices count from the end.

fn (v &Variable[T]) take_with_mode(indices []int, axis int, mode vtl.TakeMode) !&Variable[T]

take_with_mode gathers values using NumPy's raise, wrap, or clip index mode and tracks gradients by scattering them to the normalized source indices.

fn (v &Variable[T]) take_nd(indices &vtl.Tensor[int], axis int) !&Variable[T]

take_nd gathers values along an axis using a multidimensional index tensor and accumulates gradients for repeated normalized indices.

fn (v &Variable[T]) take_nd_with_mode(indices &vtl.Tensor[int], axis int, mode vtl.TakeMode) !&Variable[T]

take_nd_with_mode gathers with NumPy's raise, wrap, or clip index handling and tracks the inverse scatter in backpropagation.

fn (v &Variable[T]) take_flat(indices &vtl.Tensor[int]) !&Variable[T]

take_flat gathers from row-major logical flattening and accumulates gradients for repeated flat indices.

fn (v &Variable[T]) take_flat_with_mode(indices &vtl.Tensor[int], mode vtl.TakeMode) !&Variable[T]

take_flat_with_mode gathers from logical row-major order using NumPy's raise, wrap, or clip mode and tracks the inverse scatter in backpropagation.

fn (v &Variable[T]) boolean_index(mask &vtl.Tensor[bool]) !&Variable[T]

boolean_index selects values under a leading-dimension mask and propagates gradients back to the selected source coordinates.

fn (v &Variable[T]) masked_select(mask &vtl.Tensor[bool]) !&Variable[T]

masked_select selects broadcast-mask values in row-major logical order and scatters gradients back to their source coordinates.

fn (v &Variable[T]) masked_fill(mask &vtl.Tensor[bool], value T) !&Variable[T]

masked_fill replaces mask-selected values and blocks their input gradients.

fn (v &Variable[T]) masked_scatter(mask &vtl.Tensor[bool], source &Variable[T]) !&Variable[T]

masked_scatter copies source values to broadcast-mask positions and tracks gradients for the original input and the source prefix used in the forward pass.

fn (v &Variable[T]) take_along_axis(indices &vtl.Tensor[int], axis int) !&Variable[T]

take_along_axis gathers values and tracks the inverse scatter in backprop. Repeated indices accumulate their gradients; non-axis broadcast dimensions are reduced into the corresponding source coordinates.

fn (v &Variable[T]) scatter_add(indices &vtl.Tensor[int], updates &Variable[T], axis int) !&Variable[T]

scatter_add adds updates at indexed positions and tracks gradients for both the source tensor and update values.

fn (v &Variable[T]) put_along_axis(indices &vtl.Tensor[int], updates &Variable[T], axis int) !&Variable[T]

put_along_axis writes updates into a copy and tracks gradients. When multiple updates target the same position, only the last update receives its gradient.

fn (v &Variable[T]) put(indices &vtl.Tensor[int], updates &Variable[T]) !&Variable[T]

put replaces row-major flat positions and tracks gradients for the source and updates. Short update tensors repeat in the same order as Tensor.put.

fn (v &Variable[T]) put_with_mode(indices &vtl.Tensor[int], updates &Variable[T], mode vtl.PutMode) !&Variable[T]

put_with_mode is the differentiable counterpart of Tensor.put_with_mode.

fn (v &Variable[T]) putmask(mask &vtl.Tensor[bool], updates &Variable[T]) !&Variable[T]

putmask writes values into a copy at true mask positions and propagates gradients to untouched input positions and the repeated update values.

fn (v &Variable[T]) sum() !&Variable[T]

sum reduces every tensor element into a one-element tensor and records the broadcast backward rule when this variable requires gradients.

fn (v &Variable[T]) mean() !&Variable[T]

mean reduces every tensor element into a one-element tensor and distributes the backward gradient uniformly over the input elements.

fn (v &Variable[T]) sum_along_axis(axis int, keepdims bool) !&Variable[T]

sum_along_axis reduces one axis. Set keepdims to retain it with length one.

fn (v &Variable[T]) cumsum(axis int) !&Variable[T]

cumsum applies a cumulative sum along one axis and records its backward rule when this variable requires gradients.

fn (v &Variable[T]) cumprod(axis int) !&Variable[T]

cumprod applies a cumulative product along one axis and records its backward rule when this variable requires gradients.

fn (v &Variable[T]) cumsum_flat() !&Variable[T]

cumsum_flat computes a cumulative sum over logical row-major order and returns a one-dimensional variable.

fn (v &Variable[T]) cumprod_flat() !&Variable[T]

cumprod_flat computes a cumulative product over logical row-major order and returns a one-dimensional variable.

fn (v &Variable[T]) mean_along_axis(axis int, keepdims bool) !&Variable[T]

mean_along_axis computes one-axis means. Set keepdims to retain the reduced axis with length one.

fn (v &Variable[T]) sum_along_axes(axes []int, keepdims bool) !&Variable[T]

sum_along_axes reduces several axes. Negative axes are accepted; an empty axes list returns an elementwise copy. Set keepdims to retain reduced axes.

fn (v &Variable[T]) mean_along_axes(axes []int, keepdims bool) !&Variable[T]

mean_along_axes computes means over several axes and propagates gradients. Negative axes are accepted; an empty axes list returns an elementwise copy.

fn (v &Variable[T]) add(other &Variable[T]) !&Variable[T]

add Adds two variables together.

fn (v &Variable[T]) subtract(other &Variable[T]) !&Variable[T]

subtract Subtracts two variables.

fn (v &Variable[T]) multiply(other &Variable[T]) !&Variable[T]

multiply Multiplies two variables.

fn (v &Variable[T]) divide(other &Variable[T]) !&Variable[T]

divide Divides two variables.

fn (v &Variable[T]) remainder(other &Variable[T]) !&Variable[T]

remainder applies NumPy/PyTorch-style floating-point remainder and records its piecewise derivative. At discontinuities the divisor derivative uses floor(a / b), matching PyTorch's defined backward convention.

fn (v &Variable[T]) pow(other &Variable[T]) !&Variable[T]

pow raises a variable to a power.

fn (v &Variable[T]) exp() !&Variable[T]

exp Exponentiates a variable.

fn (v &Variable[T]) matmul(other &Variable[T]) !&Variable[T]

matmul Multiplies two matrices.

fn (v &Variable[T]) sin() !&Variable[T]

sin Sine of a variable.

fn (v &Variable[T]) cos() !&Variable[T]

cos Cosine of a variable.

fn (v &Variable[T]) tan() !&Variable[T]

tan Tan of a variable.

fn (v &Variable[T]) log() !&Variable[T]

log computes the natural logarithm of the variable element-wise. Backward: grad * (1 / x)

Note: inputs must be positive; behaviour for x <= 0 is undefined.

x := ctx.variable(vtl.from_1d[f64]([1.0, math.e, math.exp(2.0)]))
y := x.log[f64]()!
// y.value ≈ [0.0, 1.0, 2.0]

fn (v &Variable[T]) abs_op() !&Variable[T]

abs_op computes the absolute value of the variable element-wise. Backward: grad * sign(x) (0 at x=0)

Named abs_op (not abs) to avoid collision with the built-in abs method on Tensor[T] which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([-3.0, 0.0, 4.0]))
y := x.abs_op[f64]()!
// y.value = [3.0, 0.0, 4.0]

fn (v &Variable[T]) sqrt_op() !&Variable[T]

sqrt_op computes the element-wise square root of the variable. Backward: grad * (1 / (2 * sqrt(x)))

Note: inputs must be non-negative.

Named sqrt_op (not sqrt) to avoid collision with Tensor.sqrt which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([1.0, 4.0, 9.0]))
y := x.sqrt_op[f64]()!
// y.value = [1.0, 2.0, 3.0]

fn (v &Variable[T]) tanh_op() !&Variable[T]

tanh_op computes the element-wise hyperbolic tangent of the variable. Backward: grad * (1 - tanh²(x))

Named tanh_op (not tanh) to avoid collision with Tensor.tanh which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([0.0, 1.0, -1.0]))
y := x.tanh_op[f64]()!
// y.value ≈ [0.0, 0.762, -0.762]

fn (v &Variable[T]) clamp(min_val T, max_val T) !&Variable[T]

clamp clips the variable element-wise to the range [min_val, max_val]. Backward: grad is passed through where min_val < x < max_val, zero otherwise.

x := ctx.variable(vtl.from_1d[f64]([-2.0, 0.5, 3.0]))
y := x.clamp[f64](-1.0, 1.0)!
// y.value = [-1.0, 0.5, 1.0]

fn (v &Variable[T]) nan_to_num(params vtl.NanToNumConfig) !&Variable[T]

nan_to_num replaces NaN and infinite values and passes gradients through finite inputs only. Its forward and backward behavior follows PyTorch.

fn (v &Variable[T]) reshape(new_shape []int) !&Variable[T]

reshape returns a new variable with the same data but a different shape. The total number of elements must be preserved. Backward: gradient is reshaped back to the original shape.

x := ctx.variable(vtl.from_array[f64]([1.0, 2.0, 3.0, 4.0], [2, 2]))
y := x.reshape[f64]([4])!
// y.value.shape = [4]

fn (v &Variable[T]) transpose_op(perm []int) !&Variable[T]

transpose_op permutes the axes of the variable according to perm. perm must be a permutation of [0, 1, ..., ndim-1]. Backward: gradient is transposed with the inverse permutation.

Named transpose_op (not transpose) to avoid collision with Tensor.transpose.

x := ctx.variable(vtl.from_array[f64]([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], [2, 3]))
y := x.transpose_op[f64]([1, 0])!
// y.value.shape = [3, 2]

fn (v &Variable[T]) rot90() !&Variable[T]

rot90 rotates the first two dimensions counterclockwise and propagates gradients through the inverse rotation.

fn (v &Variable[T]) rot90_k(k int) !&Variable[T]

rot90_k rotates the first two dimensions counterclockwise by k quarter turns.

fn (v &Variable[T]) rot90_axes(k int, axes []int) !&Variable[T]

rot90_axes rotates in a selected plane and records its inverse for autograd.

fn (v &Variable[T]) flip(axes ...int) !&Variable[T]

flip reverses selected axes (all axes by default) and routes gradients through the same flip, which is its own inverse.

fn (v &Variable[T]) repeat(repeats int) !&Variable[T]

repeat flattens the input and repeats every value; backward sums gradients from all copies of each source value.

fn (v &Variable[T]) repeat_axis(repeats int, axis int) !&Variable[T]

repeat_axis repeats values along one axis while preserving autograd.

fn (v &Variable[T]) tile(reps []int) !&Variable[T]

tile repeats whole blocks according to NumPy's right-aligned repetition dimensions and sums gradients from each block during backpropagation.

fn (v &Variable[T]) pad(pad_width [][]int, mode vtl.PadMode, constant_value T) !&Variable[T]

pad adds values around each axis and accumulates gradients from padded positions according to the selected padding mode.

fn (v &Variable[T]) roll(shift int, axes ...int) !&Variable[T]

roll circularly shifts values, flattening by default like NumPy. Optional axes apply the same shift to each selected axis and preserve autograd.

fn (v &Variable[T]) roll_shifts(shifts []int, axes []int) !&Variable[T]

roll_shifts applies paired shifts and axes while preserving autograd.

struct AbsGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

AbsGate implements abs(x) element-wise. backward: grad * sign(x)

struct AddGate[T] {}

AddGate defines a public data structure for this module.

struct ClampGate[T] {
pub:
	min_val T
	max_val T
	a       &vtl.Tensor[T] = unsafe { nil }
}

ClampGate implements clamp(x, min, max) element-wise. backward: grad where x is within [min, max], 0 otherwise

struct ConcatGate[T] {
pub:
	axis   int
	splits []int // size of each input along the concat axis
}

ConcatGate concatenates multiple tensors along an axis. backward: split gradient back into original inputs

struct Context[T] {
pub mut:
	// A list of all variables present in an operation.
	// This list can contain duplicates
	nodes []&Node[T]
	// If no_grad is set to true, operations will not
	// be cached, and backpropagation will not be possible
	no_grad bool
	// Reusable CUDA staging for f64 forwards (nil for other T); see autograd_cuda.DeviceSession.
	device_session voidptr = unsafe { nil }
}

Context defines a public data structure for this module.

struct ContextVariableData {
pub:
	requires_grad bool = true
}

ContextVariableData defines a public data structure for this module.

struct CosGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

CosGate defines a public data structure for this module.

struct CumprodGate[T] {
pub:
	input &vtl.Tensor[T] = unsafe { nil }
	axis  int
}

CumprodGate stores the input values needed to differentiate cumulative products, including slices that contain zero values.

struct CumsumGate[T] {
pub:
	shape []int
	axis  int
}

CumsumGate reverses the cumulative-sum operation by accumulating output gradients from the end of each axis slice toward its beginning.

struct DivideGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

DivideGate defines a public data structure for this module.

struct ExpGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

ExpGate defines a public data structure for this module.

struct FlipGate[T] {
pub:
	axes []int
}

FlipGate records the axes to reverse during the backward pass. Flipping is its own inverse, so the same axes map output gradients back to the input.

struct LogGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

LogGate implements log(x) element-wise. backward: grad * (1/x)

struct MatMulGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

MatMulGate defines a public data structure for this module.

struct MeanGate[T] {
pub:
	shape     []int
	axis      int
	num_elems int
}

MeanGate implements mean reduction. backward: grad broadcast to input shape / num_elements

struct MultiplyGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

MultiplyGate defines a public data structure for this module.

struct NanToNumGate[T] {
pub:
	finite_mask &vtl.Tensor[bool] = unsafe { nil }
}

NanToNumGate passes gradients only through finite input values, matching PyTorch's nan_to_num derivative. The mask is captured during the forward pass so later mutation of the source tensor cannot change the backward result.

struct Node[T] {
pub:
	// Opaque pointer to the concrete gate instance for this node.
	gate voidptr
	// Callback that knows how to cast and run the gate.
	backward BackwardFn = unsafe { nil }
pub mut:
	// The variables that created this node
	parents []&Variable[T]
	// Wrapper around a Tensor, contains operation data
	payload &Payload[T] = unsafe { nil }
	// Debug use only, contains a name for a node
	name string
}

Node defines a public data structure for this module.

struct PadGate[T] {
pub:
	shape     []int
	pad_width [][]int
	mode      vtl.PadMode
}

PadGate stores the source shape and padding contract needed to route each padded output gradient back to its source position.

struct Payload[T] {
pub:
	// Contents of the paylod
	variable &Variable[T] = unsafe { nil }
}

Payload defines a public data structure for this module.

struct PowGate[T] {
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

PowGate defines a public data structure for this module.

struct RemainderGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

RemainderGate defines the backward rule for floating-point remainder.

struct RepeatGate[T] {
pub:
	shape   []int
	repeats int
	axis    int
	flatten bool
}

RepeatGate stores the layout needed to sum gradients from repeated copies.

struct ReshapeGate[T] {
pub:
	orig_shape []int
}

ReshapeGate stores the original shape for backward pass. backward: grad reshaped back to original shape

struct RollGate[T] {
pub:
	flat        bool
	flat_shift  int
	axes        []int
	axis_shifts []int
}

RollGate stores the inverse permutation needed by the roll backward pass.

struct Rot90Gate[T] {
pub:
	k    int
	axes []int
}

Rot90Gate stores the plane and turn count needed to invert a 90-degree rotation during backpropagation.

struct SinGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

SinGate defines a public data structure for this module.

struct SqrtGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

SqrtGate implements sqrt(x) element-wise. backward: grad * (1 / (2 * sqrt(x)))

struct SubtractGate[T] {}

SubtractGate defines a public data structure for this module.

struct SumGate[T] {
pub:
	shape []int
	axis  int
}

SumGate implements sum reduction. backward: grad broadcast back to input shape

struct TanGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

TanGate defines a public data structure for this module.

struct TanhGate[T] {
pub:
	cache &vtl.Tensor[T] = unsafe { nil }
}

TanhGate implements tanh(x) element-wise. backward: grad * (1 - tanh(x)^2) = grad * (1 - cached^2)

struct TileGate[T] {
pub:
	shape []int
	reps  []int
}

TileGate records the source shape and block repetitions needed to sum tiled output gradients back into the original tensor.

struct TransposeGate[T] {
pub:
	perm  []int
	iperm []int
}

TransposeGate stores the permutation for backward. backward: grad transposed back with inverse permutation

struct Variable[T] {
pub mut:
	// The value of the Variable.  This should not be edited outside
	// of Variable operations, as other edits will not be tracked
	// and will lead to incorrect results
	value &vtl.Tensor[T] = unsafe { nil }
	// The graph the variable is associated with.  This is a reference,
	// as a variable does not own its context
	context &Context[T] = unsafe { nil }
	// The gradient of the Variable.  This is set as a reference to
	// the value of a Variable unless `backprop` has been called, in
	// which case all related Variables will have their gradient
	// updated correctly
	grad &vtl.Tensor[T] = unsafe { nil }
	// If set to true, this variable will track its operations,
	// otherwise it will act similar to a vtl.Tensor, only calculating
	// forward operations
	requires_grad bool
	// Phase 2: optional forward-only GPU activation (`&vtl.CudaTensor[f64]` when `-d cuda`).
	gpu_activation voidptr = unsafe { nil }
mut:
	// True until an operation consumes the fresh zero gradient buffer.
	grad_buffer_fresh bool
}

Variable defines a public data structure for this module.

struct VariableData {
	requires_grad bool = true
}

VariableData defines a public data structure for this module.