autograd #

Payload defines a public data structure for this module.

fn abs_gate[T](a &Variable[T]) &AbsGate[T]

abs_gate exposes this operation as part of the public API.

fn add_gate[T]() &AddGate[T]

add_gate exposes this operation as part of the public API.

fn clamp_gate[T](min_val T, max_val T, a &vtl.Tensor[T]) &ClampGate[T]

clamp_gate exposes this operation as part of the public API.

fn concat_gate[T](axis int, splits []int) &ConcatGate[T]

concat_gate exposes this operation as part of the public API.

fn cos_gate[T](a &Variable[T]) &CosGate[T]

cos_gate exposes this operation as part of the public API.

fn ctx[T]() &Context[T]

Contexts can only be initialized as empty, and a generic type must be provided

fn divide_gate[T](a &Variable[T], b &Variable[T]) &DivideGate[T]

divide_gate exposes this operation as part of the public API.

fn exp_gate[T](a &Variable[T]) &ExpGate[T]

exp_gate exposes this operation as part of the public API.

fn gate_backward[T](gate Gate[T], payload &Payload[T]) ![]&vtl.Tensor[T]

gate_backward is kept for backwards compatibility but now delegates directly through the Gate[T] interface instead of a manual match.

fn log_gate[T](a &Variable[T]) &LogGate[T]

log_gate exposes this operation as part of the public API.

fn matmul_gate[T](a &Variable[T], b &Variable[T]) &MatMulGate[T]

matmul_gate exposes this operation as part of the public API.

fn mean_gate[T](shape []int, axis int, num_elems int) &MeanGate[T]

mean_gate exposes this operation as part of the public API.

fn multiply_gate[T](a &Variable[T], b &Variable[T]) &MultiplyGate[T]

multiply_gate exposes this operation as part of the public API.

fn node[T](gate voidptr, backward BackwardFn, parents []&Variable[T], payload &Payload[T], name string) &Node[T]

node

fn payload[T](variable &Variable[T]) &Payload[T]

payload exposes this operation as part of the public API.

fn pow_gate[T](a &Variable[T], b &Variable[T]) &PowGate[T]

pow_gate exposes this operation as part of the public API.

fn register[T](name string, gate voidptr, backward BackwardFn, result &Variable[T], parents []&Variable[T]) !

register exposes this operation as part of the public API.

fn reshape_gate[T](orig_shape []int) &ReshapeGate[T]

reshape_gate exposes this operation as part of the public API.

fn sin_gate[T](a &Variable[T]) &SinGate[T]

sin_gate exposes this operation as part of the public API.

fn sqrt_gate[T](a &Variable[T]) &SqrtGate[T]

sqrt_gate exposes this operation as part of the public API.

fn subtract_gate[T]() &SubtractGate[T]

subtract_gate exposes this operation as part of the public API.

fn sum_gate[T](shape []int, axis int) &SumGate[T]

sum_gate exposes this operation as part of the public API.

fn tan_gate[T](a &Variable[T]) &TanGate[T]

tan_gate exposes this operation as part of the public API.

fn tanh_gate[T](cache &vtl.Tensor[T]) &TanhGate[T]

tanh_gate exposes this operation as part of the public API.

fn tensor_ptrs_to_voidptrs[T](tensors []&vtl.Tensor[T]) []voidptr

tensor_ptrs_to_voidptrs converts typed gradient tensor pointers at dispatch boundaries. The typed gate implementations remain the source of truth.

fn transpose_gate[T](perm []int) &TransposeGate[T]

transpose_gate exposes this operation as part of the public API.

fn variable[T](context &Context[T], value &vtl.Tensor[T], data VariableData) &Variable[T]

variable

fn variable_gpu_activation_ptr[T](mut v Variable[T]) &voidptr

variable_gpu_activation_ptr exposes this operation as part of the public API.

fn variable_release_gpu_activation[T](mut v Variable[T])

variable_release_gpu_activation releases a stored GPU tensor handle (f64 CUDA).

fn variable_set_gpu_activation[T](mut v Variable[T], ptr voidptr)

variable_set_gpu_activation stores a GPU tensor handle (f64 CUDA).

fn variable_take_gpu_activation_input[T](mut v Variable[T]) voidptr

variable_take_gpu_activation_input moves the pending GPU activation out (f64 CUDA builds only).

interface CacheParam {}

CacheParam defines a public behavior contract for this module.

interface Gate[T] {
	backward(payload &Payload[T]) ![]&vtl.Tensor[T]
}

Gate is a generic interface for autograd operations. Any struct that implements backward for a given T satisfies Gate[T]. This allows both core math gates (AddGate, MatMulGate, …) and higher-level nn gates (LinearGate, ReLUGate, …) to participate in backpropagation without circular imports.

fn (g &AbsGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AbsGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &AddGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &AddGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

type BackwardFn = fn (gate voidptr, payload voidptr) ![]voidptr

BackwardFn dispatches a stored gate without relying on a generic interface inside Node[T]. Both parameters and return values stay opaque at the graph storage boundary to avoid V generic interface specialization drift between Payload[f32] and Payload[f64].

fn (g &ClampGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ClampGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &ConcatGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ConcatGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (ctx &Context[T]) len() int

len exposes this operation as part of the public API.

fn (mut ctx Context[T]) push(node &Node[T])

push exposes this operation as part of the public API.

fn (ctx &Context[T]) last() !&Node[T]

last exposes this operation as part of the public API.

fn (mut ctx Context[T]) pop() !&Node[T]

pop exposes this operation as part of the public API.

fn (ctx &Context[T]) variable(value &vtl.Tensor[T], data ContextVariableData) &Variable[T]

variable exposes this operation as part of the public API.

fn (ctx &Context[T]) str() string

str exposes this operation as part of the public API.

fn (g &CosGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &CosGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &DivideGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &DivideGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &ExpGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ExpGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &LogGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &LogGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MatMulGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MatMulGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MeanGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MeanGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &MultiplyGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &MultiplyGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &PowGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &PowGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &ReshapeGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &ReshapeGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SinGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SinGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SqrtGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SqrtGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SubtractGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SubtractGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &SumGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &SumGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TanGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TanGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TanhGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TanhGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (g &TransposeGate[T]) backward(payload &Payload[T]) ![]&vtl.Tensor[T]

backward exposes this operation as part of the public API.

fn (g &TransposeGate[T]) cache(mut result Variable[T], args ...CacheParam) !

cache exposes this operation as part of the public API.

fn (v &Variable[T]) abs_op() !&Variable[T]

abs_op computes the absolute value of the variable element-wise. Backward: grad * sign(x) (0 at x=0)

Named abs_op (not abs) to avoid collision with the built-in abs method on Tensor[T] which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([-3.0, 0.0, 4.0]))
y := x.abs_op[f64]()!
// y.value = [3.0, 0.0, 4.0]

fn (v &Variable[T]) add(other &Variable[T]) !&Variable[T]

add Adds two variables together.

fn (mut v Variable[T]) backprop() !

backprop Back propagates an operation along a computational graph. This operation will destroy the operational graph, populating the gradients for all variables that are predecessors of the Variable this is called on. Even if this is called on the first node in a graph, it will destroy all descendents of this variable stored by the Context

fn (v &Variable[T]) clamp(min_val T, max_val T) !&Variable[T]

clamp clips the variable element-wise to the range [min_val, max_val]. Backward: grad is passed through where min_val < x < max_val, zero otherwise.

x := ctx.variable(vtl.from_1d[f64]([-2.0, 0.5, 3.0]))
y := x.clamp[f64](-1.0, 1.0)!
// y.value = [-1.0, 0.5, 1.0]

fn (v &Variable[T]) cos() !&Variable[T]

cos Cosine of a variable.

fn (v &Variable[T]) divide(other &Variable[T]) !&Variable[T]

divide Divides two variables.

fn (v &Variable[T]) exp() !&Variable[T]

exp Exponentiates a variable.

fn (v &Variable[T]) is_grad_needed() bool

is_grad_needed exposes this operation as part of the public API.

fn (v &Variable[T]) log() !&Variable[T]

log computes the natural logarithm of the variable element-wise. Backward: grad * (1 / x)

Note: inputs must be positive; behaviour for x <= 0 is undefined.

x := ctx.variable(vtl.from_1d[f64]([1.0, math.e, math.exp(2.0)]))
y := x.log[f64]()!
// y.value ≈ [0.0, 1.0, 2.0]

fn (v &Variable[T]) matmul(other &Variable[T]) !&Variable[T]

matmul Multiplies two matrices.

fn (v &Variable[T]) multiply(other &Variable[T]) !&Variable[T]

multiply Multiplies two variables.

fn (v &Variable[T]) pow(other &Variable[T]) !&Variable[T]

pow raises a variable to a power.

fn (v &Variable[T]) reshape(new_shape []int) !&Variable[T]

reshape returns a new variable with the same data but a different shape. The total number of elements must be preserved. Backward: gradient is reshaped back to the original shape.

x := ctx.variable(vtl.from_array[f64]([1.0, 2.0, 3.0, 4.0], [2, 2]))
y := x.reshape[f64]([4])!
// y.value.shape = [4]

fn (v &Variable[T]) sin() !&Variable[T]

sin Sine of a variable.

fn (v &Variable[T]) slice(idx ...[]int) !&Variable[T]

slice exposes this operation as part of the public API.

fn (v &Variable[T]) slice_hilo(idx1 []int, idx2 []int) !&Variable[T]

slice_hilo exposes this operation as part of the public API.

fn (v &Variable[T]) sqrt_op() !&Variable[T]

sqrt_op computes the element-wise square root of the variable. Backward: grad * (1 / (2 * sqrt(x)))

Note: inputs must be non-negative.

Named sqrt_op (not sqrt) to avoid collision with Tensor.sqrt which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([1.0, 4.0, 9.0]))
y := x.sqrt_op[f64]()!
// y.value = [1.0, 2.0, 3.0]

fn (v &Variable[T]) str() string

str exposes this operation as part of the public API.

fn (v &Variable[T]) subtract(other &Variable[T]) !&Variable[T]

subtract Subtracts two variables.

fn (v &Variable[T]) tan() !&Variable[T]

tan Tan of a variable.

fn (v &Variable[T]) tanh_op() !&Variable[T]

tanh_op computes the element-wise hyperbolic tangent of the variable. Backward: grad * (1 - tanh²(x))

Named tanh_op (not tanh) to avoid collision with Tensor.tanh which does not participate in the autograd graph.

x := ctx.variable(vtl.from_1d[f64]([0.0, 1.0, -1.0]))
y := x.tanh_op[f64]()!
// y.value ≈ [0.0, 0.762, -0.762]

fn (v &Variable[T]) transpose_op(perm []int) !&Variable[T]

transpose_op permutes the axes of the variable according to perm. perm must be a permutation of [0, 1, ..., ndim-1]. Backward: gradient is transposed with the inverse permutation.

Named transpose_op (not transpose) to avoid collision with Tensor.transpose.

x := ctx.variable(vtl.from_array[f64]([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], [2, 3]))
y := x.transpose_op[f64]([1, 0])!
// y.value.shape = [3, 2]

struct AbsGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

AbsGate implements abs(x) element-wise. backward: grad * sign(x)

struct AddGate[T] {}

AddGate defines a public data structure for this module.

struct ClampGate[T] {
pub:
	min_val T
	max_val T
	a       &vtl.Tensor[T] = unsafe { nil }
}

ClampGate implements clamp(x, min, max) element-wise. backward: grad where x is within [min, max], 0 otherwise

struct ConcatGate[T] {
pub:
	axis   int
	splits []int // size of each input along the concat axis
}

ConcatGate concatenates multiple tensors along an axis. backward: split gradient back into original inputs

struct Context[T] {
pub mut:
	// A list of all variables present in an operation.
	// This list can contain duplicates
	nodes []&Node[T]
	// If no_grad is set to true, operations will not
	// be cached, and backpropagation will not be possible
	no_grad bool
	// Reusable CUDA staging for f64 forwards (nil for other T); see autograd_cuda.DeviceSession.
	device_session voidptr = unsafe { nil }
}

Context defines a public data structure for this module.

struct ContextVariableData {
pub:
	requires_grad bool = true
}

ContextVariableData defines a public data structure for this module.

struct CosGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

CosGate defines a public data structure for this module.

struct DivideGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

DivideGate defines a public data structure for this module.

struct ExpGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

ExpGate defines a public data structure for this module.

struct LogGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

LogGate implements log(x) element-wise. backward: grad * (1/x)

struct MatMulGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

MatMulGate defines a public data structure for this module.

struct MeanGate[T] {
pub:
	shape     []int
	axis      int
	num_elems int
}

MeanGate implements mean reduction. backward: grad broadcast to input shape / num_elements

struct MultiplyGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

MultiplyGate defines a public data structure for this module.

struct Node[T] {
pub:
	// Opaque pointer to the concrete gate instance for this node.
	gate voidptr
	// Callback that knows how to cast and run the gate.
	backward BackwardFn = unsafe { nil }
pub mut:
	// The variables that created this node
	parents []&Variable[T]
	// Wrapper around a Tensor, contains operation data
	payload &Payload[T] = unsafe { nil }
	// Debug use only, contains a name for a node
	name string
}

struct Payload[T] {
pub:
	// Contents of the paylod
	variable &Variable[T] = unsafe { nil }
}

struct PowGate[T] {
	a &Variable[T] = unsafe { nil }
	b &Variable[T] = unsafe { nil }
}

PowGate defines a public data structure for this module.

struct ReshapeGate[T] {
pub:
	orig_shape []int
}

ReshapeGate stores the original shape for backward pass. backward: grad reshaped back to original shape

struct SinGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

SinGate defines a public data structure for this module.

struct SqrtGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

SqrtGate implements sqrt(x) element-wise. backward: grad * (1 / (2 * sqrt(x)))

struct SubtractGate[T] {}

SubtractGate defines a public data structure for this module.

struct SumGate[T] {
pub:
	shape []int
	axis  int
}

SumGate implements sum reduction. backward: grad broadcast back to input shape

struct TanGate[T] {
pub:
	a &Variable[T] = unsafe { nil }
}

TanGate defines a public data structure for this module.

struct TanhGate[T] {
pub:
	cache &vtl.Tensor[T] = unsafe { nil }
}

TanhGate implements tanh(x) element-wise. backward: grad * (1 - tanh(x)^2) = grad * (1 - cached^2)

struct TransposeGate[T] {
pub:
	perm  []int
	iperm []int
}

TransposeGate stores the permutation for backward. backward: grad transposed back with inverse permutation

struct Variable[T] {
pub mut:
	// The value of the Variable.  This should not be edited outside
	// of Variable operations, as other edits will not be tracked
	// and will lead to incorrect results
	value &vtl.Tensor[T] = unsafe { nil }
	// The graph the variable is associated with.  This is a reference,
	// as a variable does not own its context
	context &Context[T] = unsafe { nil }
	// The gradient of the Variable.  This is set as a reference to
	// the value of a Variable unless `backprop` has been called, in
	// which case all related Variables will have their gradient
	// updated correctly
	grad &vtl.Tensor[T] = unsafe { nil }
	// If set to true, this variable will track its operations,
	// otherwise it will act similar to a vtl.Tensor, only calculating
	// forward operations
	requires_grad bool
	// Phase 2: optional forward-only GPU activation (`&vtl.CudaTensor[f64]` when `-d cuda`).
	gpu_activation voidptr = unsafe { nil }
}

Variable defines a public data structure for this module.

struct VariableData {
	requires_grad bool = true
}

VariableData defines a public data structure for this module.