Vectors cheat sheet

A vector operation does the same work on several values at once. This page explains the vocabulary for that work, first for CPU registers and then for GPU threads, with four-lane examples so you can follow every value. For Rake's own terms, see the language glossary, and for what Rake implements, see primitives, operations, and targets. Rake's GPU profiles are currently a design.

CPU · GPU

CPU

Values, lanes and registers

Think of a vector register as a row of cells. Each cell holds one element, and an instruction can operate on the whole row. A 256-bit register holds eight 32-bit floats, sixteen 16-bit integers or thirty-two 8-bit integers.

Arithmetic changes values, a broadcast copies one value into every lane, and a shuffle changes which lane supplies each result.

Term	Meaning
Scalar	One value, such as a single float.
f32 / FP32 / binary32	A 32-bit IEEE 754 float, C's `float`. `f64` / FP64 uses 64 bits.
Vector / packed value	Several elements held together in a register. In Rake, `pack` declares a record instead.
Element / lane	An element is a stored value, and a lane is its position within a vector operation.
SIMD	Single instruction, multiple data: one instruction operates on several elements.
Uniform / varying	A uniform has the same value in every lane, while a varying value can differ by lane.
Fixed-width / scalable vector / VLA	Fixed-width code assumes one register size. Scalable, vector-length-agnostic code adapts to the machine's vector length (VL), the active element count at each step.
SSE / SSE2 / XMM	x86 families with 128-bit XMM registers. SSE2 adds packed integer and double-precision operations, among others.
AVX / AVX2 / YMM	x86 families with 256-bit YMM registers. AVX2 extends packed integer operations and adds gathers.
AVX-512 / ZMM / opmask	x86 extensions with 512-bit ZMM registers and dedicated mask registers.
NEON / Advanced SIMD	Arm's fixed-width SIMD, with 128-bit vectors on AArch64.
SVE / SVE2	Arm's scalable vectors, with predicates and vector-length-agnostic loops.
RVV / VLEN / SEW / LMUL	RISC-V vectors, the hardware register length, the selected element width, and the multiplier that groups registers into one logical vector.
WebAssembly SIMD / v128	A virtual 128-bit vector. The runtime selects the physical instructions and allocates registers.

Architecture references: Intel intrinsics, Arm Advanced SIMD, Arm scalable vectors, RISC-V vectors and WebAssembly vector instructions.

Moving values between lanes

A shuffle reads values already in registers, whereas a gather reads from memory, so although both use indices, they differ in latency and memory effects.

Term	Meaning and example
Broadcast / splat	Replicate one value, from memory or a register lane: `7 → [7,7,7,7]`.
Shuffle / permute	Select and reorder source lanes, possibly repeating them: `[a,b,c,d]` with `[2,0,3,1]` gives `[c,a,d,b]`.
Swizzle / table lookup	Indexed lane selection from one vector or a small register table.
Static / dynamic shuffle	A shuffle whose indices are known at compilation, or computed at runtime.
Zip / interleave / unpack	Alternate elements from two vectors: `[a,b]` and `[x,y]` become `[a,x,b,y]`. Some instruction sets use “unpack” for this or for widening.
Unzip / deinterleave	Separate alternating elements: `[a,x,b,y]` becomes `[a,b]` and `[x,y]`.
Transpose	Exchange rows and columns, turning vectors of records into vectors of fields.
Reverse / rotate lanes	`[a,b,c,d]` becomes `[d,c,b,a]`, or cyclically `[d,a,b,c]`. Bit reversal and rotation stay inside an element.
Slide / shift lanes	Move whole elements or bytes across positions, filling from a defined value or a second source. Shifting bits inside each element is a different operation.
Extract / insert	Read one lane as a scalar, or replace one lane with a scalar.
Compress / compact	Pack selected lanes toward the start: `[a,b,c,d]` under `[1,0,1,0]` gives `[a,c,…]`.
Expand	Place dense values in selected positions: `[a,c]` under `[1,0,1,0]` gives `[a,fill,c,fill]`.
Lane-local / cross-lane	Lane-local work depends only on corresponding inputs, while cross-lane work exchanges or combines elements. Some instructions only exchange within 128-bit sub-vectors.

The GCC vector reference illustrates shuffle selection and conversion.

Masks and selection

Term	Meaning
Comparison	Compare corresponding elements to produce a mask: `[1,4,2,9] > 3 → [0,1,0,1]`.
Mask / predicate	A true or false choice per lane, held as vector bits or in a dedicated predicate register.
Active / inactive lane	A lane included in, or excluded by, the operation's mask.
Predication / merge / zero masking	Execute under a mask. Depending on the instruction, inactive lanes skip the work, keep an old value (merge masking) or become zero (zero masking).
Select / blend	Choose one of two values per lane: mask `[1,0,1,0]` selects `[a,y,c,w]` from `[a,b,c,d]` and `[x,y,z,w]`.
Any / all / none	Reduce a mask to one truth value.
Movemask / bitmask	Collect one bit per lane into a scalar. An x86 movemask usually takes each lane's sign bit.
Popcount / first set bit	Count a mask's selected lanes, or find the first one. An empty mask needs a defined result.
Masked load / store / first-fault	Memory operations on selected lanes only. Fault suppression stops some inactive accesses from faulting, and first-fault loads record how far a load got, but neither makes an invalid pointer safe.
Speculation / partial operation	Calculating both candidates before selecting. An operation valid for only part of its domain, such as square root, needs suppressed arithmetic or safe operands in inactive lanes.

Combining lanes and calculating values

A reduction produces one combined value, while a scan keeps each partial result.

Term	Meaning
Element-wise / vertical	Apply an operation independently at each lane position.
Horizontal / reduction	Combine lanes into fewer results, such as a sum or maximum.
Scan / prefix sum	Running partial results. An inclusive scan includes the current element, and an exclusive scan starts from an identity and excludes it.
Dot product	Multiply corresponding elements and sum the products, with instruction-specific accumulation width and rounding.
FMA / MAC	A fused multiply-add calculates `a*b+c` with one rounding. A multiply-accumulate may round once or twice, depending on the instruction.
Saturating / wrapping arithmetic	An overflow clamps to the limit, or wraps modulo the width: unsigned 8-bit `250+10` gives 255 or 4.
Widen / narrow / extend	Convert to a wider or narrower element type. Signed widening copies the sign bit and unsigned widening fills with zeros, while narrowing needs a rule for overflow.
Conversion / bitcast	A conversion preserves the numerical value where possible, while a bitcast reinterprets the same bits.
Logical / arithmetic shift	Shift bits inside each element, filling with zeros or, for arithmetic right shifts, the sign bit.
Transcendental functions	`sin`, `exp`, `log` and similar. A vector implementation may compute them with a polynomial or library sequence.
Ordered / unordered comparison	An ordered comparison is false when an operand is NaN, while an unordered one reports it.
NaN / infinity / signed zero	Special float values with their own comparison, minimum, maximum and arithmetic rules.
Subnormal / FTZ / DAZ	Very small floats. Flush-to-zero discards subnormal results, and denormals-are-zero treats subnormal inputs as zero.
Rounding / ULP / fast math	Rounding picks a representable result, an ULP is the spacing between representable values, and fast math permits specific departures from strict rules.
Reassociation	Regrouping, such as `(a+b)+c` to `a+(b+c)`, which can change a floating-point result, including in tree reductions.

Memory and data layout

A contiguous load reads neighbouring elements, while gather and scatter use one address per lane.

Term	Meaning
Contiguous / strided	Neighbouring lanes access neighbouring elements, `[0,1,2,3]`, or elements a constant gap apart, `[0,2,4,6]`.
Gather / scatter	Load or store through one index per lane: `out[i] = memory[index[i]]` and `memory[index[i]] = value[i]`. A scatter needs a rule for duplicate destinations.
Alignment	An address that's a multiple of a byte boundary. Unaligned access support and cost vary by instruction.
Cache line / locality	A cache's unit of transfer, a block of nearby bytes. Good locality means reusing data or using nearby data.
AoS / SoA / AoSoA	Records stored whole (`x0,y0,x1,y1,…`), as separate field columns (`x0,x1,…` and `y0,y1,…`), or as small SoA tiles. Rake's stack uses SoA.
Aliasing / overlap	Different pointers referring to the same memory, so a vectorised loop needs rules for overlapping reads and writes.
Strip mining / tail	Dividing a loop into vector-sized chunks, and the incomplete final chunk, handled by masking or separate cleanup.

Data streams and traversal

A data stream is a sequence of elements or records, in memory or arriving from a file or network, such as a column of a million floats. A stream traversal walks along it one rack at a time: it loads the next rack, calculates the result, stores it and advances. The sequence can be any length, so the final rack may have fewer active lanes.

The underscore marks an inactive lane. Loop control advances from rack to rack, and the arithmetic works across each rack's lanes, including the tail's active lanes.

A stream update is a traversal that writes its results, either to a separate output (out of place) or back into the input (in place). An in-place update needs rules for overlap, so stores don't replace inputs before they've been read. In Rake, a stack run describes the update on whole columns, and the compiler performs the traversal. Uniform loop control and address calculation advance through memory, while the lane arithmetic stays vectorised.

Compilation and cost

Term	Meaning
Intrinsic	A compiler-provided function that requests a specific instruction's semantics.
Autovectorisation	A compiler turning a scalar loop into vector work after checking dependencies.
Scalarisation	Implementing a vector operation as separate scalar work. Silent scalarisation hides a change in execution.
Loop unrolling / fusion	Repeating a loop body to expose independent work, or combining compatible passes over data.
Register pressure / spill	Demand for registers at once. A spill saves a value to memory and reloads it when registers run out.
ISA / microarchitecture	The instruction set, and the hardware implementing it. Processors with one ISA can perform differently.
Latency / throughput	The delay from input to result, and the rate of independent completed operations.
Bandwidth / arithmetic intensity / roofline	Bytes moved per unit time, arithmetic per byte moved, and the model comparing the two limits. A wider vector can leave a memory-bound loop unchanged.
ABI / calling convention	Rules for data layout, argument and result locations, and register preservation across calls.

ISPC's performance guide explains SIMD execution and layout costs, and Rake's backend describes its checked lowering.

GPU

Threads, warps and workgroups

On a GPU, each thread holds its own values, and a group of threads executes lane-parallel work, so the CPU arithmetic terms above apply to GPU lane values too. A GPU shuffle exchanges values between threads, like a CPU register shuffle with extra participation rules.

A warp is a hardware execution group, while a block is a programmer-selected group with shared memory and synchronisation.

Term	Meaning
Driver / runtime	Software that loads programs, manages memory, submits work and may compile intermediate code.
Kernel / shader	A program entry run across many GPU invocations.
Thread / work-item / invocation	One instance of the program, with its own indices and values. GPU threads aren't operating-system threads.
Lane ID	A thread's position within its subgroup, 0 to 31 in an NVIDIA warp.
Warp	NVIDIA's group of 32 threads, scheduled together for SIMT execution.
Wave / wavefront	AMD's comparable group, commonly 32 or 64 wide depending on architecture and mode.
Subgroup	Portable API term for invocations that take part in subgroup operations, with device-dependent sizes.
Block / workgroup / threadgroup	A programmer-selected group of cooperating threads, in CUDA, OpenCL/Vulkan and Metal respectively.
Grid / dispatch / launch	The blocks or workgroups submitted to run a kernel.
SM / CU	NVIDIA's streaming multiprocessor or AMD's compute unit, the hardware hosting groups of threads.
SPMD / gang	Single program, multiple data: many program instances on different data. ISPC maps a gang of instances onto CPU SIMD lanes.
SIMT	Single instruction, multiple threads: GPU execution across thread lanes, masking lanes on differing paths.
Cooperative group / tile	An explicitly defined group that cooperates and synchronises, whose size can differ from a warp's.

See NVIDIA's SIMT model, AMD architecture references and Vulkan subgroups.

Participation, control and collectives

Term	Meaning
Uniform / divergent control	Every participant in a group takes the same path, or lanes take different ones.
Divergence / reconvergence	Lanes split onto different paths and later meet again, and masked execution spends time on paths with few active lanes.
Active mask	The lanes participating at a point in execution, which can differ from the intended members of a later collective.
Independent thread scheduling	Hardware tracking each thread's progress separately, so communication can't assume warp lockstep without explicit synchronisation.
Collective / subgroup operation	An exchange, vote or reduction among threads. Every required participant needs to reach it with compatible membership and operands.
Warp shuffle	Reading another participating lane's value without a round trip through shared memory.
Shuffle up / down / XOR	Reading from a lower or higher lane index, or a lane whose ID differs by an XOR mask.
Broadcast / read-first-lane	Sharing one participating lane's value with the group.
Ballot	One predicate bit per participating lane, collected into a bitset, like a CPU movemask.
Vote / any / all	Whether some or all participating threads satisfy a predicate.
Subgroup reduction / scan	Combining values or producing prefixes across a group, which needs rules for order and inactive inputs.
Barrier	A rendezvous at a given scope, where every specified participant needs to arrive. A warp barrier can't synchronise other warps.
Memory fence / acquire / release	Ordering memory visibility. Waiting for other threads needs a separate operation, such as a barrier.
Atomic / CAS	An indivisible read-modify-write, such as compare-and-swap, with a memory order and scope. Contention can serialise it.
Scope	The threads covered by an operation: subgroup, workgroup, device or system.

The PTX instruction reference specifies shuffle membership, and NVIDIA's synchronisation guide defines barriers, scopes and ordered communication.

Memory movement and matrix operations

Term	Meaning
Global / device memory	Device-wide memory, usually GPU DRAM behind caches.
Shared / workgroup memory	Programmer-managed storage shared within a block, with capacity, bank and synchronisation limits.
Local / private memory	Thread-private storage. CUDA “local memory” can live in device memory and holds spills, whereas OpenCL “local” means workgroup-shared.
Coalescing	Combining nearby lanes' requests, including gathers and scatters, into fewer memory transactions. Alignment, element size and the address pattern decide how many.
Bank conflict	Shared memory is split into banks, and different addresses in one bank take extra steps.
Padding / tiling	Changing a layout or processing blocks to improve locality, coalescing or bank use.
CUDA stream / command queue	An ordered queue of submitted work, such as transfers and kernel launches, distinct from a data stream.
Tensor core / MMA / fragment	Matrix hardware and its warp-level multiply-accumulate interface, with each participant holding a fragment of the matrix in an often architecture-specific layout.

CUDA memory guidance explains coalescing and bank conflicts, and CUDA matrix operations have their own participation and layout requirements.

Resources, performance and compiled artifacts

High occupancy can coexist with low lane utilisation.

Term	Meaning
Occupancy	Resident warps relative to capacity, limited by registers and shared memory, either in theory for a launch or as achieved during execution.
Lane utilisation	The share of lanes doing useful work.
Register pressure / spill	On a GPU, spilled values go to device-backed local memory. Spill checks inspect memory instructions and data flow as well as resource reports.
Latency hiding	Running other warps while some wait, which needs enough eligible work.
Serial lane emulation	One thread looping over elements that were meant to run across lanes.
FLOPS	Floating-point operations per second. Specifications give peak rates, and measurements give achieved ones.
Launch / transfer overhead	The cost of submitting kernels and moving data, which can dominate tiny workloads.
PTX / virtual ISA	NVIDIA's intermediate instruction language, lowered to hardware by NVIDIA's toolchain.
SASS / cubin	NVIDIA's physical instruction assembly, and the compiled device file holding code and metadata.
ptxas / AOT / JIT	NVIDIA's assembler, and compiling ahead of time or while loading.
SPIR-V / Vulkan / OpenCL / Metal / HIP	A portable intermediate representation and the graphics and compute interfaces, each with its own execution, memory and compilation rules.
Disassembly / resource report	Decoded instructions, and counts such as registers or local memory.
Artifact verification	Checking properties of exact executable bytes for one architecture and toolchain, which don't carry over to a rebuild.

CUDA's resource discussion and binary utilities cover registers, occupancy and artifact inspection. Rake's proposed GPU profiles check lane mapping and designated register and memory properties.