include/vecmat/gemm.h file

Packed dense GEMM (C = α op(A) op(B) + β C), batch helpers, and vm_im2col.

Not a BLAS.

Functions

void vm_gemm(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout)
C = alpha * op(A) * op(B) + beta * C for dense float panels.
void vm_gemm_ref(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout)
Triple-loop reference GEMM.
void vm_gemm_batch(vm_float_t*const* C, int ldc, const vm_float_t*const* A, int lda, const vm_float_t*const* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int batch)
Batched GEMM: batch independent GEMMs with shared shape.
void vm_gemm_strided_batch(vm_float_t* C, int ldc, int strideC, const vm_float_t* A, int lda, int strideA, const vm_float_t* B, int ldb, int strideB, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int batch)
Strided batched GEMM: problems live strideX elements apart.
void vm_gemm_ex(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int op, const vm_float_t* bias)
C = alpha * op(A) * op(B) + beta * C with optional fused epilogue.
auto vm_gemm_threads(void) -> int
Resolve the GEMM worker-thread budget.
void vm_gemm_set_threads(int n)
Cap or force the GEMM worker-thread budget.
void vm_im2col(vm_float_t* col, int ld_col, const vm_float_t* img, int n, int c, int h, int w, int kh, int kw, int pad_h, int pad_w, int stride_h, int stride_w, vm_layout_t layout)
NCHW im2col into a GEMM-ready panel.

Function documentation

void vm_gemm(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout)

C = alpha * op(A) * op(B) + beta * C for dense float panels.

Parameters
C Output matrix (M×N), updated in place.
ldc Leading dimension of C.
A Left input matrix; ignored when alpha == 0 or K <= 0.
lda Leading dimension of A.
B Right input matrix; ignored when alpha == 0 or K <= 0.
ldb Leading dimension of B.
M Number of rows of op(A) and C.
N Number of columns of op(B) and C.
K Inner product length.
alpha Scale factor for the A*B product.
beta Scale factor for the existing C values.
transA If true, use A^T; otherwise A.
transB If true, use B^T; otherwise B.
layout Memory layout for A, B, and C.

Blocked packed kernel with thread-local A/B workspaces. Equivalent to vm_gemm_ex with op == VM_GEMM_OP_NONE.

void vm_gemm_ref(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout)

Triple-loop reference GEMM.

Parameters
C Output matrix (M×N), updated in place.
ldc Leading dimension of C.
A Left input matrix; ignored when alpha == 0 or K <= 0.
lda Leading dimension of A.
B Right input matrix; ignored when alpha == 0 or K <= 0.
ldb Leading dimension of B.
M Number of rows of op(A) and C.
N Number of columns of op(B) and C.
K Inner product length (columns of op(A), rows of op(B)).
alpha Scale factor for the A*B product.
beta Scale factor for the existing C values (0 skips reading C).
transA If true, use A^T; otherwise A.
transB If true, use B^T; otherwise B.
layout Memory layout for A, B, and C.

Useful for tests and tiny/fallback paths.

Computes C = alpha * op(A) * op(B) + beta * C. op(X) = X or X^T according to the matching transpose flag. Layout selects row-major or column-major indexing for A, B and C together.

void vm_gemm_batch(vm_float_t*const* C, int ldc, const vm_float_t*const* A, int lda, const vm_float_t*const* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int batch)

Batched GEMM: batch independent GEMMs with shared shape.

Parameters
C Array of batch pointers to output matrices (each M×N).
ldc Leading dimension shared by every C[p].
A Array of batch pointers to left matrices, or NULL.
lda Leading dimension shared by every A[p].
B Array of batch pointers to right matrices, or NULL.
ldb Leading dimension shared by every B[p].
M Number of rows of op(A) and C (same for all problems).
N Number of columns of op(B) and C (same for all problems).
K Inner product length (same for all problems).
alpha Scale factor for each A*B product.
beta Scale factor for each existing C[p].
transA If true, use A^T for every problem; otherwise A.
transB If true, use B^T for every problem; otherwise B.
layout Memory layout for all A, B, and C panels.
batch Number of independent GEMM problems.

A[p], B[p], C[p] are the p-th problem. If every B[p] aliases the same buffer, uses a shared-B pack path; otherwise runs per-item vm_gemm. The batch range is split across worker threads when the work is large enough.

void vm_gemm_strided_batch(vm_float_t* C, int ldc, int strideC, const vm_float_t* A, int lda, int strideA, const vm_float_t* B, int ldb, int strideB, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int batch)

Strided batched GEMM: problems live strideX elements apart.

Parameters
C Base pointer for output matrices; problem p at C + p*strideC.
ldc Leading dimension shared by every C panel.
strideC Element stride between consecutive C problems.
A Base pointer for left matrices, or NULL; problem p at A + p*strideA.
lda Leading dimension shared by every A panel.
strideA Element stride between consecutive A problems.
B Base pointer for right matrices, or NULL; problem p at B + p*strideB.
ldb Leading dimension shared by every B panel.
strideB Element stride between B problems (0 means one shared B).
M Number of rows of op(A) and C (same for all problems).
N Number of columns of op(B) and C (same for all problems).
K Inner product length (same for all problems).
alpha Scale factor for each A*B product.
beta Scale factor for each existing C panel.
transA If true, use A^T for every problem; otherwise A.
transB If true, use B^T for every problem; otherwise B.
layout Memory layout for all A, B, and C panels.
batch Number of independent GEMM problems.

Problem p uses A + p * strideA, B + p * strideB, C + p * strideC. When strideB == 0 (shared B), reuses the shared-B batch path.

void vm_gemm_ex(vm_float_t* C, int ldc, const vm_float_t* A, int lda, const vm_float_t* B, int ldb, int M, int N, int K, vm_float_t alpha, vm_float_t beta, bool transA, bool transB, vm_layout_t layout, int op, const vm_float_t* bias)

C = alpha * op(A) * op(B) + beta * C with optional fused epilogue.

Parameters
C Output matrix (M×N), updated in place.
ldc Leading dimension of C.
A Left input matrix; ignored when alpha == 0 or K <= 0.
lda Leading dimension of A.
B Right input matrix; ignored when alpha == 0 or K <= 0.
ldb Leading dimension of B.
M Number of rows of op(A) and C.
N Number of columns of op(B) and C.
K Inner product length.
alpha Scale factor for the A*B product.
beta Scale factor for the existing C values.
transA If true, use A^T; otherwise A.
transB If true, use B^T; otherwise B.
layout Memory layout for A, B, and C.
op Epilogue flags (VM_GEMM_OP_*).
bias Optional length-N bias vector (may be NULL).

Same blocked packed path as vm_gemm, plus last-K bias add and/or ReLU when requested via op.

int vm_gemm_threads(void)

Resolve the GEMM worker-thread budget.

Returns Requested thread count (>= 1 from hardware fallback when unset).

Order: vm_gemm_set_threads limit if > 0; else a positive integer from VECMAT_GEMM_THREADS (invalid values ignored); else online CPU count.

void vm_gemm_set_threads(int n)

Cap or force the GEMM worker-thread budget.

Parameters
n Thread limit (0 = auto, 1 = serial, N = cap at N).

n > 0 sets a fixed limit used by vm_gemm_threads; n == 0 (or negative, treated as 0) restores auto selection via env / hardware.

void vm_im2col(vm_float_t* col, int ld_col, const vm_float_t* img, int n, int c, int h, int w, int kh, int kw, int pad_h, int pad_w, int stride_h, int stride_w, vm_layout_t layout)

NCHW im2col into a GEMM-ready panel.

Parameters
col Destination matrix for unfolded patches.
ld_col Leading dimension of col under layout.
img Source image tensor in NCHW order.
n Batch size (N).
c Channel count (C).
h Input height (H).
w Input width (W).
kh Kernel height.
kw Kernel width.
pad_h Top/bottom padding in pixels.
pad_w Left/right padding in pixels.
stride_h Vertical kernel stride (<= 0 treated as 1).
stride_w Horizontal kernel stride (<= 0 treated as 1).
layout Row-major or column-major storage for col.

img is N×C×H×W packed as ((n*C+c)*H+y)*W+x. Output has C*kH*kW rows and N*outH*outW columns (zero-filled off-image taps). outH/outW use standard floor division with the given pads and strides; non-positive stride_h / stride_w are treated as 1.