llama.cpp b11398: treat an x86 K tail as a shape-specific fast-path condition

A tail is a tensor shape, not a performance label

llama.cpp b11398, published on October 4, 2026, is a prerelease. Its linked pull request #29806, created October 1 and merged October 4, extends x86 tinyBLAS handling for a final, incomplete K block of BF16, FP16, or FP32 matrix multiplication. The merged commit a7b94df adds partial SIMD loads and test cases around vector-width boundaries.

For a matrix multiplication, K is the shared reduction dimension. A SIMD kernel commonly consumes it in fixed-width blocks. When K % KN is nonzero, the last block is a tail. Before this patch, the PR says a non-aligned BF16 K could cause tinyBLAS to reject the operation and leave the generic CPU path to handle it. The new code processes complete blocks as before, then handles only the remaining elements with zero-filled or masked loads on supported x86 vector paths. That is an implementation boundary, not proof that every x86 CPU, data type, model, quantization, or workload becomes faster.

1. 1Pinned build + x86 ISA + element type + M/N/K tensor shape
2. 2decide whether tinyBLAS is eligible

1. 1K divided by KN
2. 2complete blocks
3. 3same kernel path

1. 1K modulo KN is nonzero
2. 2one bounded tail load
3. 3accumulate
4. 4verify numerical result

1. 1Missing ISA, different type, changed build, or failed comparison
2. 2no fast-path conclusion

The PR author reports measurements for one Qwen3.8-27B BF16 multimodal projection and one AMD 9950X, with fixed image sizes and 16 threads. Those figures motivate a shape inspection, but they are not an independent benchmark and do not transfer to another processor, thread count, image encoder, model revision, or input. The change also adds CPU test cases across small K boundaries. Its dispatch diff makes use_ref and a contiguous source separate predicates: reference mode deliberately bypasses tinyBLAS, and the tail code is compiled only for AVX, AVX2, or AVX512 paths. A generic CPU path in the author's before/after explanation is therefore a runtime choice in that implementation, not this article's rule for handling a rejected result.

The August 18, 2024 llamafile 0.8.13 release supplies a narrower historical lesson: it describes ruler reduction for F32, F16, and BF16 CPU dot products to control rounding-error accumulation. That makes numerical tolerance a relevant question alongside kernel eligibility. This was a llamafile release, not evidence that b11398 has the same numerical implementation, current build predicates, or performance result. The release's numerical improvement claim was not independently measured here.

Use one shape manifest before choosing the build

Record the build tag and commit, CPU model and enabled ISA, operating system, thread setting, tensor element types, and M/N/K for the one multiplication that matters. Add model revision and the exact input route if the multiplication belongs to a vision encoder. Without this manifest, “CPU path” collapses distinct runtime conditions into one misleading result.

This is an original, unexecuted N=1 fixture for a single offline consumer:

1. Use one isolated b11398 candidate build and one local model artifact. Do not expose a server or attach tools, credentials, or untrusted inputs. Record the processor’s reported ISA rather than guessing it from its product name.
2. First write the numerical acceptance tolerance for the element type in the manifest, before measuring; floating-point results need not be bitwise-identical. For a synthetic BF16 boundary worksheet, hold M = 1152 and N = 784 fixed, choose KN = 32 only after confirming the AVX512-BF16 condition, then compare an aligned K = 4320 (4320 % 32 = 0) with a tail K = 4304 (4304 % 32 = 16). Use identical deterministic values for the first 4304 reduction elements and zeros for the 16 added elements in the aligned case. Check each shape against its own reference dot product under the predeclared tolerance; compare timings separately, since the operation counts differ. The PR author names M = 1152, K = 4304, and that remainder for a particular model projection; this worksheet does not independently verify that model card or reuse its workload.
3. Require the predeclared, type-specific tolerance, process exit status, elapsed time, build identity, and the path actually selected if it is observable. A mismatch, unsupported ISA, use_ref selection, non-contiguous source, unavailable trace, timeout, or changed manifest rejects the comparison. It does not authorize a second backend or an automatic retest with altered conditions.

The repository declares MIT terms for its code. That does not grant a model, weight, input, or benchmark dataset license. The current security policy calls for isolation of untrusted models and inputs; a local CPU fast path does not change that boundary. No model download, compilation, CPU feature check, numerical comparison, or timing run was performed while writing this note. Such work has local compute, storage, and operator-time cost.

Keep CPU tails separate from CUDA buffer sizing

The CUDA MoE buffer note considers CUDA temporary-buffer dimensions under a specific expert/batch condition. This note instead concerns the final reduction-dimension block in x86 tinyBLAS. Both require a pinned environment and a narrow fixture, but neither predicts the other’s behavior. A learner should first identify the exact arithmetic shape and execution backend, then ask whether a source actually covers that combination.