GPU Unlock's October 24 tested-card table records results for some RTX 30-series GPUs. The Linux project says its module raises the issue rate of FP16 and BF16 operations that accumulate in FP32. A contributor measured a large gain in a dense matrix benchmark on an RTX 3090 Ti. The result depends on the card and the work being timed.

The big change

  • What changed: A community module reports a higher rate for a particular kind of matrix instruction on some Ampere GeForce cards. Its October 24 compatibility table distinguishes tested successes from an accepted card that showed no BF16 gain.
  • Why it matters: Engineers serving local models can see gains when prompt processing spends enough time in affected matrix kernels. Contributor reports put measured prefill gains well below the largest dense-matrix result, and generation speed barely moved in the cited LLM tests.
  • What to watch: The useful comparison is a measured workload on the exact card, software stack and power limit. The table's untested entries and failed BF16 result prevent its reported successes from becoming a promise across the RTX 30 series.

What the matrix number measures

The project's README at its October 24 revision describes a Linux kernel module that acts before the NVIDIA driver binds the card. It says the change doubles the issue rate of FP16/BF16 fused multiply-add operations with FP32 accumulation on named Ampere chips. That is a statement about a specific operation, not a measured doubling of a complete AI application.

The headline benchmark came from an October 18 contributor report and its linked validation note. On a physical RTX 3090 Ti at a 450 W limit, the contributor timed dense cuBLAS matrix multiplication with all three dimensions at 8,192, FP32 accumulation and medians of five samples. FP16 throughput rose from 87.46 to 118.39 TFLOPS, BF16 from 87.50 to 157.41, and TF32 from 42.58 to 82.61. Those are roughly 35%, 80% and 94% increases, respectively, calculated from the reported figures.

The setup matters. That first Ti run used a modified NVIDIA 610.43.02 driver and a custom early boot integration. A later October 22 report in the same issue described a different board with an upstream package and unmodified driver, but its before and after tests used different power limits. The reports support the existence of gains in those configurations; combining their numbers into one stock-system benchmark would erase important conditions. These are contributors' measurements, not BIG CHANGE hardware tests or an independent replication of the project.

Prefill and decode tell different stories

Large matrix multiplication rewards extra arithmetic throughput. NVIDIA's matrix multiplication guide uses an 8,192-cubed GEMM as an example of a computation-heavy operation. A local model request has other phases. Prefill processes the prompt in parallel, while decode produces output tokens sequentially and often spends more time moving data than multiplying it, as NVIDIA explains in its inference overview. The bottleneck also depends on model, batch and implementation.

That distinction appears in the user reports. In an October 24 RTX 3090 report, vLLM 0.30.0 serving a 27B BF16 model processed a 2,904-token prompt about 33% faster in prefill, while reported decode rate changed about 1.2%, from roughly 113 to 114 tokens per second. Another contributor's two-card RTX 3080 20 GB report measured 33–36% more vLLM prefill throughput across its tested prompt lengths. On the same cards, a different GGUF engine showed almost no prefill change at 8,000 input tokens and about 12–13% at 32,000 and 128,000. Its decode changes stayed within the reported run spread. These comparisons involve different models and engines, so their percentages are not interchangeable.

Even an application using some affected kernels can gain little overall. The October 22 RTX 3090 Ti reporter in issue #1 described an INT8-heavy vLLM fork where about 29% of GPU time was in BF16/FP16 kernels with FP32 accumulation. The end-to-end gain was about 5%, which the reporter said was within run-to-run noise. That measurement is a useful counterweight to applying the dense GEMM figure to an entire service.

Card support is still an empirical question

The October 24 tested-card table explicitly separates accepted device IDs from user-reported results. It lists gains on several 3090, 3080, 3070 and 3060 variants, but says an RTX 3080 Ti 20 GB OEM card completed the unlock while BF16 remained at half rate. The underlying compatibility report measured the missing gain after checking the selector state. The table lists GA103 without a report and the 3070 Ti as untested with this module.

Version matters too. The commit history records a recent revert to a working state, card-support changes, the October 24 tested-card table and a CI addition in the pinned latest merge. The public project began in September; this is an evolving compatibility record, not a new October 25 release or a settled performance specification.

For an engineer deciding whether the reported improvement applies, the numbers need to stay attached to their measurement: dense GEMM, model prefill, decode or complete request; exact card; software version; and power limit. The current reports document gains in some of those combinations and a BF16 exception in another.

Sources & further reading