On October 27, NVIDIA described how a configured KAI Scheduler and GPU Fractioning Operator can bring CUDA 13.4 memory limits to workloads sharing a Kubernetes GPU. The scheduler places the workload; components on the GPU node apply its boundary.

The big change

  • A fractional GPU request becomes a runtime boundary only when scheduling, the operator and a supported node stack are configured together. Placement accounts for the requested share; the node applies the limit to the container.
  • Memory limits and compute-sharing mode answer different operational questions. A team choosing a mode must consider how a workload failure affects its neighbors.
  • A workload may be allowed to borrow unused memory, but its recovery when that memory is reclaimed matters as much as utilization. The documented CUDA 13.4 and driver r615 limitations qualify the promised minimum in specific Run:ai reclaim cases.

The allocation travels from scheduler to node

NVIDIA’s technical account describes KAI Scheduler 0.18.0’s NvFractions mode. The operator repository explains that its runtime plugin injects the request and limit into a container before startup, while a separate component exports per-pod GPU metrics. The driver enforces the limit.

The integration is a configuration choice, not a property that appears automatically in every CUDA-enabled cluster. NVIDIA describes enabling NvFractions at KAI installation, which also installs the GPU Fractioning dependency. KAI also documents separate operator management and its installation order.

The current KAI GPU-sharing documentation describes a per-container memory request, such as 7680Mi, and an optional higher limit. Fractional workloads must use KAI and belong to a KAI queue.

KAI uses the request to decide placement, while the operator’s node plugin applies the runtime boundary when the container is created. The current KAI documentation says it waits for the fractioning configuration and node readiness before scheduling NvFractions workloads.

NVIDIA distinguishes default time-slicing from simultaneous SM-sharing. Its account says time-slicing supplies separate error domains, while SM-sharing uses a shared error domain. A single GPU runs one compute mode at a time; the scheduler does not place workloads using both modes on that device. Those statements describe this integration, not every older implementation with a similar name. NVIDIA’s MPS documentation specifies Linux with cgroup v2, CUDA 13.4 or newer and a non-MIG device. The operator repository also requires Kubernetes 1.28 or later, a supported NRI-enabled runtime and driver r615 or newer. Its GPU Operator and component versions matter as well; the repository says r615 is not the default driver in GPU Operator 26.7.1.

What happens above the request

A higher limit permits a workload to use available memory above its requested share; it does not make that extra memory permanently available. NVIDIA’s MPS partitioning documentation distinguishes a hard limit, where an allocation fails, from a softer threshold above which a process can become a candidate for eviction when another workload needs memory. Its described pressure handling is reactive: the server selects an over-threshold process and the requesting allocation retries.

That distinction affects the application, not just capacity planning. A team considering a burst limit needs to know what happens to a job if borrowed memory is reclaimed and whether it can restart or recover its work. A benchmark that only records average utilization would miss that failure path.

NVIDIA’s separate Run:ai documentation describes CUDA 13.4 and r615 limitations during memory reclamation. For Run:ai self-hosted v3.0 dynamic fractions on that combination, NVIDIA documents two exceptions. If several workloads have borrowed memory and one reclaim does not free enough, the requesting workload may not obtain its full request. In other scenarios a process allocating within its request can receive SIGKILL instead of a borrower being reclaimed; NVIDIA points to a misc cgroup fallback for affected scenarios. These are documented limitations of that Run:ai configuration, not proof of identical behavior in every KAI deployment.

Why existing sharing recipes need a version check

An August 3 NVIDIA tutorial used KAI Scheduler 0.16.4 with separate tenant Kubernetes control planes. It warned that its software-sharing setup did not enforce GPU memory isolation at the hardware level and that applications needed to respect their allocations. It discussed hardware MIG partitioning as another option.

That earlier design solves a different part of the problem. Separate Kubernetes control planes let teams manage their own administrative environments; they do not automatically establish the runtime memory boundary of processes on a shared accelerator. The October integration should therefore be evaluated as a specific changed mechanism, rather than used to rewrite the guarantees of an older deployment.

For a trial, record the operator and scheduler versions, driver, sharing mode and request and limit. Check the hard-limit allocation response, then test contention and recovery: in a Run:ai deployment matching the documented versions, include the case where several borrowers exceed their requests and a single reclaim cannot free enough. Record which process exits and whether the requester obtains its allocation. These are proposed acceptance checks; BIG CHANGE did not install or test the stack.

The sources describe an implementation path, not a measured saving for a particular cluster. Capacity gained must be weighed against the observed service and recovery behavior of the workloads actually deployed.