How to Install NVIDIA Run:ai's KAI Scheduler on Kubernetes

Your default Kubernetes scheduler treats a GPU as an indivisible brick: a pod gets a whole card or it waits. KAI Scheduler, the scheduling core of NVIDIA Run:ai, adds queues, quotas and fractional GPUs. By the end of this guide you will install KAI Scheduler with Helm, create queues, run a pod through it and have two pods sharing one GPU.

On October 13, 2026 NVIDIA open-sourced the full Run:ai platform. KAI Scheduler is Apache 2.0 and a CNCF Sandbox project, and it can be adopted on its own. You need a Kubernetes cluster, kubectl and Helm. The NVIDIA GPU Operator is required only for pods that request GPUs.

Install KAI Scheduler with Helm

Pick a version from the releases page. When I checked, v0.18.6 was marked latest. Recent releases also publish chart tags without the v prefix, which allows ranges like ~0.18.0.

helm upgrade -i kai-scheduler oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler \
  -n kai-scheduler --create-namespace \
  --version <VERSION>

kubectl get pods -n kai-scheduler

upgrade -i installs or upgrades, so rerunning it is safe. Wait until every pod in the kai-scheduler namespace is Running. KAI runs beside the default scheduler, and only pods that ask for it are touched.

Check the default queues

The install creates default-parent-queue and a leaf queue, default-queue, under it. Use the full resource name so it can't clash with other queue CRDs:

kubectl get queues.scheduling.run.ai

Create queues for your teams

Queues carry a guaranteed quota, a hard limit and an overQuotaWeight that decides who gets spare capacity. CPU is in millicores, memory in MB, GPU in whole devices, and -1 means unlimited.

# queues.yaml
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: research-team
spec:
  displayName: "Research Team"
  resources:
    cpu:
      quota: 1000
      limit: 2000
    gpu:
      quota: 1
      limit: 2
---
apiVersion: scheduling.run.ai/v2
kind: Queue
metadata:
  name: ml-team
spec:
  displayName: "ML Team"
  parentQueue: "research-team"
  priority: 200
  resources:
    cpu:
      quota: 500
      overQuotaWeight: 2
    gpu:
      quota: 1
      overQuotaWeight: 1

Apply it with kubectl apply -f queues.yaml. Workloads should reference a leaf queue, here ml-team. These values come from the queues docs; set every field you care about explicitly, because the docs disagree on what an unset quota means.

Submit a pod through KAI Scheduler

A pod needs two things: schedulerName: kai-scheduler and a kai.scheduler/queue label. Use a dedicated namespace, never kai-scheduler itself.

kubectl create namespace team-ml

# gpu-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
  labels:
    kai.scheduler/queue: ml-team
spec:
  schedulerName: kai-scheduler
  automountServiceAccountToken: false
  containers:
    - name: main
      image: ubuntu
      command: ["bash", "-c"]
      args: ["nvidia-smi; trap 'exit 0' TERM; sleep infinity & wait"]
      resources:
        limits:
          nvidia.com/gpu: "1"
          memory: 1Gi

This is the repo's quickstart GPU pod with the queue swapped.

kubectl apply -n team-ml -f gpu-pod.yaml
kubectl logs gpu-pod -n team-ml
kubectl delete pod gpu-pod -n team-ml

Seeing nvidia-smi output means the pod was scheduled by KAI and got a GPU. The delete frees the card for the next step.

Turn on GPU sharing for fractional GPUs

Older guides tell you to pass global.gpuSharing=true. The current GPU sharing docs deprecate that boolean in favor of global.gpuSharingMode, and its default is NonMemoryEnforced. So Kubernetes GPU fractioning already schedules, but nothing stops a pod from using more GPU memory than its share. Set the mode explicitly so the choice is on record:

• NonMemoryEnforced: schedules fractions, no memory isolation.
• HamiCore: enforces CUDA memory limits through HAMi-core; install kai-resource-isolator afterward.
• NvFractions: uses NVIDIA's gpu-fractioning operator.
• Disabled: rejects fractional requests.

helm upgrade -i kai-scheduler oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler \
  -n kai-scheduler --create-namespace \
  --version <VERSION> \
  --set global.gpuSharingMode=NonMemoryEnforced

Now submit two pods that each ask for half a GPU:

# half-gpu.yaml
apiVersion: v1
kind: Pod
metadata:
  name: half-gpu-a
  labels:
    kai.scheduler/queue: ml-team
  annotations:
    gpu-fraction: "0.5"
spec:
  schedulerName: kai-scheduler
  containers:
    - name: gpu-workload
      image: nvidia/cuda:13.0.2-base-ubi8
      command: ["bash", "-c"]
      args: ["nvidia-smi --query-gpu=name,memory.total --format=csv,noheader; sleep infinity"]
---
apiVersion: v1
kind: Pod
metadata:
  name: half-gpu-b
  labels:
    kai.scheduler/queue: ml-team
  annotations:
    gpu-fraction: "0.5"
spec:
  schedulerName: kai-scheduler
  containers:
    - name: gpu-workload
      image: nvidia/cuda:13.0.2-base-ubi8
      command: ["bash", "-c"]
      args: ["nvidia-smi --query-gpu=name,memory.total --format=csv,noheader; sleep infinity"]

Apply it with kubectl apply -n team-ml -f half-gpu.yaml. On a cluster with one free GPU, both pods should reach Running. Check with kubectl get pods -n team-ml -o wide.

Gotchas

• Admission rules: a pod can't mix a fractional request with nvidia.com/gpu, gpu-fraction must be above 0 and below 1, and only one container per pod can ask for a fraction.
• NvFractions is a bigger install. NVIDIA's Dynamo guide lists GPU Operator v26.7.1 or newer, driver r615 or newer, and containerd 2.0+ or CRI-O with NRI. Install gpu-fractioning before KAI, or restart the KAI operator so it detects the CRD.
• OpenShift: with a GPU Operator older than v25.10.0, add --set-string admission.gpuFractionRuntimeClassName="". Never set it to null.
• Pin the chart version. Rerunning without --version can upgrade you when you didn't plan to.
• What is still commercial: the NGC Community Product of the full platform includes no NVIDIA enterprise support, and the paid Run:ai offerings continue separately. I did not confirm the integrated-stack chart name, so this guide covers only KAI Scheduler.

Queues and fractions turn a GPU cluster from a first-come, first-served argument into something you can put quotas on, which matters the day a second team ships AI to production on it.