How to Create a GKE Node Pool on Z4M Machines

A single Z4M node can carry up to 168,000 GiB of local Titanium SSD, which is a lot of disk to hand to whichever Pod the scheduler picks first. Z4M is Google's storage-optimized machine series, and GKE now supports it. By the end of this guide you'll have a GKE node pool on Z4M (z4m-highmem-48), fenced off with a taint and checked with a test Pod, and you'll know which features still need a newer GKE version.

What the Z4M machine types give you

The Compute Engine storage-optimized docs list these VM types. Titanium SSD is attached automatically, in 14,000 GiB units.

| Machine type | vCPUs | Memory (GB) | Titanium SSD (GiB) | | --- | --- | --- | --- | | z4m-highmem-16 | 16 | 122 | 14,000 | | z4m-highmem-32 | 32 | 244 | 28,000 | | z4m-highmem-48 | 48 | 366 | 42,000 | | z4m-highmem-192 | 192 | 1,488 | 168,000 |

There is also a z4m-highmem-192-metal type. GKE does not support it, more on that below.

Step 1: Check the GKE version and zone

Per the GKE release notes, manually created Z4M node pools need GKE 1.36.3-gke.1767000 or later. Compute Engine says Z4M is only in "select zones and regions", so check both before you build anything.

# Versions available in your location
gcloud container get-server-config --location=<region>

# Zones that offer the machine type
gcloud compute machine-types list --filter='name=z4m-highmem-48'

The first command prints the versions your location offers. The second lists every zone with z4m-highmem-48. Pick your zone from that list, not from habit.

Step 2: Put the cluster on a supported version

Use a Standard cluster, since this guide creates the pool by hand. Pick a version from the get-server-config output that is 1.36.3-gke.1767000 or later.

# New cluster
gcloud container clusters create <cluster-name> \
  --location=<region> \
  --cluster-version=<1.36.3-gke.1767000-or-later>

# Or upgrade the control plane of an existing one
gcloud container clusters upgrade <cluster-name> \
  --location=<region> \
  --master \
  --cluster-version=<1.36.3-gke.1767000-or-later>

The control plane has to be on the supported version before the pool can use it.

Step 3: Create the GKE node pool on Z4M

gcloud container node-pools create z4m-pool \
  --cluster=<cluster-name> \
  --location=<region> \
  --node-locations=<zone-that-has-z4m> \
  --machine-type=z4m-highmem-48 \
  --num-nodes=1 \
  --disk-type=hyperdisk-balanced \
  --disk-size=100 \
  --node-labels=workload=storage-heavy \
  --node-taints=workload=storage-heavy:NoSchedule

--node-locations pins the nodes to a zone that has Z4M, and --num-nodes counts per zone. --disk-type=hyperdisk-balanced sets the boot disk: the Z4M page lists Hyperdisk types and Titanium SSD as the supported disks, with no Persistent Disk, while gcloud defaults to pd-balanced. The label and taint keep ordinary Pods off an expensive machine.

Titanium SSD comes with the machine type, so there is no count to set. GKE's Local SSD page documents --ephemeral-storage-local-ssd (emptyDir and container layers) and --local-nvme-ssd-block (raw block) for newer machine series. It shows no Z4M example, so try one flag on a one-node pool and confirm the result in Step 5.

Step 4: Verify the Z4M nodes

gcloud container node-pools describe z4m-pool \
  --cluster=<cluster-name> \
  --location=<region> \
  --format='value(config.machineType)'

kubectl get nodes -l cloud.google.com/gke-nodepool=z4m-pool \
  -L node.kubernetes.io/instance-type

Both should say z4m-highmem-48. If the pool fails to create, the usual suspects are the version and the zone.

Step 5: Run a test Pod on the Z4M node pool

apiVersion: v1
kind: Pod
metadata:
  name: z4m-check
spec:
  nodeSelector:
    cloud.google.com/gke-nodepool: z4m-pool
  tolerations:
  - key: workload
    operator: Equal
    value: storage-heavy
    effect: NoSchedule
  containers:
  - name: shell
    image: busybox:1.36
    command:
    - sh
    - -c
    - df -h /scratch; sleep 3600
    volumeMounts:
    - name: scratch
      mountPath: /scratch
  volumes:
  - name: scratch
    emptyDir: {}

Apply it with kubectl apply -f z4m-check.yaml, then read kubectl logs z4m-check. If /scratch is about 100G, your emptyDir sits on the boot disk and the Titanium SSD isn't backing it yet.

Automatic provisioning on Z4M needs GKE 1.37

Compute classes, cluster autoscaler, node pool auto-creation and Autopilot clusters only work with Z4M from GKE 1.37.0-gke.4713000. On 1.36.3 your pool is fixed-size, so resize it by hand with gcloud container clusters resize. On 1.37 you can request Z4M with a custom compute class:

apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: z4m-storage
spec:
  priorities:
  - machineType: z4m-highmem-48
  nodePoolAutoCreation:
    enabled: true
  whenUnsatisfiable: DoNotScaleUp

Pods opt in with nodeSelector: cloud.google.com/compute-class: z4m-storage.

Gotchas

• No bare metal. GKE's machine support page says it doesn't support -metal machine types, so z4m-highmem-192-metal is out.
• Regions are limited. Run the machine-types list check before you pick a cluster location.
• Maintenance can terminate nodes. The two docs disagree on the live migration threshold (42,000 GiB on the Compute Engine page, 48 TiB on GKE's Local SSD page), and Cloud RDMA disables migration. Design for Pod termination and rescheduling.
• Some things are off the table. Compute Engine lists no GPUs and no custom machine types for Z4M.
• Check pricing. A storage-heavy machine left idle still bills, so look up Z4M rates for your region first.

If your AI workload feeds a parallel file system, getting the version, zone and taint right on the first pool saves you a failed rollout and a very expensive idle node.