GKE dynamic slicing for TPUs: a hands-on guide to cheaper, more resilient AI inference
GKE dynamic slicing for TPUs is now generally available. Here's how it differs from static slice allocation, with a concrete configuration for an AI inference workload. You'll see where the cost and resilience gains come from and how to avoid over-provisioning TPU capacity.
Dynamic slicing changes how you plan TPU capacity on GKE. The static approach means one node pool per workload shape, sized on a guess. With dynamic slicing you give GKE a pool of identical TPU cubes, and it wires them into whatever slice each workload asks for at the moment that workload is scheduled. The model comes first here. After that you'll set it up for an inference workload: Terraform for the cluster and node pools, Kueue and a LeaderWorkerSet for the serving pods, and the kubectl and gcloud commands that tell you whether slices form and recover the way you expect.
Read the full article on LearnXops (free with a LearnXops account).