The internet connected billions of people and mobile devices, putting computers in every hand. Now, we’re in the middle of the next big technology shift, deploying millions of autonomous AI agents to work alongside employees and end users. Today, we announced new FinOps controls for Gemini Enterprise to help organizations manage project-level AI spend and eliminate token shock. But the sheer scale of the agentic era is placing new constraints at every layer of the stack, including infrastructure. AI workloads are notoriously difficult to architect, resource-intensive, and bursty, which can also lead to scaling bottlenecks and large pools of underutilized — or misutilized — compute resources.
Organizations need insights to help them extract more value from their infrastructure investments. In this blog, we outline best practices for dynamic capacity management — scheduling and utilization strategies to help you run enterprise and AI applications on a single, flexible foundation with predictable cost and performance. These capabilities are designed to augment our on-demand, Spot and committed use discount (CUD) consumption models, which provide flexible pricing and discounting for your workloads. Let’s jump in.
Here’s a quick summary
Three ways you can implement dynamic capacity management:
-
Schedule capacity for planned events. Schedule mission-critical resources (GPUs, TPUs and select VM families) ahead of planned events using calendar mode, or optimize costs for batch jobs with flexible start times using flex-start mode in Dynamic Workload Scheduler. Once you obtain the capacity, those resources are guaranteed for the specified duration.
-
Maintain service continuity by creating a fallback plan for every application. Define automated, prioritized hardware fallback lists using managed instance groups (MIGs) so your apps automatically pivot to the next approved compute option when your preferred option isn’t available.
-
Automate your entire capacity management lifecycle on a single, adaptive control plane. Google Kubernetes Engine (GKE) provides an agent-native environment to orchestrate the entire process — from fallback lists using Custom ComputeClasses, to granular hardware slicing with dynamic resource allocation, so agents can rapidly spin up in secure sandboxes and containers while it dynamically reallocating resources on the fly.
Why architectural flexibility matters
Ninety percent of enterprises want to deploy agents within the next three years, but only 17% of IT leaders feel confident their current IT setup can handle the load. Because these workloads have unique performance needs, organizations are racing to adopt specialized infrastructure, including accelerators (GPUs, TPUs) and CPUs with customized compute, memory, and storage ratios. However, agents also require access to enterprise applications and databases — often at a volume and scale that vastly exceeds typical human usage. Handling the intense demands of both agents and the applications they interact with requires a dynamic infrastructure. Infrastructure teams can leverage custom-designed processors like Google’s Axion to meet these needs, but hardware isn’t a complete solution. They also need ways to use that infrastructure wisely, solving execution inefficiencies to enable more flexibility across the stack.
How to overcome infrastructure constraints
Achieving this kind of flexibility requires a two-pronged approach: securing resources for the demand you can predict, and building automation to respond to the demand you can’t. Combining the two, you can preschedule capacity for planned events and your infrastructure can adapt to unexpected changes without manual intervention.
1. Schedule capacity for planned events
You can secure mission-critical capacity ahead of scheduled milestones, offline training, or anticipated demand surges using Dynamic Workload Scheduler. By scheduling the resources you need up front, you optimize your spend and ensure you get access to the compute resources you need. Dynamic Workload Scheduler supports hardware accelerators (TPUs and GPUs) and select CPUs with two distinct modes:
-
Flex-start mode: Use this for latency-tolerant workloads like batch processing, model training, or offline fine-tuning. Instead of requiring resources immediately, you submit a defined duration request and the system intelligently queues your job, provisioning the resources as soon as capacity becomes available. This maximizes cost-efficiency and drastically improves your ability to obtain high-demand accelerators.
-
Calendar mode: Use this for mission-critical, time-bound events like a major product launch, a scheduled migration, or a seasonal traffic surge. By specifying the exact start and end dates of your event, you create a future reservation. This guarantees the requested capacity will be available when the event begins.
