Google Cloud boosts AI agent density with GKE sandbox

Users of Google Kubernetes Engine can increase the number of AI agents running on a single node with GKE Agent Sandbox, according to Google Cloud. Internal tests showed agent density could more than triple in a cost-focused setup.

The figures come from tests on an n2-standard-48 virtual machine using OpenClaw, an open-source agent framework. Google Cloud used them to show how orchestration can change the economics of running AI agents at scale.

AI agents differ from many conventional application workloads because they often run in bursts, then sit idle while waiting for user input or other triggers. When agents are assigned static compute resources, that pattern can leave infrastructure underused, with processors and memory reserved but doing no useful work.

Google Cloud presented the issue as a balance of security, reliability and cost. For teams running untrusted multi-agent workloads, one common approach is to place each agent in its own microVM, including setups based on Kata containers, to maintain a hardware-level isolation boundary.

That model adds overhead because each microVM carries its own guest operating system. In Google Cloud's testing, that setup reached its limit at 61 OpenClaw agents on a standard GKE node before health checks began failing regularly and reliability declined.

Using GKE Agent Sandbox, the same workload reached 88 OpenClaw agents on the same virtual machine, a 44% increase on fixed capacity, while performance remained broadly similar, according to the company.

GKE Agent Sandbox uses gVisor, an open-source secure container sandbox, rather than a full guest operating system for each workload. The goal is to preserve isolation for untrusted code execution while reducing the infrastructure overhead of heavier virtualised environments.

Google Cloud also said the change lowered cost per agent by more than 30% in its tests. It added that usage of GKE Agent Sandbox rose more than sevenfold in less than four weeks after the service reached general availability in May.

Orchestration layer

The larger gains in the test came not from container isolation alone, but from orchestration. Idle agents can be checkpointed to persistent storage using GKE Pod snapshots, allowing their CPU and memory to be released back to the cluster until new work arrives, Google Cloud said.

When a new task appears, a Kubernetes controller or event gateway can trigger the system to restore the agent from a saved snapshot. Google Cloud said that process takes milliseconds, allowing operators to oversubscribe physical compute resources based on expected usage patterns rather than peak allocation.

The company said this is most useful when agent activity is intermittent. Instead of keeping every agent live at all times, operators can choose different recovery and latency profiles depending on the service being run.

Google Cloud grouped those workloads into three broad categories. Real-time coding assistants are latency-sensitive and need sub-second startup with little tolerance for queueing. Interactive background agents can accept startup times of a few seconds. Scheduled research or analysis agents can tolerate much longer waits if that reduces infrastructure costs.

To manage those differing needs, GKE supports multiple configurations across node pools and workload settings rather than a single cluster-wide policy. Google Cloud highlighted two features in particular: Agent Sandbox warm pools, which keep pre-warmed isolated sandboxes ready for fast execution, and suspend-and-resume functions for restoring agents on demand.

Test results

In a performance-focused setup using warm pools, Google Cloud said it ran 133 OpenClaw agents on the same node. In a cost-focused configuration, using higher oversubscription for more latency-tolerant workloads, it ran 274 agents on the same node while keeping startup times below five seconds.

That compares with the 61-agent baseline on microVMs and the 88-agent figure for Agent Sandbox without the broader orchestration model. Google Cloud said the combined approach can deliver up to 3.5 times greater agent density and reduce per-agent costs by as much as 75% for workloads that spend long periods idle.

The figures point to a broader shift in infrastructure design as companies move from running single AI services to managing growing fleets of software agents. For platform teams, the challenge is not only how to isolate these systems securely, but also how to avoid paying for idle capacity when usage is uneven and unpredictable.

Google Cloud's test case focused on OpenClaw, but it also cited Hermes and other open-source frameworks as examples of agent software that teams may first deploy on virtual machines before moving to more tightly managed production environments. Its argument is that orchestration should be part of the architecture from the outset rather than added later as scale problems emerge.

Another issue is the so-called thundering herd problem, in which large numbers of agents wake at once and demand compute simultaneously. Google Cloud said operators can tune the balance between cost and response times by deciding how much spare capacity to hold in warm pools and how aggressively to suspend idle workloads.

That leaves customers with a sliding scale rather than a fixed template. In Google Cloud's tests, the top-end result was 274 OpenClaw agents on one node while keeping startup times under five seconds.