Google AX: ★10,503, Apache-2.0, Kubernetes for agent workloads — and the one primitive I already needed

Google open-sourced google/ax in March 2026 and it has ★10,503 today. The one-line description is "Declare an agentic task with workspaces and model specifications" — and if you have run agents in production for more than a week, reading the spec feels like reading your own incident log written down properly.

Because the hard problems it names are not the model. They are the operational ones: an agent that burns money in a loop if nobody is watching, a sandbox that has to be isolated, a setup step every agent repeats, a model key that lives in nine different places.

I run a three-agent fleet on one Windows laptop. I solved all four of those by hand, with scripts. AX solves them with a schema. That difference is the entire point of this post.

🚀 If you only read three lines

1 · The primitives are the news. Task, Workspace, Model — three resources, declared as YAML, where token and timeout budgets and approval policies are fields in the spec. That is the guardrail pattern, formally specified.

2 · Don't install it unless you have a cluster. It needs Kubernetes, Redis, ko, a container registry, Go 1.27+ and a reachable Agent Substrate control API. It is v1alpha1 and Google's own README warns of "major breaking changes prior to a stable release."

3 · The real story is what this says about the field. A year ago "agent framework" meant a Python library. Google just shipped one as infrastructure — with a budget field and a suspend verb. That is the direction of travel, and it validates the guardrail work a lot of us did the hard way.

What AX actually is

AX is a declarative orchestrator for agent workloads. The framing is Kubernetes, explicitly: "If you have used Kubernetes, ax will feel similar." The README claims a design target of billions of tasks per cluster, built on a separate project, agent-substrate/substrate (★3,725, Apache-2.0, created May 2026), which does the sandboxed execution.

Those two numbers are worth separating. Billions per cluster is a scale claim about the scheduler, not about what your agents can do. Nothing about AX makes an agent smarter — it makes many agents isolated, resumable, and accountably budgeted.

Three primitives carry the whole design:

🧩 Task — the smallest unit of isolated execution

Not one agent. Deliberately small: a container image, a command, compute requests and limits, env vars, and references to workspaces. The concepts doc is unusually clear about why — "An agent is not one process that runs to completion; over its lifetime it plans, delegates, retries, and fans work out." So AX refuses to model that shape. It gives you one cheap thing and lets the agent compose as many as it needs. A single task can be the root of a tree.

Lifecycle is two fields: status.phase (Running, Suspended, Failed, Terminating) and conditions — WorkspaceReady and Ready, the latter being the one to wait on.

📦 Workspace — setup done once, declaratively

This is the most immediately useful idea in the repo. Before an agent's first useful action it needs repos cloned at the right revision, the tools it is allowed to call, and the skills it should carry. Every task repeats that setup, and every framework reinvents it.

A Workspace declares it once: git repositories, MCP servers and registries, and skill registries with the path they materialize to. Bind it from as many tasks as you like.

The detail that made me stop: a binding can carry a goal — plain language. On first boot the runner hands that goal to an agent that finishes the setup itself, installing a toolchain or dependencies, so the task's real command starts in a ready environment. Setup-by-agent, declared as a field.

🔑 Model — config as a resource, not an env var

The naming is a little confusing and AX knows it: "A Model is not a model." It is a named provider binding — provider, model identifier, generation parameters, and a reference to the Kubernetes secret holding the API key.

The argument is operational and correct: "The configuration lives in one place instead of in every agent's environment, so rotating a key, pinning a new model version, or tightening a parameter is one ax apply rather than a hunt through task definitions."

The CLI follows the same discipline — deliberately kubectl-shaped: apply, get, describe, watch, delete, plus three agent-specific verbs: suspend (checkpoint and pause), resume, and ssh (shell into a running sandbox, requires spec.debug: true).

The pattern I built the hard way

Here is why this repo landed differently for me than the last five agent frameworks.

My fleet has a treasurer agent. It exists for exactly one reason: agents burn money in loops. So before a production run is allowed to spend anything, a gate checks the budget, and if the balance is wrong the run is blocked. I wrote that by hand, as a script, and felt quite good about it.

Then I audited the fleet and found the gate had blocked 50 consecutive production runs and nothing had told me. No error surfaced. No alert fired. The job reported success and did nothing, fifty times, while I assumed the pipeline was working. The guardrail was correct and the observability around the guardrail did not exist, so the guardrail became a silent outage.

AX declares that same idea as a field in a spec — token and timeout budgets and approval policies on Task, with a roadmap entry to make their lifecycle phases and status conditions first-class. Which is the better design, and it is not close, because of one word: declared. My budget rule lives in a script I can only interrogate by running it. A declared budget lives in a resource you can ax get, ax describe, and ax watch. A guardrail you cannot observe is a guardrail you will eventually discover the hard way, at fifty runs.

⚠️ The honest caveat

The field exists on the roadmap for stabilization, which means the current v1alpha1 schemas are still moving. Recent commit history makes that concrete rather than theoretical: the last three days include "Remove Gateway concept to avoid bifurcation with Substrate" and "Remove temperature parameter from Model examples" — a concept deleted and a field removed, in a repo with ★10,503. Treat the shapes as a reference, not an install target.

Why I'm not installing it

I want to be precise, because "just use Kubernetes" is the wrong conclusion and so is "it's overkill." Both are lazy.

The requirements, read from the docs rather than assumed: a Kubernetes cluster and kubeconfig, ko for building and deploying control plane images, Docker or Podman for the task runner image, a container registry the cluster can pull from, Go 1.27+, and a reachable Agent Substrate Control API (in-cluster default api.ate-system.svc.cluster.local:443). make deploy lands Redis plus the control plane in the ax-system namespace.

My entire fleet is three agents on one laptop. There is no cluster to schedule onto, and the orchestrator's whole value — isolation between untrusted workloads at density — is a problem I do not have. Adopting the control plane would mean running Kubernetes to schedule three processes, which is the exact failure mode of importing infrastructure ahead of the problem it solves.

So: do not adopt it for a single-machine fleet. That is not a criticism of AX — it is the scale claim being real, and me not being at that scale. The other README line that matters is the warning, and I would hold any team to it: "We are still actively refining our core concepts, protocols, and specifications. We will likely to introduce major breaking changes prior to a stable release." Building your production lane on v1alpha1 is a bet you should take knowingly or not at all.

The four things worth stealing today

You do not need the cluster to take the design decisions. These are the parts I am copying, in order of how much they change my setup.

1 · Budgets and approval policies as declared fields

Make the spend limit and the human-approval requirement part of the workload's declaration, not a conditional buried in code. The practical version costs nothing: put the budget, the timeout and the approval requirement in the same config file that defines the task, then make the runner refuse to start if any of them is absent. A missing budget becomes a startup error instead of a silent fifty-run stall.

2 · A model registry as one resource

Provider, model ID, parameters and a secret reference — declared once, referenced everywhere. I already re-rank models centrally per role and route between providers; the improvement is making it a queryable object rather than a table I edit. When a key rotates or a version pins, that should be one edit, and it should be visible in one place.

3 · Workspaces with a natural-language goal

Declare the repos, MCP servers and skill paths, and let an agent complete the setup toward a written goal on first boot. This is the highest-leverage idea in the repo for anyone with a skill library, because it turns "clone, install, pray" into a declared state that either reaches ready or does not.

4 · Suspend and resume as real verbs

Checkpoint an idle agent and continue exactly where it stopped. Idle detection with automatic suspension is on the roadmap as a density play for clusters — but the single-machine version is just as valuable: stop paying for or occupying resources on an agent that is waiting, and make "waiting" a first-class state instead of a sleeping process.

What the roadmap tells you about where this goes

The roadmap is worth reading closely, because it is a preview of the shape agent infrastructure is converging on. Three items stand out:

Least-privilege actors, split by phase. AX plans to separate workspace setup from task execution into different actors, scoping repository and registry credentials exclusively to initialization while enforcing minimal runtime permissions and egress on the execution actor. That is a serious security model, and it is the correct answer to "the agent has my deploy key for the whole run."

Idleness detection and automatic suspension. Monitor process execution, I/O, network traffic and active sessions to spot idle tasks and checkpoint them to reclaim CPU and memory. Density engineering, declared in the platform rather than written per-agent.

Stateful task branching. Fork a running or suspended task — including its checkpointed memory and filesystem state — into several parallel tasks to explore speculative paths concurrently. This one is genuinely novel, and it is the first time I have seen a first-class primitive for "try three approaches at once from the same starting state."

Read together with the earlier three, the thesis is coherent: agents are a new kind of workload — not stateless services, not run-to-completion jobs — and they need a runtime that assumes state, isolation, spend and resumption. That is a much better description of what an agent is than any framework pitch I read last year.


Journal by Aldo Wen. Every star count, commit message, requirement and quotation on this page was read from the GitHub API and the repository's own documentation on 2026-09-25 — the spec is Google's, the receipts are checked, and the fifty blocked runs are mine.