Config push vs poll: everyday use

PR #4320 (config_delivery_mode = "push") at 88f81d812 · one gateway, Docker driver, Postgres, 250 sandboxes, plus a 100-sandbox kind cluster with 2–3 gateway replicas · October 2026

Every change reached every targeted sandbox in both modes, and the in-sandbox checks matched the new configuration.

What changed since the last report

• The code. The previous report measured #4320 at 45fc53a10. Since then the delivery was simplified: one update carries the sandbox configuration and the provider environment it names, each session has one update in flight on its ordered stream, there are no priority lanes, and results are written as they arrive and only when they change. Supervisors also start by polling again, as on main, and switch to push once their session is accepted.
• The scenarios. The previous run stressed bursts: one command per sandbox for all 250 at once, and 20 global changes in 2 seconds. This run models everyday use: one change at a time to different sandboxes, steady background activity, an occasional fleet-wide change, sandboxes stopping, starting and being replaced, a gateway restart, and gateway replicas coming and going.

How we tested

• One gateway built from the PR, with the Docker compute driver and Postgres 16, on a 20-core arm64 host. Supervisor, sandbox and gateway images were built from the same commit. Poll mode uses the default 10-second interval; push reconciles every 30 seconds.
• A fleet of 250 sandboxes per mode, recreated for each mode. With 250 sandboxes the host stays almost fully busy (load average 200–300), so both modes run on a saturated machine.
• Latency is per sandbox: from the moment the CLI command was issued to the supervisor's own log line confirming the change was applied. Both modes apply through the same supervisor code, so the measurement is identical.
• One-at-a-time scenarios change 20 different sandboxes, 3 seconds apart, so every number is one change, not a burst. Scenario groups use different sandboxes.
• Inside the sandboxes we checked that allow and deny rules flipped, that rotated bearer tokens reached the upstream, that provider placeholders appeared and disappeared, that a stopped sandbox started with the policy edited while it was stopped, and that replacement sandboxes enforced their policy. Every check passed in both modes.
• Other work shares this host, so before every scenario the harness waited until no other build or test was running.

Apply latency, p99

All change scenarios

Sandbox lifecycle

Starting, replacing and restarting take the same time in both modes, because supervisors start by polling in both. Push only changes what happens once the session is up. A graceful gateway restart on Docker stops every sandbox and resumes them one at a time, so the fleet is back after about five minutes in both modes. Changes made right after the restart arrive as fast as before it.

Gateway replicas coming and going (Kubernetes)

A kind cluster on the same host with 100 sandboxes and a gateway Deployment that starts with 2 replicas. Changes go through a NodePort, so most of them land on a replica that does not own the sandbox's session and reach the owner through peer notification. Scaling out does not move established sessions, so a new replica starts with none and forwards the changes it receives. Sessions on a replica that stops or is lost reconnect to the others.

Idle cost, 250 sandboxes

Observations

• Fleet-wide changes are the slow push case. A global policy change or a credential rotation rebuilds every session's update, 20 builds at a time on one gateway, and resolves credentials again, so it reaches all 250 sandboxes in 1.4–1.7 s at p99. On Kubernetes, with 100 sandboxes spread over replicas that build in parallel, the same changes finish in under 200 ms.
• A frozen supervisor takes its workload down after 40 seconds. The sandbox runtime freezes the workload once its supervisor misses keepalives for 10 seconds and terminates it 30 seconds later. This is the same in both modes. The first attempt at the stall scenario used 40 seconds and lost one sandbox this way; the scenario now stalls for 20 seconds.
• CLI requests during a rolling restart. On Kubernetes, 1 of 60 CLI requests in poll mode and 2 of 60 in push mode failed with an HTTP/2 keep-alive timeout while the gateway Deployment was rolling. The request reached a pod that was shutting down and never got an answer. Scaling up, scaling down and losing a replica caused no failures. This is the CLI's own API path, which this PR does not change, so it is a separate follow-up: drain connections on shutdown, or retry safe requests.
• Idle request rate. The remaining 50 requests per second in both modes are provider readiness reports, one every 5 seconds per sandbox. Moving them onto the supervisor session is a follow-up.

Not covered here

• Several hosts. The Kubernetes run uses one kind node on the same host, so its fleet is 100 sandboxes. Images were already on the node, so replicas started in seconds; pulling images would make scale-ups and rolling restarts take longer.
• Interceptor provider-profile sources, time-based credential refresh, and Vault, which need fixtures we have not built.