Config push vs poll: load test

PR #4320 (config_delivery_mode = "push") · one gateway, Docker driver, Postgres · 250 sandboxes · October 2026 · tested at d74015e8c (45fc53a10 after the rebase onto main)

Every scenario converged on 100% of the targeted sandboxes in both modes, and in-sandbox checks (allow/deny flips, injected credentials, environment placeholders) matched the new configuration every time.

How we tested

• One gateway built from the PR, with the Docker compute driver and Postgres 16, on a 20-core arm64 host. Supervisor and gateway images were built from the same commit.
• A fleet of 250 sandboxes per mode, recreated for each mode so every supervisor ran in the mode under test. Poll mode uses the default 10-second interval.
• Latency is per sandbox: from the moment the CLI command was issued to the supervisor's own log line confirming the change was applied (Policy reloaded, Setting changed, provider environment change). Both modes go through the same supervisor apply path, so the measurement is identical for push and poll.
• Three sandboxes per scenario were also probed from inside: HTTP requests through the proxy for allow/deny, the echo of the injected bearer token, and environment placeholders.
• Gateway metrics went to Prometheus, traces to Jaeger, and container and host load to cAdvisor and node-exporter.
• We ran the matrix several times. The earlier rounds found scaling problems in push mode that only showed at 250 sandboxes: per-update writes for results and readiness queued on one gateway-wide lock and stalled user writes, provider attach/detach hit resource-version conflicts, and global setting changes rebuilt every provider environment. All were fixed in the PR, with gateway lock and database pool instrumentation used to find them. The numbers below are from the final run, after those fixes.

p99 propagation latency

All scenarios

For the "one command each" rows, latency is measured from each sandbox's own command, so it excludes the time to issue all 250 commands.

Idle cost, 250 sandboxes

Push replaces 25 config polls per second with a 30-second per-session reconcile (about 8 builds/s at 250 sandboxes, a few milliseconds each). The remaining 50 requests/s in both modes are provider readiness reports, one every 5 seconds per sandbox. Moving them onto the supervisor session is a follow-up.

Tuning and follow-ups

Push latency at this scale is set by how fast the gateway builds snapshots, not by the network. A change that rebuilds only the sandbox configuration reaches all 250 sandboxes in 930 ms; a change that also rebuilds every provider environment takes 1.5 s–1.7 s; a credential rotation, which also resolves credentials for every sandbox, takes 3.9 s. These can be improved in follow-ups or by tuning:

• Build slots. Concurrent snapshot builds are fixed at twice the database pool (20 on PostgreSQL), with a quarter reserved for single-sandbox changes. Making them configurable, or sizing them from the gateway's CPUs, would raise fan-out throughput.
• Database pool size. Fixed at 10 PostgreSQL connections today; #4147 makes it configurable, and the build slots scale with it.
• Shared fan-out inputs. Each session's build loads its provider records, global settings and policy on its own. A fleet-wide change could load the shared inputs once per publication.
• Credential resolution. Each sandbox's provider environment build resolves its providers' credentials, so a rotation resolves the same credential once per attached sandbox. Resolving it once per publication would shorten rotations.
• Lock scope. Supervisor result writes and user writes still share one gateway-wide mutation lock; #4147 scopes it per sandbox, workspace and global change.
• Fan-out ordering. Concurrent fan-outs are served first come, first served. Round-robin across workspaces would keep one large workspace from delaying the others.
• Repair timing. The 30-second reconcile interval, the 1–30 second build retry backoff and the 1-minute acknowledgement timeout are constants. They bound how quickly a missed or failed delivery is repaired, and could become settings.

Not covered here

• Multiple gateway replicas. A second run on Kubernetes with three gateways covers peer notification, replica loss, and session reconnection.
• Gateway restarts. With the Docker driver a graceful restart stops and restarts every sandbox, so it measures the driver rather than config delivery.
• Interceptor provider-profile sources and time-based credential refresh, which need fixtures we have not built yet.