Reviewer walkthrough · supervisor middleware

HTTP hooks: streaming request and response middleware

The PR replaces the legacy HTTP middleware hooks with two bidirectional streaming RPCs, so middleware can inspect and rewrite bodies of any size, set headers from what it read in the body, and process response streams without lockstep.

Streaming APIHttpHooks.EvaluateRequest and EvaluateResponse, one stream per HTTP message and stage. A preflight picks continue, deny, or inspect (WHOLE_BODY or DUPLEX).
Opt in by capabilitySame HTTP_REQUEST / HTTP_RESPONSE bindings. A service that requires openshell.supervisor-middleware.http-hooks gets the new RPCs. No new HTTP operations or versioned enums.
Always fail-closedon_error: fail_open is rejected for HTTP hook services.
Uninspectable traffic unchangedtls: skip, h2c, and raw TCP connections never reach middleware. As on main, a selected fail_closed entry refuses them.
No ports in selectorsSelector patterns that contain : are rejected, except IPv6 literals. A host:port pattern never matched anything.
Legacy hooks deprecatedEvaluateHttpRequest and HttpResponsePreReturn keep working and are removed in 0.2.0.

github.com/NVIDIA/OpenShell/pull/4359

In this deck

Why a new API
Before and after
Shape of the new API
How a service selects the hook version
Migrating a legacy service
Next iterations
Preflight and body eligibility rules
Chain ordering and when the head commits
DUPLEX stalls and deadlines
Failure semantics per phase
Code map for the diff

01 · Motivation

Why a new API instead of extending the legacy hooks

The legacy hooks were built for small, buffered checks. The use cases we want next, such as signing a Git push or a large AWS request, need the whole body regardless of size, and need to change the body and its headers together.

What the legacy hooks can't do

Inspect large bodiesEvaluateHttpRequest is unary: the whole body in one message, at most 4 MiB, under one RPC deadline (500 ms default, 30 s max). A larger request fails with request_body_over_capacity, so a Git push or an S3 upload is blocked or, with fail_open, forwarded uninspected.HTTP hooks: DUPLEX has no total size or time cap. The 30 s timeout covers only time the service holds things up.
Set headers from the bodySigning reads the whole body, then rewrites it and the headers that describe it: a new Content-Length for a re-signed Git pack, a payload hash for SigV4. Legacy responses commit headers at preflight, before any body unit; legacy requests can do it only inside the 4 MiB buffer.HTTP hooks: output_start carries header mutations and may come after input_end.
Process a response stream freelyHttpResponsePreReturn is lockstep. Each body unit of up to 64 KiB gets exactly one result that fully accounts for it, and the service can't keep bytes for a later unit. It can't redact a token split across two units, decompress, or re-frame server-sent events.HTTP hooks: in DUPLEX, N input chunks and M output chunks are independent, and the stage owns every output byte.
Share one model across directionsRequests are a unary decision with an optional body replacement. Responses are a preflight, body-mode units, and trailers. A service implements two contracts with different results, limits, and errors.HTTP hooks: one lifecycle, preflight → begin → body → finish or deny, with the same body modes and limits in both directions.

Why not extend the legacy RPCs

A unary RPC can't become a streamChanging EvaluateHttpRequest to a streaming signature breaks every deployed service on the wire. Streaming requests need a new RPC either way.
Lockstep is a guarantee, not a fieldResponse services rely on one result per unit, and OpenShell relies on it to fail open from the last unit it still owns. Relaxing that on the same RPC changes behavior without changing the schema, so buf breaking passes and old and new peers silently disagree.
Old peers must refuse, not misreadAn older supervisor ignores unknown fields and enum values. It would call an extended service with legacy semantics. A separate service that the manifest marks as a required capability makes older gateways and supervisors refuse it at Describe.
fail_open stops making senseOnce a stage has emitted output for bytes it consumed, there is no unmodified input left to forward. Removing fail_open from the legacy hooks would break existing policies; a new contract can start fail-closed.

So the PR adds the HTTP hooks next to the legacy hooks, keeps both working through 0.1.x, and removes the legacy hooks in 0.2.0.

Validated with two prototypes on an earlier head of this PR

DUPLEX request stage

Git commit signing · pimlock/OpenShell#2

An external service selects DUPLEX for git-receive-pack, stores the push in a temporary file until input_end, signs the new commits with an SSH key that never enters the sandbox, and streams back the rewritten pack with its new length. The test pushes more than 4 MiB through Git's real receive-pack and checks both signatures with git verify-commit.

signing after HTTP hook stages

SigV4 signing · pimlock/OpenShell#3

A trusted built-in signs the final request after every external HTTP hook stage has transformed the body, with supervisor-resolved AWS credentials that never enter the middleware protocol. OpenShell buffers up to 10 MiB of output for signing and reports that cap to DUPLEX stages as max_output_body_bytes.

02 · High level

Same place in the proxy, different conversation with the service

The middleware still sits between network-policy admission and credential injection for requests, and before delivery for responses. What changes is how the supervisor talks to it, and which traffic it can see.

press v to toggle

What changes

	Legacy hooks	HTTP hooks
Body size	≤ 4 MiB, buffered	WHOLE_BODY ≤ 4 MiB or unbounded DUPLEX
Request vs response	Two unrelated contracts	One lifecycle; an event/result pair per RPC
Who chooses body handling	Request: always full body. Response: mode at preflight	Service, at preflight, from modes OpenShell permits
`tls: skip`, h2c, raw TCP	Not shown to middleware; `on_error` decides	Unchanged
Failures	`on_error` decides	Always fail-closed
Version selection	Default	Required capability in `Describe`

03 · Contract

Two streaming RPCs, one per HTTP direction

Requests and responses each have their own event and result messages, with the same lifecycle. The first event is always a preflight, and its result decides which of the remaining events the service will see.

proto/supervisor_middleware.proto

service SupervisorMiddleware {
  rpc Describe(...)        returns (MiddlewareManifest);
  rpc ValidateConfig(...)  returns (ValidateConfigResponse);
  rpc EvaluateHttpRequest(HttpRequestEvaluation)
      returns (HttpRequestResult);       // deprecated
  rpc EvaluateWebSocketSession(...)          // unchanged
}
service HttpHooks {
  rpc EvaluateRequest(stream HttpRequestHookEvent)
      returns (stream HttpRequestHookResult);
  rpc EvaluateResponse(stream HttpResponseHookEvent)
      returns (stream HttpResponseHookResult);
}
service HttpResponsePreReturn { ... }      // deprecated

message HttpRequestHookEvent {
  oneof event {
    preflight      = 1;
    begin          = 2;
    whole_body  = 3;
    input_chunk    = 4;
    input_end      = 5;
    session_end    = 6;
  }
}

message HttpRequestHookResult {
  oneof result {
    preflight_result = 1;
    whole_body_result  = 2;
    output_start     = 3;
    output_chunk     = 4;
    finish           = 5;
    deny             = 6;
  }
}
// responses: block_delivery = 6

message HttpRequestHookPreflight {
  HttpRequestPreflightHead request;   // responses: response head
  phase, context, middleware_name, config
  repeated HttpBodyMode permitted_body_modes;      // WHOLE_BODY, DUPLEX
  repeated HttpBodyModeUnavailable unavailable_body_modes; // mode + reason
  HttpBodyLimits limits;                // chunk, buffer, idle, output cap
  optional uint64 declared_body_bytes;
}

Exchange explorer · pick what the service decides at preflight

OpenShell → serviceservice → OpenShell

04 · Compatibility

A required capability selects the hook version

The bindings stay HTTP_REQUEST and HTTP_RESPONSE. Gateways and supervisors have rejected unmet required_capabilities at Describe since v0.1.0, so an older peer refuses an HTTP hook service instead of calling it through the legacy RPCs.

Describe → MiddlewareManifest

bindings { operation: HTTP_REQUEST   phase: PRE_CREDENTIALS }
bindings { operation: HTTP_RESPONSE  phase: PRE_RETURN }
bindings { operation: WEBSOCKET_MESSAGE ... }   // unaffected
extension {
  supported_capabilities: "openshell.supervisor-middleware.http-hooks"
  required_capabilities:  "openshell.supervisor-middleware.http-hooks"
}

Rust helper

extension: Some(http_hooks_middleware_metadata(
    MANIFEST_NAME,
    openshell_core::VERSION,
)),   // openshell_core::extension_protocol

The capability applies to every HTTP binding of the service. To serve both versions during a rollout, run two registrations.
A binding may set max_payload_bytes: 0. It then gets no body modes and can only continue or deny.

What happens for each manifest and peer

Service manifest

Gateway / supervisor before this PR

Gateway / supervisor with this PR

No capabilitytoday's legacy services

legacy hooksWorks as today.

legacy hooksWorks, deprecated. Removed in 0.2.0.

Supported + requiredthe migration target

refused at DescribeUnmet required capability. Gateway does not start; old supervisors refuse the service. Safe: never called through the legacy RPCs.

HTTP hooksCalls HttpHooks.

Supported, not requiredthe trap

would call legacy RPCsAn old peer accepts it and calls RPCs the service no longer implements.

registration rejectedThe new gateway refuses this manifest to close the hole on the left.

No RPC, message, or enum value carries a version number. buf breaking against v0.1.2 reports no breaking change.

05 · Migration

Moving a service from the legacy hooks to HTTP hooks

Both hook versions run side by side until 0.2.0. The Supported Operations docs page, docs/extensibility/supervisor-middleware/operations.mdx, describes the HTTP hooks for users.

Steps for a service owner

Require the capabilityAdd openshell.supervisor-middleware.http-hooks to both supported_capabilities and required_capabilities. Keep the bindings.
Implement the HTTP hook RPCsServe HttpHooks on the same endpoint, with EvaluateRequest / EvaluateResponse for the bindings you declare; drop the legacy RPCs.
Make policy entries fail-closedChange on_error: fail_open to fail_closed. The gateway rejects fail_open for HTTP hook services.
Check uninspectable trafficA fail-closed entry refuses tls: skip, h2c, and raw TCP connections to the hosts it selects, and the gateway rejects its overlap with tls: skip endpoints. Exclude hosts that need such connections from the selector.
Separate selectors from legacy entriesEndpoint selectors must not overlap legacy HTTP hook entries for the same direction, including built-in openshell/regex.
Roll out gateway firstUpgrade the gateway, then deploy the service. Recreate sandboxes whose supervisors predate this PR.

Legacy hooks → HTTP hooks

Legacy		HTTP hooks
`EvaluateHttpRequest`	→	`inspect: whole_body`
`Decision DENY`	→	`deny`
`HEADERS_ONLY`	→	`continue_without_body` + header mutations
`WHOLE_BODY_BYTES`	→	`inspect: whole_body`
`STREAM_BYTES`	→	`inspect: duplex`
`block_delivery`	→	`block_delivery`
`on_error: fail_open`	→	not allowed

New gateway policy checks

Overlapping selectors that mix hook versions for one direction are rejected. If a mixed chain still occurs at runtime, traffic is blocked with middleware_hook_versions_mixed.
As on main, a fail_closed entry cannot select tls: skip endpoints. Gateways and supervisors both enforce this.
Selector patterns that contain : are rejected, except IPv6 literals. A stored host:port pattern now fails validation; drop the port, since it never matched.
Stored policies with fail_open still load; supervisors run their HTTP traffic fail-closed once they reach the service.

This PRHTTP hooks available; legacy hooks marked deprecated.

0.1.xBoth versions work side by side.

0.2.0Legacy RPCs and legacy-only types removed.