Reviewer walkthrough · supervisor middleware

HTTP session hooks: streaming request and response middleware

The PR replaces the v1 HTTP middleware hooks with two bidirectional streaming RPCs, so middleware can inspect and rewrite bodies of any size, set headers from what it read in the body, and process response streams without lockstep.

Streaming APIHttpSessionMiddleware.EvaluateRequest and EvaluateResponse, one stream per HTTP message and stage. A preflight picks continue, deny, or inspect (BUFFERED or STREAM).
Opt in by capabilitySame HTTP_REQUEST / HTTP_RESPONSE bindings. A service that requires openshell.supervisor-middleware.http-session gets the new RPCs. No new HTTP operations or versioned enums.
Always fail-closedon_error: fail_open is rejected for session hook services.
Uninspectable traffic unchangedtls: skip, h2c, and raw TCP connections never reach middleware. As on main, a selected fail_closed entry refuses them.
No ports in selectorsSelector patterns that contain : are rejected, except IPv6 literals. A host:port pattern never matched anything.
v1 deprecatedEvaluateHttpRequest and HttpResponsePreReturn keep working and are removed in 0.2.0.

github.com/NVIDIA/OpenShell/pull/4359

In this deck

Why a new API
Before and after
Shape of the new API
How a service selects the hook version
Migrating a v1 service
Next iterations
Preflight and body eligibility rules
Chain ordering and when the head commits
STREAM stalls and deadlines
Failure semantics per phase
Code map for the diff

01 · Motivation

Why a new API instead of extending the v1 hooks

The v1 hooks were built for small, buffered checks. The use cases we want next, such as signing a Git push or a large AWS request, need the whole body regardless of size, and need to change the body and its headers together.

What the v1 hooks can't do

Inspect large bodiesEvaluateHttpRequest is unary: the whole body in one message, at most 4 MiB, under one RPC deadline (500 ms default, 30 s max). A larger request fails with request_body_over_capacity, so a Git push or an S3 upload is blocked or, with fail_open, forwarded uninspected.Session hooks: STREAM has no total size or time cap. The 30 s timeout covers only time the service holds things up.
Set headers from the bodySigning reads the whole body, then rewrites it and the headers that describe it: a new Content-Length for a re-signed Git pack, a payload hash for SigV4. v1 responses commit headers at preflight, before any body unit; v1 requests can do it only inside the 4 MiB buffer.Session hooks: output_start carries header mutations and may come after input_end.
Process a response stream freelyHttpResponsePreReturn is lockstep. Each body unit of up to 64 KiB gets exactly one result that fully accounts for it, and the service can't keep bytes for a later unit. It can't redact a token split across two units, decompress, or re-frame server-sent events.Session hooks: in STREAM, N input chunks and M output chunks are independent, and the stage owns every output byte.
Share one model across directionsRequests are a unary decision with an optional body replacement. Responses are a preflight, body-mode units, and trailers. A service implements two contracts with different results, limits, and errors.Session hooks: one lifecycle, preflight → begin → body → finish or deny, with the same body modes and limits in both directions.

Why not extend the v1 RPCs

A unary RPC can't become a streamChanging EvaluateHttpRequest to a streaming signature breaks every deployed service on the wire. Streaming requests need a new RPC either way.
Lockstep is a guarantee, not a fieldResponse services rely on one result per unit, and OpenShell relies on it to fail open from the last unit it still owns. Relaxing that on the same RPC changes behavior without changing the schema, so buf breaking passes and old and new peers silently disagree.
Old peers must refuse, not misreadAn older supervisor ignores unknown fields and enum values. It would call an extended service with v1 semantics. A separate service that the manifest marks as a required capability makes older gateways and supervisors refuse it at Describe.
fail_open stops making senseOnce a stage has emitted output for bytes it consumed, there is no unmodified input left to forward. Removing fail_open from v1 would break existing policies; a new contract can start fail-closed.

So the PR adds the session hooks next to v1, keeps both working through 0.1.x, and removes v1 in 0.2.0.

Validated with two prototypes on an earlier head of this PR

STREAM request stage

Git commit signing · pimlock/OpenShell#2

An external service selects STREAM for git-receive-pack, stores the push in a temporary file until input_end, signs the new commits with an SSH key that never enters the sandbox, and streams back the rewritten pack with its new length. The test pushes more than 4 MiB through Git's real receive-pack and checks both signatures with git verify-commit.

signing after session stages

SigV4 signing · pimlock/OpenShell#3

A trusted built-in signs the final request after every external session stage has transformed the body, with supervisor-resolved AWS credentials that never enter the middleware protocol. OpenShell buffers up to 10 MiB of output for signing and reports that cap to STREAM stages as max_output_body_bytes.

02 · High level

Same place in the proxy, different conversation with the service

The middleware still sits between network-policy admission and credential injection for requests, and before delivery for responses. What changes is how the supervisor talks to it, and which traffic it can see.

press v to toggle

What changes

	v1 hooks	Session hooks
Body size	≤ 4 MiB, buffered	BUFFERED ≤ 4 MiB or unbounded STREAM
Request vs response	Two unrelated contracts	One lifecycle; an event/result pair per RPC
Who chooses body handling	Request: always full body. Response: mode at preflight	Service, at preflight, from modes OpenShell permits
`tls: skip`, h2c, raw TCP	Not shown to middleware; `on_error` decides	Unchanged
Failures	`on_error` decides	Always fail-closed
Version selection	Default	Required capability in `Describe`

03 · Contract

Two streaming RPCs, one per HTTP direction

Requests and responses each have their own event and result messages, with the same lifecycle. The first event is always a preflight, and its result decides which of the remaining events the service will see.

proto/supervisor_middleware.proto

service SupervisorMiddleware {
  rpc Describe(...)        returns (MiddlewareManifest);
  rpc ValidateConfig(...)  returns (ValidateConfigResponse);
  rpc EvaluateHttpRequest(HttpRequestEvaluation)
      returns (HttpRequestResult);       // deprecated
  rpc EvaluateWebSocketSession(...)          // unchanged
}
service HttpSessionMiddleware {
  rpc EvaluateRequest(stream HttpRequestSessionEvent)
      returns (stream HttpRequestSessionResult);
  rpc EvaluateResponse(stream HttpResponseSessionEvent)
      returns (stream HttpResponseSessionResult);
}
service HttpResponsePreReturn { ... }      // deprecated

message HttpRequestSessionEvent {
  oneof event {
    preflight      = 1;
    begin          = 2;
    buffered_body  = 3;
    input_chunk    = 4;
    input_end      = 5;
    session_end    = 6;
  }
}

message HttpRequestSessionResult {
  oneof result {
    preflight_result = 1;
    buffered_result  = 2;
    output_start     = 3;
    output_chunk     = 4;
    finish           = 5;
    deny             = 6;
  }
}
// responses: block_delivery = 6

message HttpRequestSessionPreflight {
  HttpRequestPreflightHead request;   // responses: response head
  phase, context, middleware_name, config
  repeated HttpBodyMode permitted_body_modes;      // BUFFERED, STREAM
  repeated HttpBodyModeUnavailable unavailable_body_modes; // mode + reason
  HttpBodyLimits limits;                // chunk, buffer, idle, output cap
  optional uint64 declared_body_bytes;
}

Exchange explorer · pick what the service decides at preflight

OpenShell → serviceservice → OpenShell

04 · Compatibility

A required capability selects the hook version

The bindings stay HTTP_REQUEST and HTTP_RESPONSE. Gateways and supervisors have rejected unmet required_capabilities at Describe since v0.1.0, so an older peer refuses a session hook service instead of calling it through the v1 RPCs.

Describe → MiddlewareManifest

bindings { operation: HTTP_REQUEST   phase: PRE_CREDENTIALS }
bindings { operation: HTTP_RESPONSE  phase: PRE_RETURN }
bindings { operation: WEBSOCKET_MESSAGE ... }   // unaffected
extension {
  supported_capabilities: "openshell.supervisor-middleware.http-session"
  required_capabilities:  "openshell.supervisor-middleware.http-session"
}

Rust helper

extension: Some(http_session_middleware_metadata(
    MANIFEST_NAME,
    openshell_core::VERSION,
)),   // openshell_core::extension_protocol

The capability applies to every HTTP binding of the service. To serve both versions during a rollout, run two registrations.
A binding may set max_payload_bytes: 0. It then gets no body modes and can only continue or deny.

What happens for each manifest and peer

Service manifest

Gateway / supervisor before this PR

Gateway / supervisor with this PR

No capabilitytoday's v1 services

v1 hooksWorks as today.

v1 hooksWorks, deprecated. Removed in 0.2.0.

Supported + requiredthe migration target

refused at DescribeUnmet required capability. Gateway does not start; old supervisors refuse the service. Safe: never called through v1.

session hooksCalls HttpSessionMiddleware.

Supported, not requiredthe trap

would call v1 RPCsAn old peer accepts it and calls RPCs the service no longer implements.

registration rejectedThe new gateway refuses this manifest to close the hole on the left.

No RPC, message, or enum value carries a version number. buf breaking against v0.1.2 reports no breaking change.

05 · Migration

Moving a service from v1 to session hooks

Both hook versions run side by side until 0.2.0. The Supported Operations docs page, docs/extensibility/supervisor-middleware/operations.mdx, describes the session hooks for users.

Steps for a service owner

Require the capabilityAdd openshell.supervisor-middleware.http-session to both supported_capabilities and required_capabilities. Keep the bindings.
Implement the session RPCsServe HttpSessionMiddleware on the same endpoint, with EvaluateRequest / EvaluateResponse for the bindings you declare; drop the v1 RPCs.
Make policy entries fail-closedChange on_error: fail_open to fail_closed. The gateway rejects fail_open for session hook services.
Check uninspectable trafficA fail-closed entry refuses tls: skip, h2c, and raw TCP connections to the hosts it selects, and the gateway rejects its overlap with tls: skip endpoints. Exclude hosts that need such connections from the selector.
Separate selectors from v1 entriesEndpoint selectors must not overlap v1 HTTP hook entries for the same direction, including built-in openshell/regex.
Roll out gateway firstUpgrade the gateway, then deploy the service. Recreate sandboxes whose supervisors predate this PR.

v1 → session hooks

v1		Session hooks
`EvaluateHttpRequest`	→	`inspect: buffered`
`Decision DENY`	→	`deny`
`HEADERS_ONLY`	→	`continue_without_body` + header mutations
`WHOLE_BODY_BYTES`	→	`inspect: buffered`
`STREAM_BYTES`	→	`inspect: stream`
`block_delivery`	→	`block_delivery`
`on_error: fail_open`	→	not allowed

New gateway policy checks

Overlapping selectors that mix hook versions for one direction are rejected. If a mixed chain still occurs at runtime, traffic is blocked with middleware_hook_versions_mixed.
As on main, a fail_closed entry cannot select tls: skip endpoints. Gateways and supervisors both enforce this.
Selector patterns that contain : are rejected, except IPv6 literals. A stored host:port pattern now fails validation; drop the port, since it never matched.
Stored policies with fail_open still load; supervisors run their HTTP traffic fail-closed once they reach the service.

This PRSession hooks available; v1 marked deprecated.

0.1.xBoth versions work side by side.

0.2.0v1 RPCs and v1-only types removed.