Bridge Protocol

The bridge protocol is the message format used between the requester and receiver after a client request enters the bridge. Read this page when you need to understand what is sent through NATS, how responses come back, how large streams are paced, and how streams or tunnels are stopped.

HTTP work is JSON-framed. TCP tunnel data uses binary NATS frames after an initial JSON session open request.

HTTP requests that do not fit in one NATS message automatically use the existing TCP session transport. The size check includes the serialized request envelope and NATS headers, using the connected server’s INFO.max_payload. Small requests retain the http_request path. Bodies that cannot be represented as JSON also use the binary path.

The gateway selects the transport with an explicit bridge_request_fits? check. A large body is a normal routing decision. PayloadTooLargeError is reserved for an actual attempt to publish an oversized message and is not used to switch transports.

For code-level debugging, the main implementation is split between BridgeProtocol and BridgeCore:

• BridgeProtocol builds and parses protocol objects.
• BridgeCore chooses subjects, publishes requests, validates envelopes, dispatches operation handlers, routes response events, handles flow-credit frames, and publishes cancellation envelopes.

Operations

Operation	Handler	Purpose
http_request	HttpGateway#handle_bridge_request	Forward an HTTP request to UPSTREAM_URL or to an absolute proxy target.
tcp_stream	TcpTunnelBridge#handle_stream_request	Open a TCP connection for HTTP CONNECT, SOCKS5, or an oversized HTTP request, then exchange binary session frames.

Handlers are registered in ServiceRuntime#boot!. If a receiver gets an unknown operation, both backends publish a terminal HTTP 503 response; JetStream acknowledges this permanent refusal without retrying it.

Request Envelope

BridgeProtocol.request_envelope builds the JSON envelope sent by the requester:

{
  "type": "request",
  "request_id": "req-id",
  "reply_to": "from.proxy.responses.requester-service.req-id",
  "operation": "http_request",
  "payload": {
    "method": "GET",
    "path": "/path",
    "headers": {},
    "body": null
  }
}

Required fields are request_id, reply_to, operation, and payload. BridgeCore validates these fields before dispatching to a registered handler.

For HTTP requests, payload.path can be:

• a normal origin-form path such as /v1/models, forwarded relative to UPSTREAM_URL;
• an absolute HTTP URL such as http://example.com/path, used by HTTP proxy traffic.

HttpGateway strips hop-by-hop request headers before forwarding. GET and HEAD do not send a request body. Other methods carry the original HTTP body as a string, preserving whitespace, JSON scalars and quoting. Rack body strings are tagged as UTF-8 without transcoding: small valid UTF-8 bodies retain the JSON path, while invalid UTF-8 bodies use the TCP tunnel. The receiver still accepts legacy parsed JSON bodies.

Oversized HTTP Requests

The fallback opens tcp_stream with ingress_kind=http_gateway, the original HTTP method and request target, and no body in the session-open envelope. ServiceRuntime routes this ingress to HttpGateway#handle_tunnel_request. The HTTP adapter resolves the target using the same UPSTREAM_URL or absolute-form URL rules as the JSON bridge, then delegates ready host/port values to TcpTunnelBridge. It enriches session_established with http_url, the resolved URL, and http_headers, the upstream connection’s default headers, including URL-based Basic Auth. Client request headers override these defaults as on the JSON bridge. Diagnostic captures redact http_headers.

On the requester, HttpTunnel connects Faraday’s existing Async HTTP adapter to a local socket pair. The other end runs the same requester tunnel pump used by CONNECT/SOCKS5. Complete HTTP requests and responses, including TLS for HTTPS targets, cross NATS as existing binary session frames with existing byte-credit flow control. The HTTP library handles message framing; no new body-chunk protocol or end marker is required. Raw request bodies are preserved. The HTTP client performs one transport attempt, so an interrupted POST is not automatically replayed.

Both HTTP transports use native per-I/O timeouts instead of a fixed total HTTP deadline. Ordinary upstream connections configure the Async HTTP adapter’s read_timeout from the hardcoded HttpGateway::HTTP_READ_TIMEOUT (300 seconds). HttpTunnel receives the same value and applies it to its socket pair’s HTTP-client end and its fixed endpoint, including TLS. This timeout is independent of STREAM_RESPONSE_TIMEOUT. An active response can outlive the timeout value; a blocked I/O operation times out. Existing response-start/event waits, flow credit and cancellation remain in effect. Direct CONNECT/SOCKS5 sessions retain their existing lifetime.

The local HTTP response queue is bounded by the configured initial flow window and chunk size. A slow downstream client stops the HTTP adapter from reading more tunnel bytes, so existing TCP credits propagate backpressure to the upstream. The HTTP adapter requests inline execution through the TCP handler’s existing detached option, retaining a RECEIVER_MAX_INFLIGHT slot until the HTTP tunnel finishes. CONNECT and SOCKS5 sessions keep their detached lifecycle. HTTP target resolution, HTTP metadata and permanent refusal responses stay in HttpGateway; the core and TCP transport do not branch on the HTTP ingress kind.

Update requester and receiver together before relying on this fallback. The new ingress kind requires receiver-side target resolution; existing CONNECT/SOCKS5 frames and operations remain compatible. A receiver without this support rejects the session instead of forwarding an empty HTTP request. An oversized session-open envelope is rejected with HTTP 413 containing the actual NATS message size and limit. Connection failures and establishment timeouts return diagnostic gateway errors.

The NATS limit is per message, not per HTTP body or flow-credit window. Each broker on a leafnode route must accept the selected frame size; local INFO.max_payload does not describe every intermediate broker. Request bodies are still buffered by the HTTP gateway before routing, as on the existing JSON path.

Response Events

HTTP responses are returned as ordered JSON events on reply_to.

Event	Payload
response_start	HTTP status, normalized response headers, content_type, streaming, and owner metadata such as receiver_service_id, request_id, and flow_kind.
response_chunk	body for valid UTF-8 chunks, or body_encoding=base64 and body_base64 for binary chunks.
response_error	error string plus an explicit outcome (upstream_error, timeout, or canceled).
response_end	Terminal marker for the HTTP response.

text/event-stream and application/x-ndjson responses are treated as streaming. Other responses are buffered until response_end, then returned as a normal Rack response body.

The receiver sends every HTTP response as bounded chunks without first buffering its complete body. BridgeCore additionally splits a response chunk if its serialized UTF-8/base64 representation plus NATS headers would exceed max_payload. For ordinary responses, a requester advertises payload.response_flow_control=true; the receiver then uses the existing response credit window, and the requester returns credit as it collects chunks. Receivers continue to support legacy requesters that omit this flag. Streaming responses retain their existing credit behavior.

Error classification uses the explicit outcome, never the wording of error. Events from older peers without an outcome are classified as generic errors. Missing UPSTREAM_URL and unsupported operations produce a terminal HTTP 503 response in both Core and JetStream, so those permanent refusals are acknowledged instead of retried. An unavailable or invalid INFO.max_payload stops publication with a diagnostic error; the proxy does not substitute an assumed limit.

Owner Metadata

The first successful event of a flow selects the receiver that owns the rest of that flow.

• response_start carries receiver_service_id, request_id, and flow_kind for HTTP flows.
• session_established carries receiver_service_id, session_id, and flow_kind for TCP tunnel flows.

The requester stores this metadata in its request context. It is then used for owner-addressed cancellation and, for established TCP sessions, for upstream session bytes sent back to the receiver that opened the target connection.

Non-Streaming Request

sequenceDiagram
  participant Client
  participant Req as requester
  participant NATS
  participant Rec as receiver
  participant Up as upstream

  Client->>Req: HTTP request
  Req->>NATS: request envelope with reply_to
  NATS->>Rec: http_request
  Rec->>Up: forwarded HTTP request
  Up-->>Rec: HTTP response headers and body chunks
  Rec-->>NATS: response_start(streaming=false)
  Rec-->>NATS: bounded response_chunk events
  Rec-->>NATS: response_end
  NATS-->>Req: response events
  Req-->>Client: HTTP response

BridgeProtocol.wait_for_start_event waits up to NATS_RESPONSE_TIMEOUT for response_start. BridgeProtocol.collect_non_streaming_body then waits up to STREAM_RESPONSE_TIMEOUT for chunks and response_end, returning byte credit as it accumulates the ordinary response. A structured response error produces HTTP 502 or 504 instead of losing its diagnostic message.

Streaming Response

sequenceDiagram
  participant Client
  participant Req as requester
  participant NATS
  participant Rec as receiver
  participant Up as upstream

  Client->>Req: HTTP request
  Req->>NATS: request envelope
  NATS->>Rec: http_request
  Rec->>Up: forwarded HTTP request
  Up-->>Rec: SSE or NDJSON stream
  Rec-->>NATS: response_start(streaming=true)
  Req-->>NATS: response_credit
  loop each upstream chunk
    Rec-->>NATS: response_chunk
    NATS-->>Req: response_chunk
    Req-->>Client: stream chunk
    Req-->>NATS: response_credit after client write
  end
  Rec-->>NATS: response_end
  Req-->>Client: close response body

For streaming responses, the requester grants initial response credit after it receives response_start and learns the owner receiver. The receiver reserves credit before publishing response_chunk events. The requester returns credit only after writing bytes to the downstream Rack stream. Each wait for the next response event uses STREAM_RESPONSE_TIMEOUT.

If the upstream fails after a stream has started, the receiver emits response_error followed by response_end. The requester writes an in-band error formatted for the stream content type:

• SSE: event: error with JSON data.
• NDJSON: one JSON error object followed by a newline.
• Other content types: a JSON error body.

Flow Control

Flow control keeps streaming HTTP responses and TCP tunnels bounded by the side that consumes bytes. Without it, a fast receiver can publish more data into NATS than the requester can write to the client socket, and a fast client can do the same in the upstream tunnel direction.

Credit frames are normal bridge protocol messages with type=flow_credit:

{
  "type": "flow_credit",
  "request_id": "req-or-session-id",
  "direction": "response",
  "bytes": 262144,
  "service_id": "requester-1",
  "timestamp": "2026-04-24T00:00:00Z"
}

Valid directions are:

Direction	Protects	Credit sender	Credit receiver
response	HTTP streaming response chunks from receiver to requester	requester, after writing response bytes to the client	owner receiver control listener
upstream	TCP tunnel bytes from requester to receiver target socket	receiver, after writing bytes to the target socket	requester downstream session listener
downstream	TCP tunnel bytes from receiver target socket to requester	requester, after writing bytes to the client socket	owner receiver upstream session listener

Credit frames use Nats-Session-Id and Nats-Frame-Type headers to share the same routing code as session frames. HTTP response credit uses Nats-Frame-Type=response_credit and the owner-scoped control subject <request_root>.control.<receiver_service_id>.<request_id>. TCP tunnel credit uses session subjects with session_credit_upstream or session_credit_downstream.

Window sizes are derived from the effective NATS max payload and the service chunk size. With the default 32 KiB chunk size, the initial window is 1 MiB, credit is usually returned in 256 KiB batches, and the retained window is capped at 4 MiB. These values are implementation sizing, not environment variables.

If a publisher cannot reserve credit before STREAM_RESPONSE_TIMEOUT, the flow records a flow-credit timeout. HTTP streaming responses emit an in-band error when the stream already started. TCP tunnels close the session with reason flow_credit_timeout.

Cancellation

Cancellation is best effort. It is used when a downstream streaming client disconnects, a stream times out, or a tunnel writer fails. After owner selection, the requester publishes a cancel envelope to the owner-scoped subject <request_root>.cancel.<receiver_service_id>.<request_id_or_session_id>. Before the requester knows receiver_service_id, cancellation falls back to the original request subject as a best-effort early cancel.

{
  "type": "cancel",
  "request_id": "req-id",
  "cancel": {
    "request_id": "req-id",
    "service_id": "requester-1",
    "reason": "downstream_disconnect",
    "timestamp": "2026-04-24T00:00:00Z"
  }
}

Receiver-side active streams observe this envelope through the owner-scoped cancel listener or through the early fallback path in the request listener. Duplicate or late cancels are ignored.

RequestContext allows a bounded trailing event after cancel. In the current code this is one response_chunk, plus a terminal response_end or session_close when present. This keeps cancellation from corrupting an already in-flight terminal event while still stopping further work as soon as the receiver sees the cancel.

sequenceDiagram
  participant Client
  participant Req as requester
  participant NATS
  participant Rec as receiver
  participant Up as upstream

  Client->>Req: streaming HTTP request
  Req->>NATS: request envelope
  NATS->>Rec: http_request
  Rec->>Up: forwarded HTTP request
  Rec-->>NATS: response_start(streaming=true)
  NATS-->>Req: response_start
  Req-->>Client: stream starts
  Client--xReq: disconnect
  Req->>NATS: cancel envelope
  NATS->>Rec: cancel
  Rec-->>NATS: response_error or response_end

Cancellation is best effort: late response events may already be in flight when the cancel envelope reaches the receiver.

For observability, request contexts store outcomes such as completed, canceled, timeout, upstream_error, protocol_error, and session_error. The case API derives user-facing outcomes from events: success, error, timeout, canceled, or in_progress.

The requester records an outgoing session_close after publication succeeds. During HTTP tunnel cleanup, Async cancellation is deferred until publication and recording finish, including any JetStream acknowledgment. A failed close publication records a local response_error with outcome=session_error and no wire payload instead of a successful close. This lets the case API finish a locally closed tunnel even if the peer’s final event arrives after cleanup.

The existing close event carries an explicit outcome; older close events without this field retain their successful-close meaning. Timeout and error classification never depends on the reason text. The receiver preserves the close outcome and releases its credit wait after consuming the ordered close frame. A downstream HTTP disconnect sends the existing cancel envelope before tunnel cleanup. Requester pumps stop both child tasks before closing their socket, including when canceled, so cleanup does not trigger another close publication through a socket error.

Completion metrics count unique request ids. Reconstruction ratios use case outcomes, so failed, timed out and canceled cases do not contribute to the success ratio.

Binary Safety

BridgeProtocol.chunk_event sends response chunks as plain body only when the bytes are valid UTF-8. Invalid UTF-8 chunks are base64 encoded as body_base64. BridgeProtocol.chunk_body decodes both shapes on the requester side.

TCP tunnel payload bytes do not use JSON chunk events after the session is established. They are published as raw binary NATS payloads on session subjects.