Configuration

Agent Configuration

Agent variables, gateway address, frontend discovery, drain timeout, health checks.

The serving agent dials out to the gateway and carries inference requests back into the cluster. It is configured through environment variables.

Gateway connection

Variable	Required	Description
`agent.gateway.address`	yes	Gateway address as `host:port`. Finds all replicas by redialling.
`agent.enrolmentToken`	yes	Cluster’s enrolment token (same as operator)

The agent is given one address and finds every replica behind it. Each accepted handshake names the replica that took the dial and says how many there are, so the agent redials this one address until it has found them all. Scaling the gateway is a change to the plane’s deployment and never an edit to the cluster.

A gateway replica can only serve a cluster it holds a tunnel to, so coverage has to be complete. The agent treats landing on a replica it already covers as ordinary, not as an error, and stops when it has them all.

Frontend discovery

Variable	Default	Description
`agent.frontendEndpoint`	empty (discovers)	The frontend Service to forward to. Empty discovers the Dynamo frontend Service by label.
`HANGAR_FRONTEND_DISCOVERY_INTERVAL`	`30s`	How often to re-read the frontend Services
`HANGAR_FRONTEND_DISCOVERY_STALE_AFTER`	`5m`	Readiness fails when discovery is older than this; `0` turns it off

The agent discovers the cluster-local hop by asking the cluster: the Service that selects on Dynamo’s frontend component label, resolved to a ClusterIP. Discovery re-reads the frontend Services every HANGAR_FRONTEND_DISCOVERY_INTERVAL.

When the API server stops answering, the agent keeps forwarding to the addresses it last saw, and hangar_agent_frontend_discovery_age_seconds grows. Past HANGAR_FRONTEND_DISCOVERY_STALE_AFTER, /readyz fails, because those addresses no longer follow a redeployed frontend.

Shutdown

Variable	Default	Description
`HANGAR_SHUTDOWN_GRACE`	`120s`	The pod’s `terminationGracePeriodSeconds`
`HANGAR_DRAIN_TIMEOUT`	`SHUTDOWN_GRACE - 5s`	How long to wait for open streams before severing tunnels

HANGAR_SHUTDOWN_GRACE must equal the pod’s terminationGracePeriodSeconds. The drain is cut from it: by default 120s - 5s = 115s for the drain, ending at 115 seconds, with 5 seconds unspent before the kubelet’s SIGKILL.

On SIGTERM, the agent stops dialling, sends GOAWAY on every tunnel, and waits for the streams already open. The wait is HANGAR_DRAIN_TIMEOUT. Whatever is left is severed when the timeout ends.

That drain bounds which generations survive an agent rollout. A non-streamed completion sends nothing until its last token, and the header timeout lets it run for up to 45 minutes, but a rollout still cuts it when the drain ends. To carry longer generations through one, raise agent.terminationGracePeriodSeconds in the chart, or both values together elsewhere.

The gateway’s own shutdown is separate and configured on the gateway side.

Health checks

Variable	Default	Description
`HANGAR_HEALTH_ADDR`	`:8081`	Where `/healthz`, `/readyz`, and `/metrics` are served

/healthz is unconditional. /readyz reports:

How many replicas have been reached against how many the gateway says there are
Frontend discovery staleness (when HANGAR_FRONTEND_DISCOVERY_STALE_AFTER is set)

One tunnel is enough to be ready. An agent that waited for full coverage would fail its own rollout because one gateway replica happened to be restarting. The shortfall is a metric, not a readiness failure.

Forwarding

The agent forwards only a POST to one of the inference paths and answers anything else 404 unsupported_path without reaching the frontend.

Forwarded paths:

/v1/chat/completions
/v1/completions
/v1/responses
/v1/messages
/v1/embeddings
/v1/images/generations
/v1/images/edits
/v1/videos
/v1/videos/stream
/v1/audio/transcriptions
/v1/audio/translations
/v1/audio/speech

Metrics

/metrics on the health address serves Prometheus text format.

Series	Labels	Description
`hangar_agent_requests_total`	`outcome`, `code`	Every request that came down a tunnel
`hangar_agent_requests_in_flight`		Current in-flight requests
`hangar_agent_request_duration_seconds`		Request duration (histogram)
`hangar_agent_gateways_expected`		How many gateway replicas the agent expects to cover
`hangar_agent_gateway_up`	`gateway_id`	1 per covered gateway replica
`hangar_agent_dial_duplicates_total`		Dials that landed on an already-covered replica
`hangar_agent_frontend_discovery_age_seconds`		Seconds since the last successful frontend discovery

Outcomes

outcome on hangar_agent_requests_total:

forwarded: the frontend answered (code says with what)
unsupported_path: path not on the forwarded list
no_frontend: no frontend address discovered
deployment_frontend_not_ready: deployment’s frontend not ready
frontend_error: frontend error
frontend_unavailable: frontend did not answer
canceled: caller left first
aborted: response was cut after it started

Read beside the gateway’s own request series, it says whether a failure happened inside the cluster or before it.

Configuration via chart

The agent is typically installed via the cluster-setup chart. In values.yaml:

cluster:
  readOnly: false          # required: read-only cluster cannot serve

agent:
  enabled: true
  replicas: 2
  gateway:
    address: hangar.example.com:443
  frontendEndpoint: ""     # empty discovers by label
  terminationGracePeriodSeconds: 120

The chart refuses:

agent.enabled with cluster.readOnly: true
agent.enabled with an empty gateway.address
A gateway that is still a flat string or an address carrying a scheme (it is host:port, TLS assumed)
An empty enrolmentToken.value

The agent presents the same enrolment token as its identity on the tunnel (TLS for confidentiality, the token for authentication). There is no second credential to mint or rotate.