Measurement tools

hgrtransit

Modes, flags, outputs and limits of hgrtransit, the gateway to tunnel to agent path measurement tool.

hgrtransit measures the data path: client, gateway, tunnel, agent, local OpenAI-compatible frontend. It needs no GPU, no Kubernetes and no model. It is not hgrbench: that measures an engine on real hardware, this measures the plumbing with the engine replaced by a fake that sleeps on a schedule. For the path itself see Data plane.

Install

Nothing is published. Run it from a checkout of the repository with Go installed. synthetic and path also build the gateway and agent binaries from that checkout.

cd libs/transit
go run ./cmd/hgrtransit synthetic -repo ../..

Without -repo, it looks for the checkout above the working directory.

Modes

Mode	What is real	Touches shared systems
`synthetic`	Gateway and agent binaries, the tunnel, both reverse proxies. Fake frontend and fake control plane.	No. Child processes and temporary ports on this host.
`live`	A deployed gateway, its tunnel and a real engine.	Yes. Offered load on a shared serving path.
`path`	Gateway and agent binaries in front of a frontend you run. No load generated.	No. This host only.
`upstream`	The fake OpenAI-compatible frontend, served on its own.	No.
`plane`	The fake control plane, served on its own.	No.

Commands

`synthetic`

Builds the path on this host, runs a baseline straight at the frontend, then ramps through the path.

Flag	Default	Meaning
`-repo`	found above the working directory	Checkout to build the gateway and agent from.
`-ramp`	`8,32,128`	Concurrency stages.
`-hold`	`15s`	How long each stage starts requests.
`-drain`	`30s`	How long in-flight requests get to finish after the hold. Survivors are reported as censored.
`-agents`	`1`	Agent processes serving the cluster.
`-conns`	`2`	Tunnels each agent holds to each gateway replica.
`-streams`	`0`	Per-tunnel stream limit the agent advertises. `0` is the tunnel default of 256.
`-ttft`	`200ms`	Fake frontend time to first token.
`-itl`	`10ms`	Fake frontend inter-token delay.
`-tokens`	`64`	Tokens the frontend emits per answer.
`-prompt-bytes`	`512`	Request body padding.
`-stream`	`true`	Ask for SSE. `-stream=false` measures the non-streaming path.
`-baseline`	`true`	Measure the frontend directly first.
`-kill-agent-every`	off	SIGKILL one agent this often.
`-slo-success`	`0.999`	Success rate the capacity number is reported under.
`-slo-ttft-p99`	`2s`	TTFT p99 the capacity number is reported under.
`-json`	off	Write the report as JSON.
`-v`	off	Show gateway and agent logs.

`live`

Ramps against a deployed gateway. It sends HANGAR_API_KEY as the bearer token, which must be a tenant API key for the endpoint under test. It warns if the variable is unset.

export HANGAR_API_KEY=<tenant API key>
hgrtransit live -url https://<gateway-host> -model <endpoint-name>

Flag	Default	Meaning
`-url`	required	Gateway base URL.
`-model`	required	Endpoint name the target serves.
`-ramp`	`1,2,4`	Concurrency stages.
`-max-concurrency`	`8`	Refuse any stage above this. The command exits before sending anything.
`-hold`	`20s`	How long each stage starts requests.
`-drain`	`30s`	As in `synthetic`.
`-prompt-bytes`	`512`	Request body padding.
`-stream`	`true`	Ask for SSE.
`-slo-success`	`0.999`	Success rate for the capacity number.
`-slo-ttft-p99`	`10s`	TTFT p99 for the capacity number.
`-json`	off	Write the report as JSON.

live sends real completions that use GPU time and share a queue with other callers. Raise -max-concurrency only for a load you have agreed, in a quiet window, and watch the hangar_gateway_* metrics while it runs. See Metrics.

`path`

Holds the gateway and agent open in front of a frontend you run, prints a ready line to stderr, and waits. just conformance-gateway uses it. Callers use the public listener with HANGAR_API_KEY as their tenant key, which is required.

HANGAR_API_KEY=<key> hgrtransit path -frontend http://localhost:8000

Flag	Default	Meaning
`-frontend`	required	OpenAI-compatible server the agent forwards to.
`-listen`	`127.0.0.1:8080`	Gateway public listener, and the tenant host it answers.
`-model`	`gpt-oss-120b`	Public endpoint name, also passed to the frontend as the served model.
`-repo`	found above the working directory	Checkout to build from.
`-v`	off	Show logs.

`upstream` and `plane`

Command	Flags
`upstream`	`-listen` (`127.0.0.1:9900`), `-ttft`, `-itl`, `-tokens`
`plane`	`-listen` (`127.0.0.1:9901`), `-cluster` (`cluster-transit`), `-token` (`transit-enrolment-token`), `-service-token` (`transit-gateway-service-token`), `-model` (`transit-model`)

The fake plane answers the three routes the gateway’s service principal uses: GET /v1/endpoints/resolved, GET /v1/gateway/tenants and POST /v1/clusters/tunnel-auth.

Scenarios

Each is a flag combination on synthetic.

Flags	Asks
`-streams 16 -conns 1` with a ramp past 16	What happens at the per-tunnel stream limit. Past it the pool is saturated and the gateway sheds with a 503. Compare gateway CPU with the `http_503` count per stage.
`-agents 3 -conns 2`	Many tunnels to one gateway against one tunnel carrying everything.
`-kill-agent-every 3s`	What a caller sees when an agent dies mid-stream. Requests whose headers were already sent are not retried.
`-ttft 2s -itl 50ms -tokens 512`	A long generation that holds the stream open.
`-stream=false`	The non-streaming path.
`-slo-ttft-p99`, `-slo-success`	The condition the capacity number is reported under.

Output

Per stage, per request: TTFT, every inter-token gap, total duration and the failure bucket. The report aggregates these.

Figure	Detail
TTFT, inter-token latency	p50, p90, p99, and inter-token jitter. TTFT stops at the first frame with generated content. Role-only chunks, empty deltas and the usage chunk do not count.
Rates	Requests and tokens per second of stage time, drain included.
Success rate	Taken over requests that finished. Requests still running when the drain ends are censored: neither success nor failure, and counted in TTFT percentiles at the time they were cut off, which is a lower bound.
Per-leg split	`synthetic` only. TTFT split into inbound leg (gateway, tunnel, agent, local proxy) and return leg, from timestamps the fake frontend writes.
Per-process cost	`synthetic` only. Gateway and agent CPU (share of a core, one-second samples) and RSS, mean and peak per stage.
Tunnels	The gateway’s `hangar_gateway_tunnels` gauge at the end of each stage, beside the number of tunnels configured.
Held requests	The frontend’s peak in-flight count against the concurrency offered. Fewer means something between them held requests.
Baseline subtraction	The first stage also run straight at the frontend, so the path’s overhead is a subtraction.

synthetic and live write the whole report, then exit 1 if no stage held the success and TTFT objective. just transit-smoke relies on this to fail a CI job.

Leg and process columns are empty in live mode, not estimated.

What does not compare across modes

Quantity	Why
TTFT and inter-token latency	`synthetic` reports the fake frontend’s schedule plus the path. `live` reports a real engine, which dominates. Compare overhead over baseline, which only `synthetic` can produce.
Throughput	`synthetic` has no GPU. Its number is a ceiling for the transport, not a capacity for the product.
Absolute latency	Loopback is far below any WAN latency, so every synthetic figure is missing a network round trip.

Limits

It is not a network. There is no WAN latency, loss, MTU or TLS on the tunnel by default, and every process shares one machine.
It is not a model. The fake frontend never queues, runs out of KV cache or slows under load.
It exercises one gateway replica. The Service and load balancer in front of replicas are not tested.
The control plane is a fixture. It says nothing about the real plane or the table it derives.
CPU figures are one-second averages, so a shorter burst is spread across its second.
It cannot separate gateway time from tunnel time from agent time on the inbound leg.