Wingsmith
wingsmith.dev
Serving

Manage Tenants and Keys

Create tenants, mint API keys, configure auth mode, set ingress and visibility.

Tenants provide isolation for API keys, endpoints, and prefix cache. Each tenant has its own catalog of models, quota enforcement, and authentication settings.

Create a tenant

In the console, go to Access → Tenants and click Create Tenant. Fill in:

  • Slug: unique identifier, used in URLs and code (e.g. acme-corp)
  • Display name: human-readable label (e.g. Acme Corporation)
  • Subdomain: for the public listener, e.g. acme becomes acme.hangar.example.com

The slug is immutable after creation. The display name and subdomain can be edited later.

hgr does not yet support creating tenants; use the console or the HTTP API directly.

Set auth mode

A tenant’s auth mode controls how the gateway authenticates requests from it.

  • none: no authentication required. Suitable for internal listeners where network boundary is the trust layer.
  • api_key: requests must present a valid API key in the X-Api-Key header.

On the public listener, a tenant with auth: none is refused by default (returns 401 auth_required). Set HANGAR_GATEWAY_PUBLIC_ALLOW_NO_AUTH=true on the gateway to allow it.

Edit auth mode in the console on the tenant’s detail page, under Settings.

Mint an API key

API keys (prefix hgrk1.) authenticate a tenant on the public listener when auth: api_key.

From the tenant’s page, click Create Key. The plane generates a random key, stores a bcrypt hash, and displays the plaintext once. Copy it immediately; it cannot be retrieved later.

Keys are scoped to the tenant. A tenant with multiple keys shares its quota and prefix cache across them.

Set ingress and visibility

Tenant ingress

Tenant ingress controls which gateway listeners serve the tenant:

  • internal: only the internal listener (:8080, no auth, private network)
  • public: only the public listener (subdomain-based, optionally authenticated)
  • both: served on both listeners (default)

On the internal listener, every enabled tenant is reachable regardless of ingress setting, because the internal listener has no tenant identification. The ingress setting filters what the public listener serves.

Set ingress in the console under Tenant Settings or via the API:

PATCH /v1/tenants/<tenant-slug>
{
  "ingress": "public"
}

Endpoint visibility

Endpoint visibility is a second layer, per model:

  • private: served on the internal listener only
  • public: served on both listeners (if tenant ingress allows public)

A private endpoint is never served on the public listener, even for a tenant with ingress: both. Use this for internal-only models or models under development.

Set visibility when creating an endpoint in the console, or via the API:

POST /v1/endpoints
{
  "name": "llama-3-8b",
  "tenant_id": "ten_01...",
  "visibility": "public"
}

Rotate keys

To rotate a key without downtime:

  1. Mint a new key for the tenant.
  2. Update callers to use the new key (gradual rollout or simultaneous switch).
  3. Revoke the old key from the console or with DELETE /v1/tenants/{tenant}/keys/{key_id}.

A revoked key is refused immediately on the gateway’s next tenant table refresh (default 30 seconds). Revoking a key does not close open requests using it; they run to completion.

Tenant console access

A tenant can sign into the console at /login with:

  • Account: the tenant’s slug (e.g. acme-corp)
  • Password: a console token (not an API key)

Console tokens are separate from API keys and are minted from the tenant’s page. A tenant session sees only its own endpoints and serving metrics, not the fleet or other tenants.

Disable a tenant

Disabling a tenant refuses all requests on both listeners with 403 tenant_disabled.

In the console, from the tenant’s page, toggle Enabled. Disabled tenants remain in the plane’s database and can be re-enabled. Their endpoints and keys are preserved.

On the internal listener, the check runs after routing resolves the endpoint’s owner. On the public listener, it runs after the Host header identifies the tenant and (if required) the key is verified.

Isolation guarantees

Prefix cache

Every text generation request is salted with the tenant ID, so two tenants on the same deployment cannot share KV cache blocks. On the internal listener, X-Hangar-Cache-Scope narrows the salt further (e.g. one tenant serving many customers can isolate each customer’s cache).

Media requests (image, video, audio inputs) are salted with the full prompt hash to work around an engine defect, so no prefix-cache reuse occurs across turns in a multi-turn conversation carrying media. Text-only requests are unaffected.

See Prefix-cache isolation for details.

Quota

Quotas are per project, not per tenant. A tenant can own multiple projects, each with its own device quota. Workloads declare a project at creation and charge against that project’s quota.

See Quotas and Placement.

Gateway limits

Gateway rate limits are per tenant, per replica:

  • HANGAR_GATEWAY_TENANT_MAX_IN_FLIGHT: max requests in flight (default 1024)
  • HANGAR_GATEWAY_TENANT_RPS / HANGAR_GATEWAY_TENANT_BURST: token-bucket rate limit

A tenant over either limit receives 429 rate_limited with Retry-After. Every key belonging to a tenant shares its allowance.

On the internal listener, the tenant is the endpoint’s owner (known only after parsing the request body). On the public listener, it is the Host header’s tenant.

Security notes

  • API keys are never logged. The access log shows a 12-character hash prefix for correlation, never the full key.
  • Keys are stored hashed. The plane stores bcrypt hashes; plaintext keys are shown once at creation and never again.
  • Tenant credentials are stripped. X-Api-Key, Authorization, and Cookie are removed before requests leave the gateway. No cluster, agent, or engine log receives a copy.
  • Internal listener requires a NetworkPolicy. Without one, any pod that can reach the gateway can call every tenant’s models without a key. See Networking for an example policy.

Troubleshooting

404 unknown_host on public listener

The gateway has not loaded the tenant yet. Causes:

  • Tenant created within the last resync interval (default 30s). Wait and retry.
  • Gateway HANGAR_GATEWAY_PUBLIC_LISTEN is not set. Public listener is off.
  • Tenant’s subdomain does not match the Host header sent.

401 auth_required

  • Tenant’s auth mode is api_key and no X-Api-Key header was sent.
  • Key is invalid or revoked.
  • On the public listener with a tenant whose auth mode is none, the gateway refuses unless HANGAR_GATEWAY_PUBLIC_ALLOW_NO_AUTH=true.

403 tenant_disabled

The tenant is disabled. Re-enable it in the console.

Tenant metrics not visible in console

Tenant metrics require:

  • A metrics store configured (HANGAR_METRICS_SOURCES)
  • Clusters reporting metrics via remote_write (enabled with monitoring.enabled in the chart)
  • Tenant has made requests that generated usage data

Next steps