Wingsmith
wingsmith.dev
Concepts

Tenants and Projects

Tenants, projects, API keys, cache isolation, and how multi-tenancy works.

A tenant is an organization using your Wingsmith installation. Each tenant owns endpoints, API keys, and the hostname its inference requests arrive on. A project is a quota unit within a tenant, grouping workloads and their device allocations.

Tenants

A tenant represents one organization calling models through Wingsmith. You create tenants to separate traffic, quotas, and access between different groups or customers.

Each tenant has:

  • A unique slug (lowercase identifier, used in URLs and references)
  • A name (display label)
  • An ingress mode: internal or public
  • An auth mode: none or api_key
  • A host field: the single hostname that resolves to this tenant

Authentication modes

The auth field controls whether the gateway checks credentials:

  • none: No API key required. Reaching the listener is sufficient.
  • api_key: Callers must present a valid API key (hgrk1...) in the Authorization header.

You can declare public ingress with none auth if you want unauthenticated public serving. The gateway doesn’t second-guess that choice.

Gateway listeners and tenant resolution

The gateway runs two listeners:

Internal listener: No ingress from outside your network. Reaching it proves identity. No API key check. Used for trusted callers within your infrastructure.

Public listener: Exposed to the internet. Identifies the tenant from the Host header of each request. If the tenant declares auth: api_key, the gateway verifies the key before routing.

On the internal listener, any caller can reach any tenant’s models. On the public listener, a request naming one tenant’s host cannot reach another tenant’s models — it gets 404, never 403, so a guess about what exists is never confirmed.

Projects

A project sits inside a tenant. It’s the unit that quotas attach to: you set device limits per project and device class, not per tenant.

Every workload belongs to one project. When placement admits a deployment, it checks that project’s quota for the device class the workload needs. If the project would exceed its quota, the deployment stays in the pending admission phase until capacity frees up.

Projects also namespace workload names: two tenants can each have a project named production, and within each project a workload name is unique.

API keys and console access

Tenants mint two kinds of credentials:

Tenant tokens (hgrt1...): Used to sign into the console. Scoped to read-only serving views. Minted and revoked at /v1/tenants/{ref}/tokens.

API keys (hgrk1...): Used to call the gateway for inference. Presented in the Authorization: Bearer header. Minted and revoked at /v1/tenants/{ref}/keys.

Each tenant can hold many keys, one per caller. Revoking one key doesn’t affect others. Only the sha256 hash of a key is stored; the plaintext appears once at creation and nowhere else.

A tenant with no keys can still call models if its auth is none or if traffic arrives on the internal listener.

Cache isolation

Each tenant has a cache salt: a per-tenant value the gateway injects into every inference request as nvext.cache_salt. NVIDIA Dynamo’s KV cache and prefix cache are isolated by this salt, so no tenant ever reads another tenant’s cached attention state.

This happens automatically. You don’t configure the salt; the gateway derives it from the tenant identity resolved at request time.

Multi-tenancy in practice

Wingsmith is multi-tenant at the routing and quota layer. One installation serves models for many organizations:

  • Each tenant sees only its own endpoints in /v1/serving.
  • Each tenant’s API keys work only for that tenant’s models.
  • Quota enforcement is per-project, so one tenant’s usage never spends another’s allocation.
  • The same deployment can serve multiple tenants through separate endpoints, or each tenant can have dedicated deployments, depending on how you configure endpoint targets.

Admin access (console and hgr CLI) sees all tenants and all projects. There’s one admin role, no per-user accounts or SSO yet. A team sharing a plane shares that identity.