Comparison with related projects
How Wingsmith relates to Dynamo, llm-d, KServe, AIBrix, SkyServe, Run:ai, KAI Scheduler, LiteLLM and Envoy AI Gateway.
Most of these projects work inside one cluster, or in front of endpoints that already exist. Wingsmith’s layer is the one above a set of clusters: which clusters a model runs on, what each one is told to run, and how a request finds it. For its scope and design, see the architecture overview.
| Project | What it does | Relation to Wingsmith |
|---|---|---|
| NVIDIA Dynamo (operator and Planner) | Serves a model inside one cluster: frontend, KV-aware router, disaggregated prefill/decode workers. The Planner scales workers against latency targets | Wingsmith renders into it. Each Wingsmith deployment becomes one DynamoGraphDeployment in one cluster |
| llm-d | Kubernetes-native distributed inference inside a cluster: vLLM, an inference scheduler on the Gateway API Inference Extension, disaggregation | An in-cluster alternative to Dynamo. Wingsmith has no llm-d renderer |
| KServe | Model-serving CRDs (InferenceService) per cluster, with autoscaling and many runtimes | Per-cluster serving. Wingsmith does not render KServe objects |
| AIBrix | Control-plane components for vLLM in one cluster: LLM-aware gateway, autoscaler, LoRA management, distributed KV cache | Same layer as Dynamo’s own control plane, inside one cluster |
| SkyServe (SkyPilot) | Serves a model across clouds and regions by provisioning its own VMs or pods and load-balancing across them | Provisions compute itself. Wingsmith manages clusters you already run and leaves provisioning to you |
| LiteLLM, Envoy AI Gateway | API gateways in front of model providers: one API, keys, rate limits, cost tracking | Route to endpoints that already exist. Wingsmith’s gateway is narrower (no billing, no provider catalog) and is fed by the plane’s knowledge of what runs where |
| Run:ai, KAI Scheduler | GPU scheduling inside a cluster: quotas, fair share, preemption, for training and inference | Device scheduling, which Wingsmith leaves to the cluster. Wingsmith does not preempt or run training |
NVIDIA Dynamo
Dynamo is the only serving runtime Wingsmith drives today, with vLLM, SGLang or TensorRT-LLM inside it. Clusters run the Dynamo platform, which the cluster-setup chart installs. Because the plane never sends Kubernetes manifests, a second renderer is possible, but none exists.
What Wingsmith adds on top of a single cluster’s Dynamo today:
- Clusters that only dial out. No kubeconfig in a central database and no inbound port on any cluster (topology).
- Adoption before management. Models already running in a cluster are observed and can be put under management by an explicit decision, without a restart.
- Tenants and endpoints across clusters. One public model name, per-tenant hosts and API keys, per-tenant KV-cache isolation, and weighted or ordered routing to deployments on several clusters.
- Fleet inventory per device. Device identity and health from DCGM, a device-class table shared by plane and operator, and placement against it.
- Measurement of the serving path. Conformance probes of an endpoint against a reference provider (
hgrconform), load tests (hgrbench), and request-path measurement (hgrtransit).
NVIDIA has signalled multi-cluster coordination for Dynamo. If that ships, the overlap with Wingsmith is cross-cluster placement and routing.
llm-d, KServe and AIBrix
All three work inside one cluster. Wingsmith renders only Dynamo graphs, so using one of them as the runtime would mean a new operator renderer, not a change to the plane.
SkyServe
SkyServe provisions its own compute. Wingsmith assumes the clusters exist and that you operate them. Use SkyServe if you want a system that provisions GPUs for you, and Wingsmith if you run your own clusters and want a control plane that places models across them.
LiteLLM and Envoy AI Gateway
These sit in front of providers and endpoints they know nothing else about. Wingsmith’s gateway only routes to clusters the plane manages: it reads the routing table the plane derives, and can split by weight or fail over in a declared order across the clusters it holds a tunnel to. It does not meter or bill. Use LiteLLM or Envoy for external APIs or for self-hosted endpoints Wingsmith does not manage.
Run:ai and KAI Scheduler
Wingsmith picks the cluster. The scheduler inside each cluster picks the devices. The two work together: Run:ai or KAI inside each cluster, Wingsmith above them.
When Wingsmith fits
Use Wingsmith if:
- You run your own GPU clusters and want a control plane that places models across them.
- You want to adopt models already running without restarting them.
- You want clusters that only dial out, with no inbound connections.
- You want one gateway that routes to models across many clusters.
- You can render into NVIDIA Dynamo only.
Do not use Wingsmith if:
- You want a system to provision GPU VMs for you.
- You need a serving runtime other than Dynamo.
- You are calling external APIs like OpenAI.
- You need training or batch workloads. Wingsmith serves inference only.