A Kubernetes control plane that discovers every provider across every site, scores them, and hands Praxis a flat file to route from - so the request hot path never has to ask a cluster anything.
One sentence carries the whole design: Grid decides what should be routable; Praxis AI performs the actual routing. Grid never proxies model traffic, terminates data-plane TLS, or touches a request.
Without Grid, every gateway would need static knowledge of every backend, every remote cluster, every credential rule, every health signal. That does not scale across sites and providers. Grid turns that moving control-plane state into a local file the gateway reads cheaply.
grid-config.json and goes.Grid is a Kubernetes operator, so its inputs are custom resources in the grid.praxis-proxy.io/v1alpha1 API group. Three of them define everything the routing layer needs.
A logical grid: SWIM seeds, TLS settings, gateway references, region/zone geography. The membrane everything else lives inside. Supports multi-tenancy - one cluster can host multiple networks.
One participating cluster: its discovery state, egress address, trust material for mTLS. Phases track lifecycle from Pending through Discovered to Active. A site becomes Active only after cert fingerprint verification passes.
A declaration of model capacity: model names, backend kind (local, remote, cloud, API), endpoint, health config, auth strategy, and access policy via site label selectors.
Not a CRD - the rendered grid-config.json ConfigMap that Praxis AI's grid_route filter consumes. Pre-sorted candidates with scores, admission state, and credential references.
The operator reconciles continuously. For each network it lists local providers, scrapes their metrics, folds in provider records from remote sites via CRDTs, scores everything, and writes the overlay.
The scoring engine blends six normalized signals with configurable weights: locality (3.0), queue depth (3.0), KV-cache utilization (2.0), prefix-cache hit ratio (2.0), P99 latency (2.0), and cost per token (1.0). Missing values default to a neutral 0.5; providers reporting unhealthy are dropped entirely.
Post-scoring, candidates are enriched with admission state (NewAndExisting, ExistingOnly, or Excluded based on metric thresholds) and locality tier (SameSite, SameZone, SameRegion, CrossRegion). The final sort is: admission state first, then locality tier, then score, then freshness. Praxis just picks from the top.
The locality signal encodes a clear preference order by backendKind - use capacity you own and that is near before falling out to third-party APIs.
Credentials never leak into routing data: Grid projects a reference to a Kubernetes Secret into the overlay, and the token is injected later by Praxis from a mounted secret. Token bytes never appear in ConfigMaps, logs, status fields, or tracing spans.
Grid learns "which sites are alive" with SWIM, a gossip-based membership protocol. Grid wraps the foca crate and carries it over AES-256-GCM encrypted UDP.
There is no central registry pinging everyone. Each site periodically probes a random peer; if it goes quiet, others double-check before anyone is declared gone. News of a join or failure spreads like an infection - reaching the whole mesh in a handful of rounds.
SWIM detects a downed site in seconds without flooding the network, and scales at O(log n) gossip cost. Its membership events feed the operator controllers, which react by re-rendering the overlay. CRDT state broadcasts piggyback on SWIM probe messages - no separate transport needed.
Knowing which sites are alive is only half the job. Each site also needs the others' provider state - and every site updates its own view at once. Grid propagates this as a CRDT: a data structure that merges automatically and always converges, with no coordinator.
A CRDT - Conflict-free Replicated Data Type - is a data structure whose merge operation obeys three algebraic properties:
These three rules mean messages can arrive out of order, duplicated, or batched, and every site still lands on identical final state. This convergence is called strong eventual consistency.
The unit of exchange between sites is a GridStateSnapshot, carrying lightweight control-plane facts: provider capabilities, lifecycle phase, and normalized scoring metrics.
Provider records use last-writer-wins: each write carries a (revision, writer_id) tuple. Higher revision wins. Equal revisions break ties by lexicographic writer_id. The rule is deterministic, so every site chooses the same winner.
Capabilities (models, tools, agents) use an add-wins OR-Set. Each add creates a unique tag; a remove only tombstones the tags it has observed. If one site adds a model while another removes it, the add's fresh tag survives the remove's stale tombstone list. Losing a capability you just registered because a stale remove arrived late would be worse than briefly keeping one.
Each site keeps its own slot; the real value is the sum. Merge takes the max of each slot. Used for per-tenant budget tracking - under partition, each site sees a lower bound rather than hard-rejecting.
Piggybacked on SWIM probes. Four independent invalidation lanes (state, gateway, cert, metadata) so a gateway address update cannot block a provider state broadcast. Per-origin revision ordering rejects stale messages.
SWIM traffic is encrypted with AES-256-GCM. The wire format is compact: 4-byte magic, 1-byte version, 12-byte nonce (OS RNG), ciphertext, 16-byte GCM tag - 33 bytes of overhead.
Wrong key or tampered data: GCM tag verification fails, packet silently dropped. Plaintext packets when a key is configured: rejected on the magic bytes. Encryption proves membership in the shared key group.
Site-to-site trust uses mTLS with SHA-256 cert fingerprint pinning. A GridSite becomes Active only after its cert fingerprint matches the configured trust value. Public cert PEM is propagated via SWIM broadcasts - structural checks discard any message containing private key markers.
grid-config.json, none of this touches a live request.