AI Grid Network (AGN) Documentation

routing scope, deployment-owner responsibilities, and topology boundaries.
On this page

Routing

Architecture

  • Overview: operator responsibilities, data-plane boundaries, workspace crates, and terminology.
  • Auth and Policy: provider authentication strategies, access policy, and trust model.
  • Enrollment: how a site becomes a member: parties, the flow, trust boundaries, and the planned object-driven shape.
  • Consumer Config: operator-generated consumer Praxis ConfigMap and the GatewayRef.praxisConfig API.
  • OpenTelemetry: exporter configuration, secret handling, image requirements, and the Praxis 0.7.1 trace-linkage limit.
  • External Client Ingress: GTM/GLB edge selection, AGN provider routing, trust boundaries, affinity, snapshot delivery, and provider-boundary ownership.

Operations

  • Fleet Dashboard: optional hub web UI drawing every site on a map with health from its own Prometheus; deployed only when you install its chart.
  • Adding an Inference Provider: step-by-step workflow for in-cluster, existing-service, and external HTTPS providers.
  • Metrics Reference: every metric the operator and the gateway export, with types, labels, and what each one means.
  • Operations: local environment setup, validation commands, and operator workflows.
  • CI Kind E2E: validation tiers, gate sequence, sequencing requirements, and environment dependencies.

Examples

  • Provider Traffic Selection: runnable three-cluster topology for AGN selection groups and request-time round-robin provider choice.
  • Praxis demos: deployable examples with automated runtime proofs of routing, failover, security boundaries, and provider lifecycle.

Integration Qualifications

  • Provider Traffic Qualification: proves multi-cluster discovery, accepted-overlay delivery, request-time round-robin selection, provider attribution, and stable routing.
  • Static Weighted Provider Qualification: proves configured capacity propagation, weighted overlay convergence, proportional request selection, hot reload, and equal-weight recovery. This is the release qualification for weightedRandom, static placement, provider capacity, weighted-overlay, or cross-site capacity propagation changes.
  • Combined-site Lifecycle Qualification: is the release qualification for combined-site routing, provider add/remove/re-add, session fallback, rollout, trust, or lifecycle changes.
  • Distributed Token Quota Qualification: proves shared identity-scoped quota enforcement across gateway replicas, regional provider selection, expiry, restart persistence, fail-closed state storage, and storage-network isolation.
  • Single-cluster Multi-gateway Qualification: proves shared overlay delivery and independent consumer/provider gateway behavior within one Kind cluster and one GridSite.
  • Provider draining: documents graceful provider maintenance, gateway-wide selection, and reversible drain state.

These integration tests create their environments through Forge and execute through first-class Rust xtask commands. Their topology READMEs document image preparation, execution, evidence, and cleanup.

Installation

Development

  • Release Process: versioning, validation, artifact publication, and release workflow.
  • Development: build, test, format, lint, and coverage.
  • Conventions: coding style, testing requirements, documentation rules, and commit attribution.
  • Developing: Conventions: shared Praxis coding, tracing, testing, and review conventions.
  • Developing: Type Design: shared Praxis guidance for serde, enums, newtypes, and representable states.
  • Unicode Audit: tracked text inventory and intentional Unicode exceptions.
0.3.x CRD Field Migration

The 0.3.x release accepts both the previous and current names for CRD spec fields.

Adding an Inference Provider

Step-by-step workflow for adding a new inference provider to an existing AGN installation. Covers in-cluster backends, existing services, and external HTTPS APIs.

Administrative provider draining

AGN supports graceful maintenance of inference providers through the optional InferenceProvider.spec.trafficPolicy.drain field.

AGN Operations

This guide covers AGN production installation, network formation, site lifecycle, routing configuration, security, and observability.

AGN Routing Architecture and Overlay Contract

Describes how Grid renders, versions, distributes, and secures routing state for Praxis gateways.

AI Grid Network Routing Guide

AI Grid Network (AGN) routes inference traffic across provider gateways using a multi-dimensional policy rather than a single load-balancing algorithm.

Architecture Overview

AI Grid Network (AGN) is the control plane that prepares routing state for Praxis AI gateways.

Authentication & Access Policy

AI Grid Network (AGN) deployments involve up to three distinct authentication layers. Each serves a different trust boundary and must not be conflated.

CI Kind E2E Implementation

This document describes the CI coverage for AGN’s Kind-based end-to-end validation suite.

Cross-site site selection

How a grid gateway serving gridServing chooses a site for each request in grid_site_route, what feeds the choice, and what you can change.

Custom Resource Definitions

API group: grid.praxis.fast/v1alpha1

Development

Development and validation guidance for AI Grid Network (AGN).

Development conventions

Points to the canonical Grid development conventions for code, tests, documentation, and pull requests.

Development Conventions

This is the canonical development policy for AI Grid Network (AGN). The

Enrollment

Enrollment is how a cluster, from then on a site, gets its grid identity: a name the grid assigned and a certificate the grid CA signed.

Examples

Browse 41 linked source examples for Praxis Grid, with pages pinned to the selected source snapshot.

Existing-Cluster Helm Installation

This Helm-based installation workflow installs the AGN Operator and Praxis gateways on existing Kubernetes clusters.

External Client Ingress

External client ingress extends AGN’s workload-routing model to a stable public endpoint. A global traffic manager selects an edge. AGN and Praxis then select and reach an eligible provider.

FIPS support

For a FIPS build, the security-relevant crypto (the TLS stacks and certificate identity hashing) must route through the system openssl (dynamically linked libssl and libcrypto).

Forge tooling

Install and use the pinned praxis-forge CLI for Grid topology validation and qualification workflows.

Installation

Introduces Grid installation paths for existing Kubernetes clusters, site enrollment, and AWS deployments.

Installing on AWS

Plan multi-site Grid deployments on AWS, including private connectivity, load balancers, and chart settings.

Metrics Reference

Every metric the grid exports, what it means, and where to scrape it. Three components expose Prometheus endpoints and they answer different questions.

OpenTelemetry for Grid gateways

Grid gateway builds can export traces to an OTLP/gRPC collector. Export is configured on the gateway process; routing overlays remain limited to routing state.

Operator-Generated Consumer Config

The AGN Operator can generate the consumer Praxis ConfigMap from routing overlay data. This is an opt-in feature on each GatewayRef.

Polling Cross-Site Load Signals

A gateway prefers the least-loaded site serving a model, so it needs each site’s load.

Prefix Affinity and Stored-State Pinning

The grid gateway’s grid_site_route filter keeps a conversation on the site that already holds its prompt in the prefix cache, while that site’s queue allows.

Provider Scoring

For how scoring interacts with routing policy, selection groups, and request-time selection, see the AGN Routing Guide.

Release Process

AI Grid Network (AGN) uses Semantic Versioning.

Repository Unicode audit

The audit covers the tracked repository text after integrating main at af82146cf9d23fa1fa5d7ffb587fb5053ca23a92.

Signal propagation

Signals are the provider load a site observes, such as queue depth and KV-cache pressure.

Site Enrollment

A site enrolls with a grid by redeeming a one-time token at the enrollment service.

Technical Preview Scope and Limitations

AI Grid Network (AGN) is a network-routing component for AI traffic.

Type Design

Make invalid states unrepresentable. The type system and serde should enforce constraints at parse time, not at runtime.