intelligent_route

Selects an upstream cluster from a site/capability descriptor by matching either an inference model name or MCP tool name.

Configuration Notes

This filter is registered by the AI proxy (not Praxis core) because it encodes AI-specific routing semantics: ordered candidate consumption, admission-state filtering, session affinity, and MCP tool-call routing. Praxis core provides the generic filter runtime; this filter adds the intelligent routing candidate model on top.

Modes: - Static: candidates are declared inline in the YAML config. - Overlay: candidates are loaded from an overlay file (routing-overlay.json envelope or legacy routing-config.json) and hot-reloaded via [ArcSwap] when the file changes.

Behavior: - If the request path matches a configured skip_paths prefix (management and discovery endpoints), the filter returns Continue before any model lookup, setting ctx.cluster to management_cluster when one is configured. - If ctx.cluster is already set by an earlier filter, the selection is preserved and no metadata is written. - If no routing source is present, the filter returns Continue without routing. - If the model header or MCP tool name is blank, oversized, or invalid, the filter rejects with 400. - If a matching candidate is found, ctx.cluster is set and bounded route-decision metadata is written. - If no matching candidate is found, the filter rejects with 404.

Selection: session affinity is resolved first. New requests use the overlay’s selection mode within the first viable producer-defined group. Missing group or policy metadata uses deterministic first-admitted ordering. Praxis AI does not recompute source geography, load, or score. weightedRandom is overlay-only: every non-empty weighted candidate list must supply a selection group and an integer traffic_weight from 1 through 1000. Weights are applied only among candidates in the first viable group. Inline static candidates cannot enable weighted selection. A validated versioned overlay may have no candidates in any selection mode; matching requests are rejected with HTTP 404. Static candidate lists and legacy flat overlays remain non-empty. admission_state=none is never eligible. existing_only is eligible only through an already-bound session affinity entry.

Metadata: every request leaves intelligent_route.outcome in the in-process filter metadata: routed, reused, no_candidate, invalid, skipped, management, or preserved. When a capability lookup happened, intelligent_route.candidate_count records how many candidates the serving snapshot carries for it before admission filtering. On successful selection, bounded metadata is written under the same namespace (kind, name, site, cluster, local_site, stable_id, admission_state, and optionally rank, selection_group, selection_mode, and selection_tier). When session affinity is enabled, session.bound, session.reused, and session.failover keys are also written. All keys are available to the access log as metadata.<key>, including on 400 and 404 rejections. When the selected cluster is present in provider_hop_clusters, client-supplied x-ai-routing-candidate, x-ai-routing-request-id, and x-ai-routing-revision values are removed and replaced with the selected stable ID, a generated provider-hop request ID, and the serving overlay revision (envelope mode only). These AI-owned, non-reserved headers are sent only to an mTLS-authenticated provider gateway; the provider must run peer_identity_trust before consuming them. No credential reference or value is forwarded. No request-time database, control-plane, or metrics lookups are performed.

Metrics: the filter exports Prometheus series through the shared recorder. Label values come from closed vocabularies or from the bounded snapshot contract, never from client input, and candidate stable_id values are never used as labels.

• praxis_ai_intelligent_route_decisions_total{outcome, kind} — one per request; outcome uses the metadata vocabulary above and kind is inference_model, mcp_tool, or none before a lookup exists. - praxis_ai_intelligent_route_selected_total{cluster, site, selection_mode} — one per fresh selection (session reuse is not a selection). - praxis_ai_intelligent_route_session_total{event} — bound, reused, or failover, when session affinity is enabled. - praxis_ai_intelligent_route_session_table_entries — entries in the session tables of every instance in this process, including expired entries not yet evicted; instances add up and a dropped instance subtracts its table. - praxis_ai_intelligent_route_snapshot_candidates — candidates in the serving snapshot, published by the instance serving requests and cleared when it is dropped. - praxis_ai_intelligent_route_snapshot_reloads_total{result} — accepted, unchanged, rejected, or read_error, per overlay reload attempt.

Decision latency is covered by the core praxis_filter_duration_seconds{filter="intelligent_route"} histogram, which the proxy records only when metrics.filter_duration: true is set.

MCP lookup: if mcp.method filter metadata is set to tools/call and mcp.name is present, mcp_tool candidates are matched. Other MCP methods (initialize, notifications/*, etc.) skip routing.

Hot reload: when reload.enabled is true (the default in overlay mode), the filter watches the overlay file’s parent directory for filesystem events. On change, the file is re-read, SHA-256 hashed, parsed, validated, and atomically swapped in when its semantic revision changes via ArcSwap. In-flight requests continue using their previously loaded snapshot. Unreadable or invalid files retain the previous snapshot. Kubernetes ConfigMap projected volumes use atomic symlink replacement (..data), which the watcher detects as a Create/Modify event on the parent directory. The overlay ConfigMap must not use subPath volume mounts — subPath bypasses the ..data symlink mechanism and the watcher will not detect updates.

Envelope contract: The configuration producer publishes routing-overlay.json as a versioned, content-addressed envelope alongside the legacy routing-config.json payload. The AI-owned v1 shape is:

source_generation is a positive monotonic generation of the source resource. Unknown additive envelope, provenance, and candidate fields are accepted for forward compatibility; credential objects reject unknown fields to prevent secret material from entering the overlay.

Any reserved envelope field selects strict envelope parsing; malformed envelopes never fall back to legacy. When expected_overlay_scope is configured, legacy payloads are rejected so scope validation cannot be bypassed. Praxis AI recomputes the RFC 8785 canonical SHA-256 digest over network, local_site, and the ordered candidate list before accepting a snapshot.

The revision lifecycle is observable as: - rendered: The producer constructed the envelope. - distributed: The producer applied it to the destination ConfigMap. - accepted: Praxis AI parsed, scope-checked, and digest-verified it. - serving: the active snapshot used for new routing decisions, including an authoritative no-route decision from an empty candidate list.

Invalid cold-start envelopes fail filter construction. Invalid reloads retain the same-process last-known-good snapshot. Envelope-mode provider hops carry the serving revision from the same immutable snapshot used for candidate selection.

Scope: overlay hot reload swaps the candidate list and local_site only. It cannot add or remove load_balancer clusters or change cluster endpoints or TLS configuration; those changes require a full pipeline reload or pod restart. Overlay files never carry credential values. An already-configured projected credential resolver can use a newly selected, already-mounted Secret without rebuilding the filter pipeline. Every cluster name that may appear in any overlay version must already be configured in the downstream load_balancer filter. Pipeline validation checks the clusters this filter reports: the clusters list when configured, otherwise static candidates (they live in the same configuration), always plus management_cluster. Overlay candidates are never reported on their own, so an overlay revision the running gateway accepted cannot fail the next restart. With clusters configured, a reload naming a cluster outside the list is rejected and the previous snapshot is retained; without it, such a reload is accepted and the request fails when the load_balancer cannot resolve the cluster.

Supports two modes:

Static mode — candidates and local_site are specified inline:

Overlay mode — candidates are loaded from an overlay envelope:

clusters lists the clusters the downstream load_balancer declares. When present it is authoritative: pipeline validation checks every entry against the load balancer, and any overlay revision naming another cluster is rejected.

Configuration