Model To Header Trust Existing

Routes on a model header that a trusted hop in front of Praxis has already set, skipping the filter entirely when the header is present

Versions marked “overview” do not contain this page. Selecting one opens that version’s documentation overview.

Category: Practical
Task: Routes on a model header that a trusted hop in front of Praxis has already set, skipping the filter entirely when the header is present

Prerequisites: The product runtime and any backend services referenced by this configuration.

Run it: Use ghcr.io/praxis-proxy/ai:0.8.0 and follow the container quickstart to mount and start the configuration.

This configuration comes from the selected release. The example has not been run here; external services are not bundled.

Download the source file.

# AI Inference Model Routing: Trust an Existing Model Header
#
# Routes on a model header that a trusted hop in front of Praxis has
# already set, skipping the filter entirely when the header is present.
# The `unless: {headers_present: ["X-AI-Model"]}` condition keeps the
# incoming header as is. A request without one still has the body's
# "model" field promoted into X-AI-Model by the filter, as usual.
#
# Security: this turns off the anti-spoofing strip when the header is
# present, so whoever sends the header picks the backend. Only use it
# when every request reaches Praxis through a hop that sets or scrubs
# the header.
#
listeners:
  - name: inference-gateway
    address: "127.0.0.1:8080"
    filter_chains:
      - extract-model
      - routing

filter_chains:
  # Skip the filter when X-AI-Model is already present from a trusted
  # hop; otherwise promote the body's "model" field into it.
  - name: extract-model
    filters:
      - filter: model_to_header
        conditions:
          - unless:
              headers_present: ["X-AI-Model"]
        header: X-AI-Model

  # Route based on X-AI-Model, wherever it came from.
  - name: routing
    filters:
      - filter: router
        routes:
          - path_prefix: "/"
            headers:
              x-ai-model: "mistral-7b-instruct"
            cluster: mistral
          - path_prefix: "/"
            headers:
              x-ai-model: "granite-3.1-8b"
            cluster: granite
          - path_prefix: "/"
            cluster: default
      - filter: load_balancer
        clusters:
          - name: mistral
            endpoints:
              - "127.0.0.1:3000"
          - name: granite
            endpoints:
              - "127.0.0.1:3001"
          - name: default
            endpoints:
              - "127.0.0.1:3002"

insecure_options:
  allow_private_endpoints: true # example proxies to local backends