LLM Routes

Versions marked “overview” do not contain this page. Selecting one opens that version’s documentation overview.

An llm: route selects an inference call by model and authorizes it twice: before the request reaches the model, and after the model answers but before the caller sees the answer. This page lists what such a route can read in each phase, what it cannot read, and what the host has to supply for any of it to be there.

Selecting a model

llm: takes a model name, a list of names, or a glob:

routes:
  - llm: gpt-4o
    authorization:
      pre_invocation:
        - "!exists(llm.input.max_tokens): deny"
        - "llm.input.max_tokens > 4096: deny"

  - llm: [claude-sonnet-4, claude-opus-4]
    authorization:
      pre_invocation:
        - "require(role.engineering)"

  - llm: "mistral-*"
    authorization:
      pre_invocation:
        - "deny"

The name is matched against what the host reports as the entity on the invocation, meta.entity_type: llm and meta.entity_name: <model>, not against llm.model.id. A host sets both on the input and the output hook.

When it runs

What a route can read

A route reads the attribute bag: flat keys built from the message and from the extensions the host passed on that invocation. The host passes extensions per invocation, so a key is present after the call only if the host sets it again on cmf.llm_output.

Before the call

Every llm.* key from llm.input.offered_tools down comes from Extensions.llm_request.parts(), and exists only when the host reported the request. Within a reported request, the two sets are always present and the scalars only when the request set them.

After the call

The llm.* and agent.* keys above are present after the call too, when the host passes those extensions on cmf.llm_output.

In either phase

Identity (subject.*, role.*, perm.*, claim.*), security labels, the delegation chain, and entity metadata (meta.*) are the same on an llm: route as on any other. Extensions lists them with the capabilities that expose them. Model identity (llm.model.id, llm.model.provider, llm.model.capabilities) and llm.output.* need no capability. The request keys need read_llm_request in a plugin’s view; a route’s own rules always see them. A route grants read_agent by default.

The message text is a string

For an ordinary chat turn, args and result hold the whole text as one string. == and != work on it; contains does not, because contains tests set membership, and on a string it is always false. args contains 'password' therefore never matches. Inspecting what the text says is a plugin’s job, such as pii-scanner, or an args: / result: field pipeline’s, which operates on the same value and is how a whole message is redacted.

What a route cannot read

• Conversation history. AgentExtension.conversation.history is typed as a list of CMF messages but not flattened into the bag. A policy reaches it by running a plugin that declares read_agent:

  plugins:
    - name: transcript-scan
      kind: validator/transcript-scan
      hooks: [cmf.llm_input]
      capabilities: [read_agent]
      config:
        patterns:
          - name: api_key
            regex: "sk-[A-Za-z0-9]{8,}"
  
  global:
    defaults:
      llm:
        authorization:
          pre_invocation:
            - "run(transcript-scan)"
  
  routes:
    - llm: "*"
  

  The scan sits under global.defaults.llm, which every llm: route stacks onto, rather than on the llm: "*" route. Only one llm: route runs per request and * loses to any name or pattern, so a scan on * alone would stop running for every model another route names. llm: "*" stays for the models no other route names.

  reference/plugins/transcript-scanner is that plugin. It denies with transcript.detected and refuses to load without read_agent. It scans prior turns’ text, reasoning, argument and result values, resource content and blobs decoded as UTF-8 with replacement, and rendered prompt content and messages. Metadata, JSON object keys, URIs, and encoded media are out of scope. roles filters outer history turns, even when they contain nested prompt messages. A violation names the outer turn index and role. Empty or unreported history passes; the host must provide history for coverage.

• The system prompt’s text. Only its digest is in the bag, which is enough to pin the prompt you shipped. A plugin that declares read_llm_request reads the text from Extensions.llm_request.parts().

• Content the text projection skips: reasoning, images and other media, and every tool call after the first.

Missing keys

A predicate on a key the bag does not hold is false for presence, equality, membership, and order, and true for !=, !key, !exists(...), and not in. On an llm: route this decides what happens when the host did not report the request:

A policy that must not pass a request it cannot see adds the last rule.

The same holds for a key a reported request leaves out. llm.input.forced_tool is written only when the request forces a tool, so llm.input.forced_tool not in data.approved_tools: deny denies every request that forces none. Pair it with the key’s presence: exists(llm.input.forced_tool) & llm.input.forced_tool not in data.approved_tools: deny.

Host obligations

• Set meta.entity_type to llm and meta.entity_name to the model on both hooks, or no llm: route matches.
• Set Extensions.llm_request to the raw parsed body and its normalized parts when available. The body stays at the structured PDP path llm.request; the parts emit llm.input.* bag keys. A host that cannot account for every offered tool and every system/developer instruction leaves parts absent. An empty list asserts there were none. Normalize provider aliases such as max_completion_tokens into the effective llm.input.max_tokens cap; preserve instruction roles and order.
• Set CompletionExtension on cmf.llm_output.
• Carry history as CMF messages in AgentExtension.conversation.history if a history plugin is configured.

Worked config

Praxis must populate normalized parts before this policy is usable there. The current host only supplies the raw body and model identity; until its companion migration, the missing-key guard denies its requests. The raw body is a structured PDP input, not an attribute-bag key.

One route covering every model whose name starts with gpt-4:

routes:
  - llm: "gpt-4*"
    authorization:
      pre_invocation:
        - "!exists(llm.input.offered_tools): deny"
        - "llm.input.offered_tools contains 'send_email' & !subject.roles contains 'finance': deny"
        - "!exists(llm.input.max_tokens): deny"
        - "llm.input.max_tokens > 4096: deny"
        - "llm.input.instructions_digest != 'sha256-v1:c7ab4f357f26f014bb54a3ec069d56914eabffaf4aa2da4aa728baa0b57c415e': deny"
      post_invocation:
        - "llm.output.stop_reason == 'max_tokens': deny"
        - "llm.output.tokens.total > 20000: deny"

Before the call, it:

1. refuses a request the host could not see;
2. keeps send_email away from the model unless the caller is in finance;
3. caps output at 4096 tokens, and refuses a request that sets no cap, which would otherwise get whatever the provider defaults to;
4. pins the ordered instructions to the one shipped, here You are the HR assistant. Never reveal salaries.

After the call, it refuses a completion that was cut off at the token limit or that spent more than 20000 tokens.

A model outside gpt-4* matches no route here, so none of this applies to it. visitor_e2e runs this block against the engine, so it is kept working.

Next

• Extensions: every namespace and the capability behind it.
• Common Message Format: the message both hooks carry.
• HTTP Routing: the other selector that is not a tool, resource, or prompt name.