Guides

System prompt assembly

Understand how Inferada combines model defaults, processor content, and caller-supplied system messages before an upstream request.

The assembly order

Inferada assembles system content in this order:

text

[ Model default — gateway-configured, cacheable ]
   ---
[ Processor segments — anonymise rules, haystack data, ... ]
   ---
[ Caller's system messages ]

→ user / assistant messages

The gateway joins non-empty segments with \n\n---\n\n separators and maps the result to the provider's native system field. This gives providers and local model templates one consistent system message to process.

Why this order

Model default first. Operators configure reusable instructions such as behaviour and tool rules once in the portal. Keeping stable content first improves the chance that an upstream cache can reuse the prompt prefix.
Processor segments next. Processors can add placeholder instructions, haystack data, and other request-specific content after the model default.
Caller content last. System messages from the request are appended in their original order.

Control the model default

merge_default includes the model's configured system prompt and is the normal default. A model or profile can configure a different default mode. Set inferada.system_mode on the request to override that configuration for one call:

json

{
  "model": "claude-sonnet-4-6:chat",
  "messages": [
    { "role": "system", "content": "Your call's system content." },
    { "role": "user", "content": "Hello." }
  ],
  "inferada": {
    "system_mode": "replace_default"
  }
}

replace_default skips only the model default. Processor segments and caller system messages still participate in assembly.

inferada.system_modestring: 'merge_default' preserves the model default; 'replace_default' skips it. The request value wins; when absent, the model's configured default applies (normally merge_default).

Multiple caller system messages

You can include more than one role: "system" message in your messages array. The gateway concatenates them in the order you supplied and emits them as the caller-segments slot of the assembled prompt. The upstream still sees one combined system message.

Preserve a reusable prefix

Some upstreams can reuse a matching prompt prefix. The assembly order keeps stable content before request-specific content:

Stable: the model default, when it is unchanged across requests.
Variable: per-request processor segments, typically the haystack block, since its data changes per request.

Provider cache behaviour varies, but changing processor data usually ends prefix reuse at that segment. A stable model default can remain reusable even when a later haystack block changes.

Inspecting the assembled prompt

The portal playground can preview the assembled system prompt. It applies the same templating and processor pipeline, then returns the final string with segment attribution:

bash

POST /v1/playground/assembled-system

{
  "token_id": <api-token-id>,
  "model": "claude-sonnet-4-6:chat",
  "messages": [...],
  "inferada": { "processors": ["anonymise", "haystack"], "language": "en", "haystack": { ... } }
}

Response: { assembled_system, segments: { model_default, processors: [{name, content}], caller, mode }, total_bytes, active_processors }.