Haystack

When to use it

Use haystack for structured context that belongs to one request, such as a customer profile, recent orders, account state, or support tickets. The gateway renders the data into the assembled system prompt. Haystack is not a vector store, embedding cache, or persistent memory; send the required data with each request.

Compared with formatting this data manually, haystack provides:

• Schema-guided PII handling for individual fields.
• Dot-path templating: your system prompts can reference {{ haystack.customer.name }} directly. See Prompt templating.

Quick start

Add haystack to the processor list and provide data:

{
  "model": "claude-sonnet-4-6:chat",
  "messages": [
    { "role": "system", "content": "Answer the user using the attached customer context." },
    { "role": "user", "content": "Why was my latest order delayed?" }
  ],
  "inferada": {
    "processors": ["haystack"],
    "language": "en",
    "haystack": {
      "data": {
        "customer": { "name": "Jane Doe", "email": "jane@example.com", "tier": "pro" },
        "orders": [
          { "id": "ord_1", "total": 49.99, "status": "delayed" },
          { "id": "ord_2", "total": 19.99, "status": "shipped" }
        ]
      }
    }
  }
}

The gateway adds a YAML <haystack> block to the assembled system prompt. When redaction is enabled, it replaces detected values with <INF_HAY_TYPE_N> placeholders before the upstream call and restores matching placeholders in a buffered response.

Schema

A schema is optional. Add one when the model needs field descriptions or redaction needs explicit instructions for individual fields.

{
  "customer": {
    "type": "object",
    "description": "The end-user we're answering questions about",
    "properties": {
      "name":  { "type": "string",   "description": "Full legal name", "pii": "PERSON" },
      "email": { "type": "string",   "description": "Primary contact email", "pii": "EMAIL_ADDRESS" },
      "tier":  { "type": "enum",     "values": ["free", "pro", "enterprise"], "description": "Subscription tier" }
    }
  },
  "orders": {
    "type": "array",
    "description": "Customer's recent orders, newest first",
    "items": {
      "type": "object",
      "properties": {
        "id":     { "type": "id",       "description": "Internal order ID", "pii": false },
        "total":  { "type": "currency", "currency": "EUR", "description": "Order total" },
        "status": { "type": "enum",     "values": ["shipped", "pending", "cancelled", "delayed"] }
      }
    }
  }
}

Field-def keys:

typerequiredstring

string | number | boolean | null | object | array | enum | date | datetime | currency | id | markdown | code

descriptionstring

A short explanation of the field. Rendered as an inline comment in YAML and Markdown output.

piistring | boolean

false to skip redaction, true or "auto" to use key heuristics and detection, or an entity type such as "PERSON" to assign that placeholder without detection.

valuesarray

Allowed values for type: "enum". Strings or numbers.

currencystring

ISO 4217 code (EUR, USD, ...). Only meaningful for type: "currency".

propertiesobject

Children for type: "object"; same shape as the top-level schema.

itemsobject

Element shape for type: "array".

The full machine-readable contract lives at https://api.inferada.com/schemas/haystack/v1.json. Use that $schema URL for editor autocomplete and validation. The inferada-haystack-author skill will generate a schema from a TypeScript type or sample JSON for you. Download it from https://api.inferada.com/skills/inferada-haystack-author.zip.

PII redaction

With redact: true, the processor visits each string leaf in this order:

1. Schema pii: false → never scanned.
2. Schema pii: "PERSON" (or another explicit type) → assign that placeholder without detection.
3. Schema pii: true, "auto", or absent → apply key heuristics and statistical detection. Keys such as email, phone, name, address, iban bias detection toward the matching entity type.

Placeholders use the form <INF_HAY_TYPE_N>, disjoint from the <INF_TYPE_N> namespace used by the anonymise processor for free-text user messages, so the two compose without collision. Restoration in the response is buffered-only: streaming responses include the redacted placeholders verbatim.

For free-text user/assistant messages, use anonymise. For caller-side structured data, use haystack with redact: true. You can enable both on the same request. See PII for the underlying detection endpoints.

Output format

The rendered block goes into a system message wrapped in an <haystack> tag. Set format to choose how the body is encoded.

yamldefault

Compact representation for nested data. Schema descriptions render as inline # comments; currency and date values include their unit.

jsonoverride

Pretty-printed JSON without comments. Use it when the model needs to reproduce the data as structured output.

markdownoverride

Headings, bullet lists, and blockquoted descriptions for prose-oriented context.

xmloverride

One element per key, with sanitised tag names. Use it when your model or prompt expects XML.

Limits

The gateway validates haystack data before the model call and returns 400 on the first violation:

• Serialised JSON ≤ 64 KB.
• Depth ≤ 8 nested levels.
• Array length ≤ 500 entries per node.
• Total leaves ≤ 2,000.
• Schema type/enum mismatch (e.g. number declared, string sent).

Error responses include a reason tag (haystack_too_large, haystack_too_deep, haystack_schema_mismatch, …) and the failing path. See Errors for the response shape.

Templating access

Haystack also makes its data available to system-prompt templates. Reference a field with {{ haystack.<path> }}, with a dot for object access and brackets for array indices:

Greet {{ haystack.customer.name }}.
The most recent order total was {{ haystack.orders[0].total }}.
All tiers: {{ haystack.tiers }}        ← arrays/objects render as JSON automatically

With redact: true the substituted values are the redacted ones (<INF_HAY_PERSON_1> instead of the real name), and the response-side restore covers them too. See Prompt templating for path-resolution and escaping rules.

Reference

inferada.processorsrequiredarray

Must include "haystack" to enable the feature.

inferada.languagerequiredstring

BCP-47 / ISO-639-1 language code. Required when redaction is on (always, by default).

inferada.haystack.datarequiredobject | array

Deep JSON tree. Leaves must be string, number, boolean, or null.

inferada.haystack.schemaobject

Optional. Field-by-field metadata; see the Schema section above.

inferada.haystack.formatstring

yaml | json | markdown | xml. Default: yaml.

inferada.haystack.redactboolean

Default: true when the org has a PII service configured. Set false to skip redaction even when configured.