Tools

What a tool does

A tool is a named function a model can call while producing a response. Enable catalogue tools for the organisation, then attach them to a model. If a request sends its own tools array, those definitions replace the catalogue for that call. See passthrough mode.

For catalogue tools, the gateway executes the call and returns the result to the model within the same request. For caller-supplied definitions, the client owns execution.

Catalogue tools

Enabling a tool

From Organisation → Tools, select Enable. If the tool requires configuration, enter it before enabling the tool.

Next, open a model under Models and add the tool to its enabled tools. Chat completions for that model can then invoke it.

What you see in the response

Tool execution appears in response metadata:

• inferada.timings.tools_ms: total time spent running tools for this request.
• inferada.tools: one entry per call with its name, latency, and success / error status.
• usage.tools: the same entries, persisted in request logs for the history page.

Limits and safeguards

• Each tool call has a per-call timeout (default 10 s, hard-capped at 30 s).
• The gateway caps tool-call chains at 5 steps by default (configurable per service model), so the model can't spin infinitely.
• API keys you configure are encrypted at rest and never returned to the client or the model.
• Tools that fetch a customer-supplied URL refuse private / link-local / cloud metadata addresses, so scanning your internal network from inside a prompt doesn't work.

Web search

web_search gives a model one search contract regardless of the provider configured for the organisation.

Configuring it

On the Tools page, enable web_search and supply:

• Provider: tavily or brave. Tavily returns extracted page content and a synthesised answer; Brave returns links + snippets only.
• API key: your account's key with the chosen provider. It's stored encrypted at rest.
• Default max results (optional): used when the model omits max_results; defaults to 5.
• Search depth (Tavily only, optional): basic or advanced. Advanced is slower and more expensive but returns richer content. Brave ignores this knob.

The search provider bills the account associated with the configured key; Servada does not rebill those calls. Attach the tool to a model under Models before using it.

What the model sees

The LLM-facing input shape is always:

{
  "query": "EU AI act enforcement timeline",
  "max_results": 5,                           // optional, 1-20
  "freshness": "past_month",                  // optional: anytime | past_day | past_week | past_month | past_year
  "topic": "news"                             // optional: general | news (Tavily honours, Brave ignores)
}

And the result shape is always:

{
  "query": "EU AI act enforcement timeline",
  "answer": "The EU AI Act entered into force on …",   // Tavily only; Brave omits
  "results": [
    {
      "title": "EU AI Act — entry into force",
      "url": "https://example.eu/ai-act",
      "snippet": "Short summary, ~200-300 chars.",
      "content": "Full extracted page text.",          // Tavily only; Brave omits
      "published_date": "2025-04-01T00:00:00.000Z",    // when the provider supplies it
      "score": 0.91                                    // Tavily relevance score
    }
  ]
}

content and answer are not always present: Tavily fills them, Brave doesn't. Agents that depend on the full extracted text should configure a Tavily-backed org; agents that only need ranked links work fine on either provider.

Adding more providers

The provider list is closed for now (Tavily, Brave). If you need Exa, Serper, Linkup, or another search backend, ask Servada to wire it in. The architecture supports adding providers without changing the agent-facing tool name or shape.

Send client-run tool definitions

When a request includes a tools array, those definitions replace the configured catalogue for that request. The gateway returns the model's tool calls to the client, which executes them and sends the results in a later turn.

Streaming a tool call (Chat Completions)

{
  "model": "qwen3.5-35b",
  "stream": true,
  "messages": [
    { "role": "user", "content": "Run git diff and tell me what changed." }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "bash",
        "description": "Execute a shell command and return stdout/stderr.",
        "parameters": {
          "type": "object",
          "properties": { "command": { "type": "string" } },
          "required": ["command"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

The gateway streams OpenAI-compatible tool_calls deltas. The first chunk carries the ID and name, later chunks carry JSON-string fragments, and the stream ends with finish_reason: "tool_calls":

data: {"choices":[{"index":0,"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_a","type":"function","function":{"name":"bash","arguments":""}}]}}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"command\":"}}]}}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"git diff\"}"}}]}}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]

Sending the tool result back

Run the call locally, then post the result as a role: "tool" message alongside the assistant turn the gateway just emitted:

{
  "model": "qwen3.5-35b",
  "messages": [
    { "role": "user", "content": "Run git diff and tell me what changed." },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        { "id": "call_a", "type": "function",
          "function": { "name": "bash", "arguments": "{\"command\":\"git diff\"}" } }
      ]
    },
    { "role": "tool", "tool_call_id": "call_a", "content": "diff --git a/foo …" }
  ],
  "tools": [ /* same array as the first turn */ ]
}

What the gateway skips on this path

• Server-side execution. The gateway never invokes a caller-supplied tool; the client must execute it.
• Output post-processing. The gateway skips anonymise restoration, spellcheck, and other response-text post-processors. The assistant turn is agent payload (text preamble plus tool-call JSON), not human prose, and our prose-cleaning layer would corrupt tool arguments.
• Forced buffering. A processor that would normally force a buffered response when streaming is requested still lets streaming through here, because agent callers ask for streaming on purpose.
• Pre-processors keep running. PII guard, prompt shield, and other input-direction processors still validate and transform the inbound request. The override rule covers tool definitions and upstream output, not your protections on inbound traffic.