Services

Default and explicit routing

An organisation can configure more than one service for a scope. For example, two stt services may use different speech models or upstreams. Each service has its own configuration and limits. Requests without service_id use the service marked is_default; requests with an ID use that service explicitly.

Completion services work differently: each attached model defines its upstream binding. The gateway resolves model across enabled completion services. Set inferada.service_id only when you need to choose among services that expose the same model name. Other scopes accept service_id at the top level.

Fallback upstreams for tooling

Decisions, PII analysis and redaction, language checking and correction, OCR, speech recognition and speech synthesis can use one primary and up to four ordered fallback upstreams within the same service. Configure them under Fallback upstreams in the service settings. The administration API accepts an ordered fallback_upstream_idsarray; an empty array disables fallback. Existing completion routing is unchanged.

Choose replicas that accept the same request contract: model identifiers, languages, voices and output formats. No compatibility key or extra service is needed. Inferada checks organisation ownership, scope, API format, processing class and billing mode. PII fallbacks must support the primary’s languages; TTS fallbacks must use the same upstream template as the selected backend. Combined TTS services retain the requested Piper or Kokoro backend and voice. Re-check compatibility when changing upstream models.

Inactive targets are skipped. Connection failures, timeouts, HTTP 408, 429, 500, 502, 503, 504 and 529 can trigger the next target. HTTP 400, 404 or 422 only triggers fallback when its JSON error code is model_not_found or unsupported_model. Other validation, authentication and policy failures stop immediately. Requested model identifiers and service defaults stay fixed across attempts. Speech output is never replayed after it is handed to the client.

The first active target’s effective timeout bounds the attempt chain. Each attempt uses its own timeout, capped to the remaining request budget. Adding a fallback does not shorten an active attempt. A failure can trigger fallback only while time remains; a target that consumes the whole budget leaves no time for another attempt. A timed-out or disconnected request may still be running on the backend. Upstreams use their own credentials and headers. Request history contains one logical request with a separate upstream log for each attempted target; limits and usage are applied once. Voice catalogue discovery and completion processors keep their existing routing.

Model IDs and profiles

A completion model may use the form <base>:<profile>, such as gpt-4o:chat. The base identifies and prices the upstream model. A profile inherits from that base and adds behaviour defaults such as a system prompt or sampling parameters. An ID without a profile suffix addresses the base model.

A base is tied to a protocol shape (openai-compatible, anthropic-compatible, …), not to a specific provider account. The same upstream model_id can therefore exist as two separate bases, one per protocol, each with its own pricing. Within one protocol there is exactly one base per model_id, shared by every organisation that routes through it.

Pricing always resolves through the parent base; see How pricing inherits for the resolution chain.

Override inherited model settings

When you attach a profile to a service model, the profile's system prompt, default parameters, processor list, and processor settings become the model's defaults. The Behaviour and Processors tabs each expose an Override toggle per field so you can replace any of those bundles for a specific model.

An override replaces the complete bundle; it does not merge individual keys. With the toggle off, the model uses the profile's bundle. With it on, enter every value the model should use. Unspecified default_params fall back to provider defaults, and processors or processor settings from the profile no longer apply. Turn the override off to restore inheritance.

To add one processor to an inherited list, override the processor bundle and select both the inherited processors and the new one.

List services

curl https://api.inferada.com/v1/services?scope=stt \
  -H "Authorization: Bearer inf_YOUR_TOKEN"

{
  "services": [
    {
      "id": "01J5Z...",
      "scope": "stt",
      "name": "Speech-to-Text (Local Whisper)",
      "active": true,
      "is_default": true
    },
    {
      "id": "01J6A...",
      "scope": "stt",
      "name": "Speech-to-Text (OpenAI)",
      "active": true,
      "is_default": false
    }
  ]
}

Route a request with service_id

Pass service_id alongside the normal request body (or as a form field for multipart endpoints). The gateway validates that the service belongs to your org and matches the endpoint's scope, then routes there instead of the default.

curl https://api.inferada.com/v1/audio/transcriptions \
  -H "Authorization: Bearer inf_YOUR_TOKEN" \
  -F service_id=01J6A... \
  -F file=@speech.mp3

An unknown service_id returns 404. An ID from another organisation or the wrong scope returns 403.

For chat completions and responses, keep the standard model field and place the optional selector under our additive namespace: "inferada": { "service_id": "01J6A..." }. Most callers should omit it; model-driven routing selects the unique enabled service automatically.