How it works.

The path chooses the provider.

A provider entry in your configuration names an upstream URL, its API format, and its authentication policy. The name in /provider/<name>/… selects that entry. On a native route, with no bridge configured, Torana removes the prefix and joins the remaining path to the upstream URL.

Harness sends

<Torana origin>/provider/primary/v1/messages

Torana selects

The primary provider entry

Upstream receives

/v1/messages at that entry’s URL

A configured default provider can handle unprefixed requests. Native routes keep the client and upstream on the same API. An opt-in protocol bridge explicitly names a client contract and an upstream contract, then builds the destination’s endpoint and request. For example, Torana can translate Anthropic Messages to OpenAI Chat Completions and return an Anthropic-shaped response. No plugin is required.

Native routing stays within the same configured format. Compatible bridge targets share the same bridge.client contract, even when their upstream APIs differ. A permitted routing plugin can select a compatible target; configured fallbacks rebuild the accepted request for their own protocol, model, and credentials. Request plugins run once.

Only recognized inference endpoints enter the request and response plugin pipeline. On native routes, model lists, token-counting, account APIs, and other auxiliary paths pass through as ordinary proxy traffic. Under any configured bridge, auxiliary APIs are not emulated or forwarded: they return 400, even when the client and upstream contracts match. Torana does not intercept the harness’s local tool execution or every MCP connection.

Configure a bridge →Read the path resolver ↗

One format for plugin logic.

Each supported provider adapter reads the incoming JSON into a shared request representation. Plugin authors work with messages, tools, model settings, and ordered content blocks instead of writing a separate JSON parser for each provider. The technical name is an intermediate representation, or IR.

OpenAI

POST …/chat/completions
POST …/responses

Anthropic

POST …/messages

Gemini / Vertex AI

POST …:generateContent
POST …:streamGenerateContent

Gemini Code Assist

The same generation methods, with the Code Assist request envelope

On native routes, text, thinking, tool calls, tool results, cache markers, and provider-specific blocks retain their order. Provider extensions and opaque fields have dedicated preservation paths. An accepted change is encoded back into the original API shape. When no provider-visible state changes, Torana keeps the validated request body’s original bytes.

A cross-contract bridge instead converts the accepted shared request to the configured upstream API. Common text and function-tool loops work across supported contracts; signed reasoning, cache breakpoints, opaque continuation items, and provider-native tools are not portable. Unsupported request semantics are rejected before an upstream call rather than silently discarded.

Shared plugin logic is reusable across compatible APIs; individual features still have provider-specific rules. On native Responses routes, a plugin cannot rewrite history held by OpenAI behind previous_response_id. A cross-contract bridge rejects that server-managed history reference and requires complete history on each turn. Torana does not manage Gemini’s remote cache resources.

Check the support matrix →Read the shared types ↗

Plugins run at defined points.

Go and Rust plugins compile to WebAssembly and use Torana’s ABI v1 SDK. A plugin declares the hooks it implements. Torana runs enabled plugins in the configured order, checks their output, and passes each accepted change to the next plugin.

On a bridge, request hooks see shared client content and the configured upstream model alias. Response hooks see the actual upstream’s shared response before client serialization. Host accounting retains upstream-native usage; client-facing usage is converted where representable. The bridge guide explains these boundaries.

1. Before the upstream request

   run_before_request

   A hook receives the shared request. It can pass it through or return a replacement within its write grants. Separate, permission-checked host calls can block the request, provide a synthetic response, select a route, or set its identity. An accepted block stops the request before it reaches the provider.

2. As response events arrive

   run_on_stream_chunk

   The adapter turns the provider’s stream into typed events: text, thinking, tool-call fragments, boundaries, usage, and completion. Hooks can observe events, replace allowed content, or use the permitted stream operations to buffer, drop, emit, or stop. Torana validates event order and protected content before serializing it back to the harness.

3. When the response is complete

   run_after_response

   For a non-streaming JSON response, this hook can change the response before it is sent. JSON responses first pass through the stream hooks as synthetic events. For a streaming response, the final hook receives status, timing, and usage with mutable=false and no assembled message body. Stream content must be inspected in the stream hook; already-sent bytes cannot be rewritten.

Plugins can also declare run_on_http_request to serve an approved local UI or API, and run_on_tick for scheduled work. Those hooks are separate from the inference request lifecycle.

If a plugin fails

The approved failure_mode controls traps, timeouts, and invalid hook output. With pass, Torana rejects the failed plugin’s replacement and continues with the last accepted state. With block, it refuses the request or unsent response; during a stream, it terminates the stream. A failure in the final observational streaming hook is recorded, since the body has already been sent.

An explicit block remains effective even if that plugin later fails. Invalid routing targets keep the original route. Malformed inference input is a host validation error and is rejected before upstream, independently of a plugin’s failure mode.

A signed-stream violation discovered after affected bytes were sent terminates the stream even in pass mode. Torana cannot safely roll back content the harness already received.

Read the hook pipeline ↗Read SDK dispatch ↗

Installation is followed by approval.

A plugin bundle contains its compiled WASM, a manifest, and optional configuration schema and agent descriptor. Torana computes a SHA-256 digest over those files. Approval binds the executable and its declared behavior to the exact bundle you reviewed. Changing any of those files requires a new approval.

The manifest requests capabilities; the operator approves them and enables the plugin. Every requested permission must be approved before it can run. Write grants are narrow: changing tool-result text, tool definitions, or the model requires the corresponding grant. File, credential, outbound HTTP, and model-service access also use approved resource bindings and limits.

WASM guests run in a wazero sandbox with bounded memory, instance counts, and execution time. They have no ambient filesystem or network access. Torana brokers approved access through host calls. System clocks are available to guests, so deterministic transformations are an authoring responsibility.

Approval does not make arbitrary output valid. Torana still checks request shape, provider constraints, host-owned metadata, and signature provenance, including for a plugin with every write grant.

Review plugin installation →Read the sandbox ↗

A local view, for people and agents.

Run torana open to open the dashboard for the live instance. It shows request activity and usage, provider configuration, and the installed plugin pipeline. You can inspect permissions and configure the plugins you enable.

The JSON API at /_torana/api/v1/ describes available operations with their paths, input and output schemas, and declared risk. Local software agents can read the request feed and configuration, manage plugins, and discover operations contributed by enabled, approved plugins.

These control routes are loopback-only. Browser mutations require a matching local origin; non-browser mutations require X-Torana-Local-Request: 1. The conversation list contains metadata, not a copy of the conversation’s messages. Treat software with access to the local administration API as an operator.

The CLI uses this same local API to inspect traffic, apply revision-checked settings including bridges, and manage the pipeline. Source-build users run ./torana from their checkout; the reference uses torana on PATH.

Use the CLI ↗Explore the agent API ↗

What stays local, and what leaves.

The proxy, WASM execution, dashboard, and default local stores run on your machine. Torana does not require a Torana account or a hosted Torana service. That does not mean the conversation is offline.

• Provider traffic: the request and its configured credentials go to the selected upstream. A remote provider receives that content; a supported local model endpoint can keep that hop local.
• Plugin services: approved HTTP and model host calls can send plugin-selected data to their configured endpoints. A compactor using a remote summarization model makes an additional model request.
• Optional infrastructure: a configured Redis cache or OpenTelemetry exporter can send data to those services. Plugin logging and persistence depend on the plugins and storage configuration you choose.

Review a plugin’s source, requested capabilities, resource bindings, and configuration to understand its data flow. “Runs locally” describes where Torana executes.

Cache identity and signatures matter.

Providers often cache a repeated prompt prefix. A transformation of the same tools, system prompt, and history should produce the same result every time. Adding timestamps, random IDs, or text from the newest turn to that prefix can turn a cache hit into a miss. Preserve existing cache markers unless a permitted transformation intentionally changes them.

Some provider content carries opaque signatures. These tokens are tied to particular content and positions. A plugin cannot copy a signature onto unrelated content or keep a stale signature over changed content. The host enforces those bindings; authors should use the SDK’s provenance-aware helpers and respect each provider’s mutation rules.

Those preservation rules describe native routes. A mismatched protocol bridge cannot carry provider-specific signatures or cache breakpoints into an unrelated API. Prefer a native route for workflows relying on them.

A compaction plugin also changes what the next model call can see. Its value depends on the workload, retained information, extra model calls, and provider cache behavior. Torana provides the hooks and measurements; it does not promise that compaction saves money.

Read the caching rules ↗Read mutation checks ↗

Your agent can ask Torana, too.

Traffic interception and MCP are separate paths. Hooks run automatically as model requests and responses pass through the proxy. Torana’s optional local MCP server lets your harness discover and call explicit operations: inspect usage, ask why PII withheld a result, or request a confirmed plugin change.

Plugins declare operations in agent.json and handle them through their existing WASM HTTP hook. Torana exposes four discovery/invocation tools rather than adding every plugin operation to the harness’s tool list. Session-specific calls require a verified match to the model tool call observed in proxied traffic; user confirmation governs eligible changes.

Connect MCP tools →Expose an operation →