> ## Documentation Index > Fetch the complete documentation index at: https://docs.igrisecurity.com/llms.txt > Use this file to discover all available pages before exploring further. # LLM Gateway > Route any LLM provider through Igris for audit, policies, and cost tracking in 5 minutes. # LLM Gateway Turn on governance for your LLM calls in 5 minutes. Works with OpenAI, Anthropic, Google Gemini, Groq, Mistral, Cohere, and 54 more providers out of the box. ## Quickstart ### 1. Get an Igris API key Sign up at [app.igrisecurity.com/signup](https://app.igrisecurity.com/signup), then create a new API key under **Settings → API Keys**. Your key starts with `ig_`. ### 2. Create a connection for your LLM provider Connections are encrypted credential vaults. They bind your upstream provider API key to an Igris slug you can reference from code without ever exposing the real key. 1. Go to **Connections** in the dashboard 2. Click **New Connection** 3. Select type **LLM** 4. Pick your provider — e.g. **OpenAI** 5. Paste your provider API key 6. Optionally restrict to specific models (e.g. `gpt-4o`, `gpt-4o-mini`) 7. Save and note the slug (e.g. `openai-prod`) ### 3. Make your first call The model field uses `@slug/model` syntax. Igris resolves the slug, injects credentials, enforces policies, and logs everything — before forwarding to the upstream provider. ```ts theme={null} import { Igris } from "@igris-security/sdk"; const igris = new Igris({ apiKey: "ig_..." }); const response = await igris.chat.completions.create({ model: "@openai-prod/gpt-4o", messages: [{ role: "user", content: "Hello!" }], user: "alice@corp.com", }); console.log(response.choices[0].message.content); ``` ### 4. See it in the audit trail Back in the dashboard, open **LLM → Audit Trail**. You'll see your call with: * Provider + model used * Input and output token counts * Cost in USD (calculated server-side) * End-to-end latency * User identity passed via `user` field * Policy action (allowed / denied / alerted) Head to **LLM → Cost** for spend breakdowns over time. ## Streaming Pass `stream: true` to get an `AsyncIterable` of SSE chunks. The gateway uses minimal SSE-parse buffering to extract usage data — no extra round-trips are added. ```ts theme={null} const stream = await igris.chat.completions.create({ model: "@openai-prod/gpt-4o", messages: [{ role: "user", content: "Write a haiku about governance" }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ""); } ``` Token usage is extracted from the final SSE chunk (or the terminal usage field for providers that support it) and recorded in the audit trail even for streaming responses. ## Passing metadata for policy conditions The `user` field and arbitrary metadata can be passed via extra request headers. The SDK's `connectLlm()` method is the cleanest way to attach per-request context: ```ts theme={null} const conn = igris.connectLlm("openai-prod", { user: "alice@corp.com", traceId: "trace-abc123", metadata: { role: "developer", team: "platform" }, }); // Use conn.baseUrl + conn.apiKey + conn.headers with any OpenAI-compatible client ``` ## Endpoint and protocol The gateway is OpenAI-compatible. All requests target: ``` https://api.igrisecurity.com/llm/:slug/* ``` For example, a chat completions call via raw HTTP: ```bash theme={null} curl https://api.igrisecurity.com/llm/openai-prod/v1/chat/completions \ -H "Authorization: Bearer ig_..." \ -H "Content-Type: application/json" \ -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}' ``` ## Concepts ### Connections A **connection** is an encrypted credential vault that maps an Igris slug to an upstream provider API key. Your application code never holds the real provider key — it only knows the slug (e.g. `openai-prod`). The gateway resolves the slug at request time, injects the real credential, and forwards to the upstream provider. Connections can be scoped to a specific organization, restricted to a subset of models, enabled or disabled without a code change, and rotated by updating the vault entry in the dashboard. A single slug can be reused across as many callers as you like — policy rules target the slug. See [Connections](/connect/connections) for CRUD operations via the API. ### Providers A **provider** is a registered upstream LLM service — OpenAI, Anthropic, Groq, Mistral, and [56 more](/connect/providers) — 60 in total. Each provider registration captures: * **Slug** — the identifier used in connection creation and model routing (e.g. `openai`, `anthropic`) * **Base URL** — where the gateway forwards requests * **Auth style** — `bearer`, `x-api-key`, or `query-param` * **Supported endpoints** — which of `chat.completions`, `embeddings`, `images.generate`, `audio.transcriptions`, `audio.speech`, or `passthrough` this provider supports Providers with `baseUrl: null` (Ollama, HuggingFace, Triton, Modal, `openai-compatible`) require a `customBaseUrl` to be set on the connection for self-hosted deployments. See the full [Provider Catalog](/connect/providers). ### Policies A **policy** is an ordered list of `PolicyRule` objects attached to an organization or connection. Rules are evaluated in order — first match wins. Each rule has four main parts: #### Target What the rule matches. Three discriminated union variants: ```ts theme={null} // Match a specific LLM model or a glob { kind: "llm_model"; model: "gpt-4o" } { kind: "llm_model"; model: "gpt-4*" } // Match a specific endpoint { kind: "llm_endpoint"; endpoint: "chat.completions" } // Match an MCP tool (for MCP governance rules in the same policy) { kind: "mcp_tool"; tool: "delete_*" } ``` #### Action What happens when the rule matches: `"allow"`, `"deny"`, or `"alert"`. Deny returns HTTP 403. Alert records an alert event without blocking the request. #### Conditions Optional metadata conditions that must also match for the rule to fire. Keys are dotted paths into the request context (e.g. `metadata.role`, `user`). Operators: `eq`, `neq`, `in`, `nin`. ```json theme={null} { "conditions": { "metadata.role": { "in": ["intern", "contractor"] } } } ``` #### Limits, Guards, and Content Controls Rules can also carry: * **`limit`** — rate limit on requests, tokens, or dollars per minute/hour/day * **`tokenGuard`** — cap `max_tokens` or reject requests exceeding an input token estimate * **`contentGuard`** — runs detectors (PII, secrets, prompt injection, custom regex/keywords) over the prompt and optionally the response, with `action: "deny" | "redact" | "alert"` * **`logContent`** — whether to persist the full prompt and completion to `llm_call_bodies` ### Audit Trail Every request through the gateway produces an **audit event** with: * `type: "llm_call"` * Provider + model (resolved after gateway routing) * `inputTokens`, `outputTokens`, `cachedTokens` * `costCents` (integer, USD cents × 100 — sub-cent precision) * Latency in milliseconds * `userId`, `traceId`, `connectionSlug` * Policy action taken * `requestId` for correlation Query audit events from the SDK with `igris.auditEvents.list()` or browse them in the dashboard under **LLM → Audit Trail**. If `logContent: true` is set on a matching rule, the full prompt and completion are stored in `llm_call_bodies` and linked via the audit event ID. ### Cost Tracking Cost is computed server-side using a live pricing snapshot. The `costCents` column stores the value as an integer representing USD cents × 100 — so `150` means \$0.0150. The **LLM → Cost** dashboard aggregates spend by day, provider, model, connection, and user. Anomaly detection also monitors cost-per-minute as one of its five signal dimensions. ### Anomaly Detection The `LlmAnomalyDetector` runs five parallel signal trackers on every request: | Signal | Trigger | | --------------- | -------------------------------------------------------- | | Cost spike | Cost/minute exceeds baseline EWMA by configurable factor | | Token burn | Tokens/minute exceeds threshold | | Response length | Output tokens for a single response exceeds threshold | | Model shift | Observed model diverges from expected model baseline | | Error rate | HTTP 4xx/5xx rate exceeds threshold | When a signal fires, an alert event is written and optionally delivered via webhook. Alerts do not block requests by default — configure a `deny` action on the policy rule to make them blocking. ## Three Usage Styles ### 1. SDK native (recommended) Use `igris.chat.completions.create()` directly. The `@slug/model` model prefix tells the SDK which connection to route through. Zero external dependencies beyond the Igris SDK. ```ts theme={null} const igris = new Igris({ apiKey: "ig_..." }); await igris.chat.completions.create({ model: "@openai-prod/gpt-4o", messages: [...] }); ``` ### 2. connectLlm escape hatch Use `igris.connectLlm(slug, options)` to get a `{ baseUrl, apiKey, headers }` object and wire it into any OpenAI-compatible SDK client. This is the zero-migration path when your app already uses the OpenAI Node SDK. ```ts theme={null} import OpenAI from "openai"; import { withIgris } from "@igris-security/sdk/adapters/openai"; const igris = new Igris({ apiKey: "ig_..." }); const openai = withIgris(new OpenAI({ apiKey: "placeholder" }), igris, "openai-prod"); // Now openai.chat.completions.create() routes through Igris ``` ### 3. Raw HTTP Any HTTP client that can set `Authorization: Bearer ` and target `https://api.igrisecurity.com/llm//v1/chat/completions` works directly. This is useful for language runtimes without an Igris SDK (Python, Go, Rust, etc.). ```bash theme={null} curl https://api.igrisecurity.com/llm/openai-prod/v1/chat/completions \ -H "Authorization: Bearer ig_..." \ -H "Content-Type: application/json" \ -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}' ``` ## Request Modes: Passthrough vs Transformed Most providers use **transformed mode** — the gateway rewrites the request body and response to match the OpenAI chat completions schema, regardless of what the upstream expects. This means you can switch providers without changing your application code. **Passthrough mode** (`endpoint: "passthrough"`) forwards the raw request body directly to the provider without transformation. Use this for providers with non-standard APIs (image generation, 3D model generation, audio) where the OpenAI schema doesn't apply. ## Metadata Channels Request context (user identity, trace IDs, metadata for policy conditions) can be provided through three channels, resolved in priority order: 1. **Individual headers** — `X-Igris-Metadata-: ` (highest priority). One header per metadata field. Example: `X-Igris-Metadata-role: developer`. 2. **JSON blob header** — `x-igris-metadata: {"user":"alice","role":"developer"}` plus special sentinels `_user` and `_trace_id` for the core identity fields. 3. **Request body** — `body.user` (OpenAI convention). Lowest priority, for compatibility with clients that already set the OpenAI `user` field. The `connectLlm()` method handles encoding all three channels automatically when you pass `user`, `traceId`, and `metadata` options. ## What's next? All 60 supported providers with slugs, base URLs, and auth styles. Create and manage encrypted credential vaults for upstream providers.