​

Architecture

​

System Overview

​

Components

​

API Server (Hono)

• MCP Gateway (/v1/mcp/:slug) — intercept MCP tool calls, evaluate policies, forward to upstream
• LLM Gateway (/llm/:slug/*) — OpenAI-compatible API proxy for 60 LLM providers (see LLM Gateway below)
• REST API (/api/v1/*) — CRUD for policies, servers, sessions, audit events, billing
• Auth (/api/auth/*) — Better Auth endpoints for session management
• SSE (/api/v1/events) — real-time event stream for dashboard updates

​

Web Frontend (Next.js)

• Governance management (servers, policies, sessions)
• Real-time observe dashboard with risk heat maps
• Organization settings, member management, billing

​

Database (Neon PostgreSQL)

​

Cache (Upstash Redis)

• Policy cache — hot policies cached with TTL to avoid DB lookups on every tool call
• Rate limiting — sliding window counters for rate-limit policy rules
• SSE pub/sub — event fan-out to connected dashboard clients
• Session state — fast lookup for kill-switch status

​

Authentication (Better Auth)

• Email/password and OAuth (GitHub, Google) login
• Organization-scoped sessions with RBAC (operational)
• API key generation for programmatic access (proxy)
• Session tokens stored as igris.session_token cookies (or __Secure-igris.session_token in production)

​

LLM Gateway

​

Route shape

/llm/{connection-slug}/v1/chat/completions   → chat.completions (transformed)
/llm/{connection-slug}/v1/embeddings         → embeddings (transformed)
/llm/{connection-slug}/**                    → passthrough to upstream

​

Authentication

​

What the gateway does on each request

1. Authenticates the API key and resolves the connection (slug → provider + encrypted credential)
2. Evaluates policies (allow / deny / alert / redact) against the model and endpoint
3. Checks pre-forward token guards and content guards
4. Checks rate limits (requests / tokens / dollars, Redis sliding window)
5. Forwards the request to the upstream provider with credential injection
6. Streams the response to the client via a single-pass tee (client stream + accumulator)
7. Writes an audit event with token counts, cost, and model after the response completes
8. Updates the LLM anomaly detector asynchronously (cost spike, token burn, error rate, model shift, response length)

​

Provider support

​

Enablement

​

S3 Audit Archive

​

Archive job

1. Finds days in audit_events older than retentionDays + 1 buffer days
2. Serializes each day’s events as gzip NDJSON
3. Writes the file to S3 with a deterministic key ({orgId}/{YYYY-MM}/{YYYY-MM-DD}.ndjson.gz) — idempotent overwrite
4. In a single Postgres transaction: inserts a manifest row into audit_archives, then deletes the archived rows from audit_events

​

Key implications

• audit_events does not grow unboundedly in Postgres
• Restored rows are marked with restored_from_archive_id so they are not re-archived
• The audit trail is complete but not immutable — rows are deleted from Postgres after archiving

​

Proxy Flow

1. Request arrives at /v1/mcp/:slug with the tool name and arguments
2. Auth check — validate API key or session cookie
3. Org resolution — determine which organization owns this connection
4. Policy evaluation — load rules from cache (or DB on cache miss), evaluate first-match against the tool name with conditions
5. Action execution:
   • allow → forward to upstream, log the event
   • deny → return error to client, log the denial
   • alert → forward to upstream, log + emit anomaly event via SSE
6. Anomaly check — evaluate rate spike and destructive pattern detectors
7. Audit write — persist the event to audit_events with timing, result, and metadata
8. SSE broadcast — push real-time event to connected dashboard clients

​

Deployment

• API server runs as a systemd unit on EC2 (Ubuntu)
• Next.js frontend deployed to Vercel (or a separate EC2 instance)
• Neon for managed PostgreSQL (serverless, auto-scaling)
• Upstash for managed Redis (serverless, per-request pricing)
• Migrations applied automatically at API startup via drizzle-orm/postgres-js/migrator (programmatic — not drizzle-kit). The API refuses to start if any migration fails.