Production Guide
Runtime target
The gateway's nitro.config.ts sets preset: 'bun' — it's built and run as a Bun server, not deployed through a different Nitro preset (Node, serverless, edge) without re-evaluating whether the bun-specific bits (native Bun.randomUUIDv7(), drizzle-orm/bun-sql) still apply. The dashboard is a standard Nuxt 4 app and can follow Nuxt's own deployment guidance for whichever target you choose.
Before going live
Peppergenerated once, backed upAPI_KEY_PEPPER must be stable across every gateway instance and every restart — rotating it invalidates every issued key. Store it in a secrets manager, not a .env file on a single box. See Security. Model registryreviewedmodels.config.ts is a Phase 1 starter, not a fixed catalog — confirm the models, fallback chains, and pricing match what you actually want to expose before shipping. See Model Registry & Routing. Provider credentialsscoped and rotatedDatabasemigrated, backed upbun run db:migrate in each app) and a backup policy — the gateway's holds every application's policy and API key hashes; losing it without a backup means re-provisioning every application. Management keyissued and restrictedRunning multiple gateway instances
The gateway is designed to run as more than one process: the circuit breaker and latency tracker sync their in-memory state through Postgres every 5 seconds specifically so multiple instances converge on the same view of provider health without a shared cache like Redis. Put a load balancer in front of as many gateway instances as you need; no instance holds state that isn't either in Postgres or reconstructible from it.
Graceful shutdown
plugins/shutdown.ts closes the database connection pool on Nitro's close lifecycle hook, relying on the bun preset to drain in-flight requests and SSE streams first. Send whatever signal your process manager uses for graceful shutdown (not SIGKILL) and give it a few seconds before forcing termination, especially if long-running streaming requests are common.
Observability
Every request — success or failure — produces a request_logs row queryable through the management API or the dashboard. The gateway also uses evlog for structured process logs (plugins/evlog.ts), initialized directly rather than through evlog's own Nitro module integration, specifically to avoid overriding the gateway's OpenAI-compatible error envelope — see Errors.
Rotating the pepper (disaster scenario only)
Rotating API_KEY_PEPPER is destructive by design — see Security. Only do it if you believe the pepper itself has been compromised. It invalidates every issued application and management key simultaneously; plan for re-issuing keys to every application before rotating, not after.