Providers

Authorization: Bearer lattice_sk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Every provider implements the same interface (server/providers/types.ts), so the dispatch loop never needs to know which provider it's calling:

type ProviderAdapter = {
  id: ProviderId;
  createChatCompletion(
    request: ChatCompletionRequest,
    ctx: ProviderRequestContext,
  ): Promise<ChatCompletionResponse>;
  streamChatCompletion(
    request: ChatCompletionRequest,
    ctx: ProviderRequestContext,
  ): AsyncGenerator<ChatCompletionChunk, void, void>;
};

type ProviderRequestContext = {
  apiKey?: string;
  providerModel: string;
  signal: AbortSignal;
};

Two shared helpers keep every adapter's error handling consistent:

• requireApiKey(ctx, provider) — throws a 500 provider_error if a cloud adapter is invoked with no apiKey. This should be unreachable in practice, since registry validation already refuses to boot with a model pointing at an unconfigured provider.
• assertProviderOk(response, provider, label) — throws a ProviderRequestError(provider, status, message) on any non-2xx upstream response, which is what the circuit breaker's retry logic inspects.

Cloud vs. self-hosted#

A self-hosted provider counts as configured the moment it has a baseUrl — Ollama and vLLM both already speak an OpenAI-shaped /v1/chat/completions, and typically run with no auth in front of them at all, so requiring a key would just be friction with no security benefit in the common case.

Streaming transport#

Every streaming adapter parses Server-Sent Events through one of two shared parsers in server/providers/sse.ts:

• parseDataOnlySseStream — OpenAI- and Gemini-shaped data: {json} lines, terminated by a literal data: [DONE].
• parseNamedEventSseStream — Anthropic-shaped event: <name> / data: {json} blocks.

Pages in this section#

• OpenAI, Anthropic & Google — the three bespoke cloud adapters and what's distinctive about each one's request/response mapping.
• Ollama & vLLM — the shared openai-compatible adapter and how to point it at a self-hosted endpoint.
• Adding a Provider — implementing ProviderAdapter for a provider not shipped today.