deepseek-harness/.agents/notes/implemented/architecture/2026-08-03-pi-ai-declared-provider-catalog.md

Agent Note: pi-ai routes are declared providers, not catalog lookups

Status: implemented

English | 中文

Problem

dsh-llm-pi-ai treated the pi-ai package's generated catalog as the boundary of what could be configured. A route key had to name an installed provider (resolveProfiles rejected anything else), model listing returned getBuiltinModels(provider) verbatim, and request-time model resolution looked the id up in that same catalog and overrode only baseURL. Three consequences followed, and all three were dead ends rather than gaps: an OpenAI-compatible gateway, a self-hosted server, or a provider newer than the installed catalog could not be configured at all; a model the catalog had not caught up with failed with UNKNOWN_MODEL even against a correct endpoint; and a model's context window and output cap were whatever the pinned pi-ai release said, so a deployment could neither correct a stale value nor supply one for a model pi-ai had never described. Upgrading the package was the only way to move any of it.

The adapter also streamed through streamSimple from @earendil-works/pi-ai/compat, an entry point whose own module documentation declares it a temporary compatibility surface — its catalog reads are @deprecated, and it is deleted when pi-ai finishes its ModelManager migration. The three configuration limits and the deprecated dependency have the same fix, because pi-ai's supported runtime (createModels() / createProvider()) is built around a provider being declared rather than looked up.

Decision

A provider route is a declaration, and the installed catalog is its default. resolveProfiles no longer checks route keys against getBuiltinProviders(). Instead each route resolves to a materialized model list plus the pi-ai Provider that serves it:

• catalog.ts merges the installed catalog under the profile's own entries. A profile's models list replaces the route's catalog (an absent or empty list serves it unchanged), and each entry defaults its unset fields from the installed model of the same id. Only the fields the harness consumes are configurable — id, name, contextWindow, maxTokens; 2026-08-08-pi-ai-per-model-reasoning-declarations later added reasoningEfforts and compat, which is also where the original "reasoning rides the installed entry or is absent" stance was revisited (a bare capability flag stays rejected; a full per-level declaration with wire spellings does not have its problem). Pricing and input modalities remain absent from the surface because nothing reads them: replay.ts zeroes pi-ai's cost metadata and context.ts keeps only text blocks. Materialization spreads the installed entry and overrides the configured fields, rather than enumerating the result: an enumerated rebuild silently drops every Model field this package does not model, which is how headers went missing from an nvidia route once already.
• provider.ts builds the route's Provider. A catalog route that keeps its catalog protocol reuses the installed provider with getModels() replaced; every other route is built by createProvider() over a protocol table whose entries are the same @earendil-works/pi-ai/api/*.lazy factories pi-ai's own provider factories use. That table is narrower than pi-ai's full API set on purpose — it holds only protocols a profile can completely describe with a key, an endpoint, and headers, so Bedrock (SigV4 plus a region), Vertex (project, location, ADC), Azure (provider environment plus an api-version), and Codex (OAuth) are absent rather than offered as routes that cannot authenticate. Catalog routes still reach them through their own provider; only an explicit override is refused.
• adapter.ts turns each resolution into an immutable snapshot — the profiles plus a createModels() collection holding those providers — and every operation captures a whole snapshot before its first await.
• A model's explicitly configured maxTokens becomes the seam's defaultMaxTokens. The value inherited from the installed catalog does not: pi-ai requires Model.maxTokens as the model's output capability, while defaultMaxTokens is a cap the deployment chose to send on requests that name none, and materializing the former as the latter would start capping every request at a number nobody picked.

Snapshots, not a shared collection

Models.streamSimple() resolves its provider lazily, when the returned stream is first consumed — which is after the adapter has awaited the route's credential. A single collection mutated in place would therefore let a request that started under one configuration finish under another, or fail on a provider that no longer exists, even though llm.prepareCall() already froze that step's config and captured its adapter registration. A configuration change builds a new collection and leaves the one in use alone, so the seam's per-step freeze holds all the way down: switching models mid-reply takes effect on the next step, never inside the one in flight.

The directory replaces atomically

The configurable-provider directory follows the profiles, so it changes whenever a declared route appears or leaves. Withdrawing the old registration and making a new one cannot express that: a candidate set the registry refuses — a profile keyed deepseek-official, which llm-deepseek already declares — would leave this plugin's whole directory withdrawn and the Models page empty, silently, because the settings-change callback contains the failure. registerConfigurableProviders therefore returns a handle carrying replace(entries) with the same validate-the-candidate-set-first atomicity registerAdapter has, and the plugin uses it. A refused swap costs a diagnostic; the previous entries keep serving.

Resolution fails loud and names the route and model at fault: a model the catalog does not describe falls back to the route's own defaultContextWindow/defaultMaxTokens, so a listing that discloses nothing but ids still yields a serviceable route; a route the catalog does not ship needs api, baseURL, and a non-empty models list. Because the built Provider is part of the resolution result, a protocol or model error keeps the last good route set serving, exactly as a bad settings snapshot already did.

The configurable-provider directory is now the installed catalog joined with every route the current profiles declare, re-registered when that set changes. Without the join a hand-declared route would have no settings address and no configuration surface could show or edit it.

A capability whose only level does nothing is reported unavailable

pi-ai reports a model with no reasoning metadata as supporting the single level off, and the adapter used to pass that straight through. It reaches the seam as a one-item effort list, which every surface renders as a picker holding one selectable control — and that control is a lie: off becomes an omitted reasoning option at dispatch, byte-for-byte the request that naming no effort already produces. A provider whose own default is to think keeps thinking while the surface shows off selected.

reasoningInfo therefore omits the Service Definition's reasoning field whenever model.reasoning is falsy. The condition is the model's own metadata, not where the model came from, so this covers every hand-declared model whose entry declares no reasoningEfforts (2026-08-08-pi-ai-per-model-reasoning-declarations made declared efforts carry that metadata) and the 251 installed-catalog models pi-ai marks as non-reasoning. Those previously offered the lone off; they now offer nothing, and the surface shows the provider default alone. Models that do carry reasoning metadata are untouched — their level list still crosses the seam unfiltered, off included, because there it selects between real alternatives.

Credentials stay outside pi-ai

pi-ai's Models carries its own credential concept — a CredentialStore keyed by provider id, with envApiKeyAuth resolving credential.key ?? env(VAR). Adopting it would have created a second credential source of truth beside ctx.credentials and, worse, reintroduced the ambient fallback the harness deliberately forbids: a named-but-missing apiKeyEnv must fail with MISSING_CREDENTIAL rather than authenticate with whatever unrelated key the environment holds.

ModelsImpl.applyAuth honours options.apiKey as the request's key, but only through a provider that declares an api-key method: resolveProviderAuth short-circuits to that method when the override is present, and otherwise falls through to the credential store and then to ambient discovery, returning nothing — and so failing the request with Provider is not configured — when the provider has no api-key method at all. The harness therefore resolves the route's key through its own seam, as before, and passes the result as the request's apiKey; the collection is constructed with no credential store.

A route's auth follows from that. A catalog route keeps the installed provider's own auth, which preserves provider-native ambient discovery for a profile naming no credential, and keeps it through an api override too: which environment a provider reads is a property of the provider, not of the wire format its models speak. The exception is a catalog provider with no api-key method — openai-codex authenticates through OAuth alone — where a profile that names a credential also gets the harness method beside the provider's own, because otherwise its configured key would be refused before any request went out. A keyless profile on such a route adds nothing and keeps the honest refusal: this adapter holds no OAuth store to resolve through. A hand-declared route gets a harness-owned ApiKeyAuth that reports configured-but-keyless rather than unconfigured, leaving the requirement to the protocol — which is where it lives: pi-ai's OpenAI-compatible implementation still demands a key or an Authorization header, and says so itself.

Alternatives considered

• Keep createProvider() but skip the Models collection, streaming through provider.streamSimple(model, ctx, {apiKey}). Smallest diff and the credential path is untouched, but createProvider's auth is a required field that this path never invokes — a required-by-signature implementation with no caller. It also leaves refreshModels needing a hand-built RefreshModelsContext, and keeps the adapter off the runtime pi-ai actually supports.

• Reuse the installed provider for catalog routes and createProvider() only for declared ones, with no shared resolution. Zero risk to catalog behavior, but catalog materialization, endpoint override, and per-model configuration would each exist twice, and a catalog route that repoints its protocol would have to jump paths mid-resolution. The chosen split confines the asymmetry to provider construction, where it is forced by pi-ai not exposing a built provider's API implementations.

• Rebuild every route through createProvider(), including catalog ones. Fully symmetric, but a built Provider does not expose its api, so the protocol table would become the ceiling on which providers work — Bedrock loads its Smithy module through a separate entry point and would silently stop working.

• Expose pi-ai's whole Model shape (cost, input modalities, thinkingLevelMap, compat). Maximum configurability, but no current consumer read those fields then, so a configured price or modality would change nothing while reading as supported. The consumer-driven half of this arrived later: 2026-08-08-pi-ai-per-model-reasoning-declarations opened reasoning (as reasoningEfforts, not a raw thinkingLevelMap) and the two reasoning-dispatch compat switches once selectors and dispatch actually consumed them; cost and modalities stay closed for the original reason.

• Keep one mutable Models collection and re-sync it. Fewer allocations, and correct for every operation that resolves synchronously. It is exactly wrong for the one that does not: stream() awaits a credential between capturing its model and dispatching it.

• Simulate an atomic directory swap with dispose-then-register. No seam change, and it works whenever the new set is valid — which is the case that never needed atomicity.

• A runtime dynamic catalog — fetchModels plus ModelsStore, refreshed in the background. Rejected for this change: it makes the model list external mutable state needing cache, invalidation, and an offline path, and the product need is a one-shot discovery action whose result the user adopts into settings.yaml. That action belongs to the configuration surface and is deferred with it; settings.yaml stays the single source of truth for what a route serves.

Consequences

Configuring a provider no longer depends on a pi-ai release. A gateway, a self-hosted server, or a model newer than the pinned catalog is a settings.yaml edit, and a stale context window can be corrected in place. The deprecated /compat import is gone, so pi-ai deleting it is no longer a breaking event. defaultMaxTokens now flows from configuration when a deployment states one, without inventing a cap from catalog metadata.

What it costs: settings.yaml grows for a declared route, because it must state its endpoint, protocol, and model ids. api applies to a whole route, so a mixed-protocol catalog route cannot host a model of the other protocol — splitting it across two route keys is the workaround. Nothing queries a provider's /models, so a model list is only as current as its last edit. Reported error shape shifts in one case: a route whose auth resolves to nothing now surfaces pi-ai's own diagnostic as an error finish chunk before any network call, where the previous adapter sent a keyless request and surfaced the provider's 401.

Testing

tests/catalog.spec.ts covers the contract end to end against local mock servers: a hand-declared route streaming to its own endpoint with its own credential, its appearance in the configurable-provider directory, per-model overrides defaulting from the installed catalog, a model added to a catalog route, protocol repointing with and without an endpoint override, catalog-only metadata surviving an override, the keyless posture and its Authorization-header workaround, an OAuth-only catalog route authenticating with the key its profile names while a keyless one stays unconfigured, a repointed route keeping its catalog auth, and every resolution failure that names a route or model. tests/catalog.spec.ts also pins the snapshot and directory contracts: an in-flight request whose route set changes during its credential await still reaches the endpoint it resolved against, the next request picks up the new one, a colliding declared route leaves the directory whole, and a declared route's entry appears and leaves with its profile. packages/llm/llm/tests/topology.spec.ts covers replace — refusing a candidate another registration owns while keeping the current set, accepting a swap over its own entries, allowing an empty set, and failing after disposal. tests/sdk-options.spec.ts re-targets the SDK boundary from the removed /compat import to the protocol table's lazy api module, which also pins that a setup failure arrives as a terminal error chunk rather than a throw. The twin's design-verification role is unchanged.