fix(llm): capture an immutable snapshot per pi-ai operation

Review found four defects in the declared-provider work.

`PiAiAdapter` reused one `Models` collection and mutated it whenever the
configuration changed. `Models.streamSimple()` resolves its provider
lazily — when the stream is first consumed, which is after the adapter
awaits the route's credential — so a configuration change landing in
that window let an in-flight request finish under a configuration it
never resolved against, or fail on a provider that no longer existed.
Each resolution now produces an immutable snapshot and every operation
captures one before its first await, which is what makes the seam's
per-step freeze (`llm.prepareCall()`) hold end to end: switching models
mid-reply takes effect on the next step, never inside the one in flight.

`defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`.
The two answer different questions: pi-ai requires that field as the
model's output capability, while the seam's is a cap the deployment
chose to send on requests naming none, so every request had started
carrying a number nobody picked. Only an explicitly configured cap
reaches the seam now.

The configurable-provider directory was refreshed by disposing its
registration and making a new one. A candidate set the registry refuses
— a profile keyed `deepseek-official`, which llm-deepseek declares —
left the whole directory withdrawn and the Models page empty, silently,
because the settings callback contains the failure. The seam's
registration handle now carries `replace()` with the same
validate-first atomicity `registerAdapter` has.

The protocol table offered every pi-ai streaming API, including four
whose authentication a profile cannot express: Bedrock signs with SigV4
over AWS credentials and a region, Vertex needs a project, a location,
and ADC, Azure needs provider environment plus an api-version, and Codex
uses OAuth. Offering them handed back routes that cannot authenticate.
Catalog routes still reach them through their own provider.
This commit is contained in:
Yichen Jiang
2026-08-04 11:42:35 +08:00
parent d6126c25f2
commit 4c80cab108
21 changed files with 476 additions and 104 deletions
+13 -11
View File
@@ -22,12 +22,8 @@
import { createProvider } from '@earendil-works/pi-ai'
import type { Api, ApiKeyAuth, Model, Provider, ProviderStreams } from '@earendil-works/pi-ai'
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy'
import { azureOpenAIResponsesApi } from '@earendil-works/pi-ai/api/azure-openai-responses.lazy'
import { bedrockConverseStreamApi } from '@earendil-works/pi-ai/api/bedrock-converse-stream.lazy'
import { googleGenerativeAIApi } from '@earendil-works/pi-ai/api/google-generative-ai.lazy'
import { googleVertexApi } from '@earendil-works/pi-ai/api/google-vertex.lazy'
import { mistralConversationsApi } from '@earendil-works/pi-ai/api/mistral-conversations.lazy'
import { openAICodexResponsesApi } from '@earendil-works/pi-ai/api/openai-codex-responses.lazy'
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy'
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy'
import { piMessagesApi } from '@earendil-works/pi-ai/api/pi-messages.lazy'
@@ -35,18 +31,24 @@ import { catalogProvider } from './catalog.ts'
/**
* Wire protocols a configured route may name, mapped to pi-ai's lazily loaded
* implementations. The table is pi-ai's own streaming API set: each entry is
* the factory that pi-ai's matching provider factory uses, so a hand-declared
* route reaches exactly the implementation a catalog route would.
* implementations. Each entry is the factory that pi-ai's matching provider
* factory uses, so a hand-declared route reaches exactly the implementation a
* catalog route would.
*
* The table is deliberately narrower than pi-ai's full streaming API set: it
* holds only the protocols a profile can *completely* describe with a key, an
* endpoint, and headers. Bedrock signs with SigV4 over AWS credentials and a
* region, Vertex needs a project, a location, and application-default
* credentials, Azure needs provider environment plus an api-version, and
* Codex authenticates through OAuth — none of which this configuration shape
* can express, so offering them would hand back a provider that cannot
* authenticate. Catalog routes still reach those protocols through their own
* provider; only an explicit override is refused.
*/
const PROTOCOLS: Readonly<Record<string, () => ProviderStreams>> = {
'anthropic-messages': anthropicMessagesApi,
'azure-openai-responses': azureOpenAIResponsesApi,
'bedrock-converse-stream': bedrockConverseStreamApi,
'google-generative-ai': googleGenerativeAIApi,
'google-vertex': googleVertexApi,
'mistral-conversations': mistralConversationsApi,
'openai-codex-responses': openAICodexResponsesApi,
'openai-completions': openAICompletionsApi,
'openai-responses': openAIResponsesApi,
'pi-messages': piMessagesApi,