fix(llm): capture an immutable snapshot per pi-ai operation
Review found four defects in the declared-provider work. `PiAiAdapter` reused one `Models` collection and mutated it whenever the configuration changed. `Models.streamSimple()` resolves its provider lazily — when the stream is first consumed, which is after the adapter awaits the route's credential — so a configuration change landing in that window let an in-flight request finish under a configuration it never resolved against, or fail on a provider that no longer existed. Each resolution now produces an immutable snapshot and every operation captures one before its first await, which is what makes the seam's per-step freeze (`llm.prepareCall()`) hold end to end: switching models mid-reply takes effect on the next step, never inside the one in flight. `defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`. The two answer different questions: pi-ai requires that field as the model's output capability, while the seam's is a cap the deployment chose to send on requests naming none, so every request had started carrying a number nobody picked. Only an explicitly configured cap reaches the seam now. The configurable-provider directory was refreshed by disposing its registration and making a new one. A candidate set the registry refuses — a profile keyed `deepseek-official`, which llm-deepseek declares — left the whole directory withdrawn and the Models page empty, silently, because the settings callback contains the failure. The seam's registration handle now carries `replace()` with the same validate-first atomicity `registerAdapter` has. The protocol table offered every pi-ai streaming API, including four whose authentication a profile cannot express: Bedrock signs with SigV4 over AWS credentials and a region, Vertex needs a project, a location, and ADC, Azure needs provider environment plus an api-version, and Codex uses OAuth. Offering them handed back routes that cannot authenticate. Catalog routes still reach them through their own provider.
This commit is contained in:
@@ -92,6 +92,12 @@ export interface ResolvedPiAiProviderProfile
|
||||
* serving requests.
|
||||
*/
|
||||
piProvider: Provider
|
||||
/**
|
||||
* Per-request output caps this profile explicitly configured, by model id.
|
||||
* The seam materializes one only into a request that names no cap of its
|
||||
* own, so a catalog capability must not appear here.
|
||||
*/
|
||||
configuredMaxTokens: ReadonlyMap<string, number>
|
||||
}
|
||||
|
||||
/** Plugin configuration: the provider routes this instance owns. */
|
||||
@@ -200,7 +206,7 @@ export function resolveProfiles(
|
||||
// always shown route keys, and a catalog route must not silently rename
|
||||
// itself on every configuration surface just because it gained a profile.
|
||||
const displayName = source.displayName ?? provider
|
||||
const models = resolveRouteModels({
|
||||
const catalog = resolveRouteModels({
|
||||
provider,
|
||||
...source.api === undefined ? {} : { api: source.api },
|
||||
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
|
||||
@@ -216,12 +222,13 @@ export function resolveProfiles(
|
||||
retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`),
|
||||
...rest.headers === undefined ? {} : { headers: { ...rest.headers } },
|
||||
...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } },
|
||||
configuredMaxTokens: catalog.configuredMaxTokens,
|
||||
piProvider: buildProvider({
|
||||
provider,
|
||||
displayName,
|
||||
...source.api === undefined ? {} : { api: source.api },
|
||||
...source.baseURL === undefined ? {} : { baseURL: source.baseURL },
|
||||
models,
|
||||
models: catalog.models,
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user