fix(llm): capture an immutable snapshot per pi-ai operation
Review found four defects in the declared-provider work. `PiAiAdapter` reused one `Models` collection and mutated it whenever the configuration changed. `Models.streamSimple()` resolves its provider lazily — when the stream is first consumed, which is after the adapter awaits the route's credential — so a configuration change landing in that window let an in-flight request finish under a configuration it never resolved against, or fail on a provider that no longer existed. Each resolution now produces an immutable snapshot and every operation captures one before its first await, which is what makes the seam's per-step freeze (`llm.prepareCall()`) hold end to end: switching models mid-reply takes effect on the next step, never inside the one in flight. `defaultMaxTokens` was materialized from the catalog's `Model.maxTokens`. The two answer different questions: pi-ai requires that field as the model's output capability, while the seam's is a cap the deployment chose to send on requests naming none, so every request had started carrying a number nobody picked. Only an explicitly configured cap reaches the seam now. The configurable-provider directory was refreshed by disposing its registration and making a new one. A candidate set the registry refuses — a profile keyed `deepseek-official`, which llm-deepseek declares — left the whole directory withdrawn and the Models page empty, silently, because the settings callback contains the failure. The seam's registration handle now carries `replace()` with the same validate-first atomicity `registerAdapter` has. The protocol table offered every pi-ai streaming API, including four whose authentication a profile cannot express: Bedrock signs with SigV4 over AWS credentials and a region, Vertex needs a project, a location, and ADC, Azure needs provider environment plus an api-version, and Codex uses OAuth. Offering them handed back routes that cannot authenticate. Catalog routes still reach them through their own provider.
This commit is contained in:
@@ -44,7 +44,7 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { AdapterRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
|
||||
import type { AdapterRegistrationHandle, DirectoryRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
|
||||
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
|
||||
import { PiAiAdapter } from './adapter.ts'
|
||||
import { catalogProviderIds } from './catalog.ts'
|
||||
@@ -151,13 +151,21 @@ export function apply(ctx: Context, config: Config): void {
|
||||
// mounts — dormant or not — so configuration surfaces can offer every
|
||||
// pi-ai provider before any route exists. Hand-declared routes join it as
|
||||
// profiles appear, and leave with them.
|
||||
let directory: (() => void) | undefined
|
||||
let directory: DirectoryRegistrationHandle | undefined
|
||||
let directoryFacts: unknown
|
||||
const ensureDirectory = (): void => {
|
||||
const entries = directoryEntries(profiles())
|
||||
if (deepEqualJson(entries, directoryFacts)) return
|
||||
directory?.()
|
||||
directory = ctx.llm.registerConfigurableProviders(entries)
|
||||
// Atomic replace, never dispose-then-register: a route another adapter
|
||||
// family already declares (a profile keyed `deepseek-official`) would
|
||||
// otherwise leave this plugin's whole directory withdrawn and the Models
|
||||
// page empty. The candidate set is validated first, so a collision keeps
|
||||
// the previous entries serving and only costs a diagnostic.
|
||||
if (directory === undefined) {
|
||||
directory = ctx.llm.registerConfigurableProviders(entries)
|
||||
} else {
|
||||
directory.replace(entries)
|
||||
}
|
||||
directoryFacts = entries
|
||||
}
|
||||
ensureDirectory()
|
||||
@@ -199,8 +207,16 @@ export function apply(ctx: Context, config: Config): void {
|
||||
onChange: () => {
|
||||
ensureRegistrationFacts()
|
||||
// The directory follows the profiles the registry accepted, so a route
|
||||
// that failed to register is not advertised as configurable.
|
||||
ensureDirectory()
|
||||
// that failed to register is not advertised as configurable. A refused
|
||||
// directory swap is contained here for the same reason the registry's
|
||||
// is: the previous entries keep serving, and `directoryFacts` stays put
|
||||
// so returning to a working configuration re-applies.
|
||||
try {
|
||||
ensureDirectory()
|
||||
} catch (error) {
|
||||
ctx.logger.error('llm-pi-ai: keeping the previous configurable-provider directory after a refused update')
|
||||
ctx.logger.error(error)
|
||||
}
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user