fix(llm): size unknown models and refuse a section that cannot be served

Three defects surfaced while driving the Models page.

A hand-declared model needed an explicit contextWindow and maxTokens,
but a provider listing usually returns ids and nothing else — so the
page happily wrote a profile the adapter then rejected, which took the
whole namespace down silently. Capacities now fall back to the route's
`defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768). Both
are guesses by construction, which is why they are route fields a
deployment corrects once rather than constants buried in the adapter;
the fallback sizes the model and never becomes a per-request cap.

That silent failure was the second defect. A schema-valid profile the
adapter could not serve was stored and only rejected later, disabling
every route in the namespace with nothing said. `dsh-settings` gains an
optional `validate` on registration — a check for what a schema cannot
express — and `llm-pi-ai` refuses an unserviceable section at the write
that produced it. A stored section that fails keeps the namespace's last
good value, as a schema failure already did, so an externally edited
document still cannot strand the owner. The plugin's own last-good
fallback goes with it: nothing reaching it can fail any more.

Third, a model with no reasoning metadata advertised the single level
`off`, which pi-ai translates to *omitting* the reasoning option — the
same request naming no effort produces. Selecting it disabled nothing,
so a provider whose default is to think kept thinking with `off` shown
as selected. Such a model now reports no reasoning capability at all,
which is the seam's way of saying the control is unavailable, and the
per-model `reasoning` flag is gone: without a thinkingLevelMap to spell
levels it could only invent them.

The protocol table narrows to the three a hand-declared route reaches
today, most-reached first so a surface offering a choice defaults to the
one gateways actually speak.
This commit is contained in:
Yichen Jiang
2026-08-04 13:32:56 +08:00
parent 4c80cab108
commit f376ee23d1
22 changed files with 368 additions and 108 deletions
+5 -6
View File
@@ -335,12 +335,11 @@ describe('provider profile lifecycle', () => {
ReasoningEffortId('xhigh'),
ReasoningEffortId('max'),
])
await expect(ctx.llm.resolveModelInfo('openai', 'gpt-4.1'))
.resolves.toMatchObject({
reasoning: {
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
},
})
// A catalog model without reasoning is the same case as a hand-declared
// one: pi-ai reports the single level `off`, which translates to omitting
// the reasoning option — exactly what naming no effort already does. The
// capability is reported unavailable rather than offering that control.
expect((await ctx.llm.resolveModelInfo('openai', 'gpt-4.1')).reasoning).toBeUndefined()
})
it('uses a supported profile reasoning value as the model default and rejects an unsupported one', async () => {
+55 -4
View File
@@ -99,6 +99,26 @@ describe('hand-declared providers', () => {
})
})
it('offers no reasoning control it could not honour', async () => {
const server = await mockServer([])
const ctx = await harness(gateway(`${server.url}/v1`))
// pi-ai reports a model with no reasoning metadata as supporting the single
// level `off`, but `off` is translated to *omitting* the reasoning option —
// byte-for-byte the same request as naming no effort — so a provider whose
// own default is to think would keep thinking with `off` selected. The
// capability is reported unavailable instead of offering that control.
expect((await ctx.llm.resolveModelInfo('acme-gateway', 'acme-large')).reasoning).toBeUndefined()
// A catalog route is unaffected: its models carry the metadata that makes
// `off` actually disable thinking.
const withCatalog = await harness({ providers: { deepseek: { apiKey: 'k', baseURL: server.url } } })
const [catalogModel] = getBuiltinModels('deepseek')
if (catalogModel === undefined) throw new Error('the installed catalog ships no deepseek model')
expect((await withCatalog.llm.resolveModelInfo('deepseek', catalogModel.id)).reasoning?.efforts.map(e => e.id))
.toContain('off')
})
it('joins the configurable-provider directory so a settings surface can reach it', async () => {
const server = await mockServer([])
const ctx = await harness(gateway(`${server.url}/v1`))
@@ -111,13 +131,44 @@ describe('hand-declared providers', () => {
})
})
it('rejects a model whose capacity the catalog cannot supply', () => {
it('sizes a model the catalog cannot describe from the route\u2019s own fallbacks', () => {
const resolved = resolveProfiles({
'acme-gateway': {
api: 'openai-completions',
baseURL: 'https://acme.test',
// A listing endpoint that discloses nothing but ids still yields a
// serviceable route.
models: [{ id: 'bare' }, { id: 'sized', contextWindow: 8192, maxTokens: 512 }],
},
'tuned-gateway': {
api: 'openai-completions',
baseURL: 'https://tuned.test',
defaultContextWindow: 4096,
defaultMaxTokens: 256,
models: [{ id: 'bare' }],
},
})
const modelsOf = (route: string): readonly { id: string; contextWindow: number; maxTokens: number }[] =>
resolved.get(route)?.piProvider.getModels() ?? []
expect(modelsOf('acme-gateway')).toMatchObject([
{ id: 'bare', contextWindow: 262_144, maxTokens: 32_768 },
{ id: 'sized', contextWindow: 8192, maxTokens: 512 },
])
// The fallback is a guess, so a deployment whose gateway serves smaller
// models corrects it once for the whole route.
expect(modelsOf('tuned-gateway')).toMatchObject([{ id: 'bare', contextWindow: 4096, maxTokens: 256 }])
// Only an explicitly configured cap is a request default; a fallback is
// the model's capability and stops there.
expect(resolved.get('acme-gateway')?.configuredMaxTokens.get('bare')).toBeUndefined()
expect(resolved.get('acme-gateway')?.configuredMaxTokens.get('sized')).toBe(512)
})
it('rejects a model the route cannot identify', () => {
const declare = (model: LlmPiAi.PiAiModelProfile): (() => unknown) =>
() => resolveProfiles({ 'acme-gateway': { api: 'openai-completions', baseURL: 'https://acme.test', models: [model] } })
expect(declare({ id: 'acme-large', maxTokens: 1 })).toThrow(/needs a contextWindow/)
expect(declare({ id: 'acme-large', contextWindow: 1 })).toThrow(/needs a maxTokens/)
expect(declare({ id: '', contextWindow: 1, maxTokens: 1 })).toThrow(/empty id/)
expect(declare({ id: '' })).toThrow(/empty id/)
expect(() => resolveProfiles({
'acme-gateway': {
api: 'openai-completions',
@@ -146,13 +146,16 @@ describe('request-level dynamic profiles', () => {
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
})
it('keeps the last good profiles when a settings snapshot names an unknown provider', async () => {
it('refuses a settings write this adapter could not serve, leaving its routes alone', async () => {
const dir = await home()
const ctx = await boot(dir, { providers: { openai: {} } })
// Schema-valid but catalog-invalid: the resolver rejects it and the
// last good route set keeps serving.
await ctx.settings.update(NS, { providers: { 'not-a-real-provider': {} } })
// Shape-valid but unserviceable: a route the catalog does not ship and
// that lists no models of its own. The section schema resolves the whole
// profile set, so this is refused where it is written rather than stored
// and then quietly disabling every route in the namespace.
await expect(ctx.settings.update(NS, { providers: { 'not-a-real-provider': {} } }))
.rejects.toThrow(/resolves no models/)
expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai'])
})