feat(llm): add model-specific reasoning effort controls

This commit is contained in:
Yichen Jiang
2026-07-25 07:47:51 +08:00
parent 65d29da8a1
commit 8372340f9c
50 changed files with 1046 additions and 88 deletions
+7 -3
View File
@@ -12,6 +12,8 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
- `ctx.llm.resolveModelReasoning(provider: string, model: string): Promise<LlmModelReasoningInfo | undefined>` Resolve ordered adapter-owned reasoning efforts and an optional deployment default for one exact route.
- `ctx.llm.resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
@@ -20,6 +22,8 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
Reasoning effort is also an exact-route capability, but its identifiers are opaque adapter-owned strings rather than a core enum. `resolveModelReasoning()` validates and detaches the ordered display metadata; `undefined` means the model has no selectable effort. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Invalid capability metadata fails with `INVALID_MODEL_REASONING`; an unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
### Events
| Event | Mode | Purpose |
@@ -28,7 +32,7 @@ Context capacity is a separate correctness query, not a catalog decoration or gl
### Extension points
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, and `resolveModelContext()` when exact capacity is known; the defaults use the route id as its name, advertise no models, and return no capacity.
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, `resolveModelContext()` when exact capacity is known, and `resolveModelReasoning()` when a model exposes selectable efforts; the defaults use the route id as its name, advertise no models, and return neither capacity nor reasoning metadata.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)
@@ -39,7 +43,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
### Call configuration (`call-config.ts`)
`LlmCallConfig` is the provider + model + sampling scalars of one conversation's requests (`provider`, `model`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement and the loop logs a real change. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
`LlmCallConfig` is the provider, model, optional adapter-owned reasoning effort, and sampling scalars of one conversation's requests (`provider`, `model`, `reasoningEffort`, `temperature`, `maxTokens`, `stop` — each mapping 1:1 onto the same-named `GenerateOptions` field). It is per-conversation state recorded in the session log as part of the request header (see the dsh-session `request/header` events), never a silently-adjustable per-call knob: the `agent/request` waterfall proposes a replacement, `resolveCallConfig()` validates and defaults it, and the loop logs the effective value before dispatch. `callConfigEquals(a, b)` is the field-wise real-change detector; `deepFreeze(value)` is the ownership helper the loop applies to every built request before dispatch (`llm/stream` listeners and adapters read, never rewrite). `markAgentLoopRequest()` gives that exact object process-local loop provenance, and `isAgentLoopRequest()` lets observers distinguish it from independently logged auxiliary calls that may also be frozen and session-associated. `GenerateOptions.purpose` classifies logged auxiliary compaction and session-title calls so adapters can apply purpose-specific transport policy without changing ordinary conversation requests.
### App attribution (`attribution.ts`)
@@ -61,7 +65,7 @@ Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-l
## Model Experience
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
None, as the service adds no model-bound text, schema, or message; it only materializes and logs an adapter-configured reasoning effort.
#### KV Cache effect
+12
View File
@@ -38,3 +38,15 @@ export type ProviderRequestId = Branded<'ProviderRequestId'>
export function ProviderRequestId(id: string): ProviderRequestId {
return id as ProviderRequestId
}
/** Adapter-owned identifier for one model's selectable reasoning effort. */
export type ReasoningEffortId = Branded<'ReasoningEffortId'>
/**
* Brand an adapter-owned reasoning-effort identifier.
* @param id - the opaque identifier exposed by one model capability.
* @returns the same string, branded; no validation is performed.
*/
export function ReasoningEffortId(id: string): ReasoningEffortId {
return id as ReasoningEffortId
}
+16 -7
View File
@@ -1,24 +1,27 @@
/**
* Conversation call configuration and freeze utilities. Provider routing,
* model, and sampling values are request-header state that can affect cache
* reuse; request waterfalls replace them and the loop logs changed snapshots
* instead of allowing silent per-call drift.
* model, reasoning effort, and sampling values are request-header state that
* can affect cache reuse; request waterfalls replace them and the loop logs
* changed snapshots instead of allowing silent per-call drift.
* @module dsh-llm/call-config
*/
import type { GenerateOptions } from './types.ts'
import type { ReasoningEffortId } from './brand.ts'
/** Process-local identities of request objects assembled by dsh-agent-loop. */
const AGENT_LOOP_REQUESTS = new WeakSet<GenerateOptions>()
/**
* Provider + model + sampling scalars of one conversation's requests. Every field maps
* 1:1 onto the same-named `GenerateOptions` field; the loop builds requests
* from the logged header rather than accepting these per call.
* Provider, model, reasoning effort, and sampling scalars of one conversation's
* requests. Every field maps 1:1 onto the same-named `GenerateOptions` field;
* the loop builds requests from the logged header rather than accepting these
* per call.
*/
export interface LlmCallConfig {
provider: string
model: string
reasoningEffort?: ReasoningEffortId
temperature?: number
maxTokens?: number
stop?: string[]
@@ -33,7 +36,13 @@ export interface LlmCallConfig {
* @returns whether every field (including the `stop` list, element-wise) matches.
*/
export function callConfigEquals(a: LlmCallConfig, b: LlmCallConfig): boolean {
if (a.provider !== b.provider || a.model !== b.model || a.temperature !== b.temperature || a.maxTokens !== b.maxTokens) return false
if (
a.provider !== b.provider
|| a.model !== b.model
|| a.reasoningEffort !== b.reasoningEffort
|| a.temperature !== b.temperature
|| a.maxTokens !== b.maxTokens
) return false
if (a.stop === undefined || b.stop === undefined) return a.stop === b.stop
return a.stop.length === b.stop.length && a.stop.every((s, i) => s === b.stop?.[i])
}
+109 -3
View File
@@ -12,12 +12,14 @@ import type {
LlmFailure,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
Message,
StreamChunk,
} from './types.ts'
import type { ProviderRequestId } from './brand.ts'
import { deepFreeze } from './call-config.ts'
import { callConfigEquals, deepFreeze } from './call-config.ts'
import type { LlmCallConfig } from './call-config.ts'
import { HarnessError } from './error.ts'
import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts'
import type { AdapterFailureScope } from './adapter-failure.ts'
@@ -144,6 +146,20 @@ export abstract class LlmAdapter {
return Promise.resolve(undefined)
}
/**
* Resolve selectable reasoning efforts for one exact model. Absence means
* the model has no selectable reasoning-effort capability.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
*/
resolveModelReasoning(
_provider: string,
_model: string,
): Promise<LlmModelReasoningInfo | undefined> {
return Promise.resolve(undefined)
}
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
@@ -262,6 +278,90 @@ export class LlmService extends Service {
return { contextWindow: context.contextWindow }
}
/**
* Resolve selectable reasoning efforts from the adapter that owns one exact
* route. Metadata is validated and detached; an absent result means an
* effort selector is unsupported for that model.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @returns detached reasoning metadata, or `undefined` when unsupported.
*/
async resolveModelReasoning(
provider: string,
model: string,
): Promise<LlmModelReasoningInfo | undefined> {
const reasoning = await this.registration(provider).adapter.resolveModelReasoning(provider, model)
if (reasoning === undefined) return undefined
if (reasoning.efforts.length === 0) {
throw new LlmError(
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
const seen = new Set<string>()
const efforts = reasoning.efforts.map((effort) => {
if (
typeof effort.id !== 'string'
|| effort.id.length === 0
|| typeof effort.name !== 'string'
|| effort.name.length === 0
|| (effort.description !== undefined && typeof effort.description !== 'string')
|| seen.has(effort.id)
) {
throw new LlmError(
`adapter returned invalid or duplicate reasoning effort metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
seen.add(effort.id)
return {
id: effort.id,
name: effort.name,
...effort.description === undefined ? {} : { description: effort.description },
}
})
if (reasoning.defaultEffort !== undefined && !seen.has(reasoning.defaultEffort)) {
throw new LlmError(
`adapter returned an unknown default reasoning effort for provider "${provider}" model "${model}"`,
'INVALID_MODEL_REASONING',
)
}
return {
efforts,
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
}
}
/**
* Validate a conversation call config against its exact model capability and
* materialize an adapter-configured default. Unsupported explicit efforts
* reject before provider I/O; no clamping or aliasing is performed.
* @param config - provider/model route and optional request controls.
* @returns a detached config only when a default must be materialized.
*/
async resolveCallConfig(config: LlmCallConfig): Promise<LlmCallConfig> {
const reasoning = await this.resolveModelReasoning(config.provider, config.model)
const requested = config.reasoningEffort
if (reasoning === undefined) {
if (requested !== undefined) {
throw new LlmError(
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${requested}"`,
'UNSUPPORTED_REASONING_EFFORT',
)
}
return config
}
const effective = requested ?? reasoning.defaultEffort
if (effective === undefined) return config
if (!reasoning.efforts.some(effort => effort.id === effective)) {
throw new LlmError(
`provider "${config.provider}" model "${config.model}" does not support reasoning effort "${effective}"`,
'UNSUPPORTED_REASONING_EFFORT',
)
}
return requested === effective ? config : { ...config, reasoningEffort: effective }
}
private registration(provider: string): { adapter: LlmAdapter; provider: LlmProviderInfo } {
const registration = this.adapters.get(provider)
if (!registration) throw new LlmError(`no adapter registered for provider "${provider}"`, 'NO_ADAPTER')
@@ -298,8 +398,14 @@ export class LlmService extends Service {
): AsyncGenerator<StreamChunk> {
let iterator: AsyncIterator<StreamChunk>
try {
const adapter = this.registration(options.provider).adapter
const stream = adapter.stream(this.forAdapter(options, adapter))
const resolvedConfig = await this.resolveCallConfig(options)
const resolvedOptions = callConfigEquals(options, resolvedConfig)
? options
: Object.isFrozen(options)
? deepFreeze({ ...options, ...resolvedConfig })
: { ...options, ...resolvedConfig }
const adapter = this.registration(resolvedOptions.provider).adapter
const stream = adapter.stream(this.forAdapter(resolvedOptions, adapter))
iterator = stream[Symbol.asyncIterator]()
} catch (error: unknown) {
throw markLlmAdapterFailure(failures, error)
+24 -1
View File
@@ -5,7 +5,7 @@
*/
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId, ProviderRequestId } from './brand.ts'
import type { CallId, ProviderRequestId, ReasoningEffortId } from './brand.ts'
/** Serializable provider-boundary facts; policy decides whether they are retryable. */
export interface LlmFailure {
@@ -161,6 +161,27 @@ export interface LlmModelContext {
contextWindow: number
}
/** Display metadata for one adapter-owned reasoning effort. */
export interface LlmReasoningEffortInfo {
/** Opaque stable value accepted by {@link GenerateOptions.reasoningEffort}. */
id: ReasoningEffortId
/** Human-readable effort name for selectors and diagnostics. */
name: string
/** Optional user-facing distinction from otherwise similar efforts. */
description?: string
}
/** Selectable reasoning efforts for one exact provider/model route. */
export interface LlmModelReasoningInfo {
/** Supported efforts in adapter-preferred display order. */
efforts: readonly LlmReasoningEffortInfo[]
/**
* Adapter-configured default materialized into requests when callers omit
* an effort. Absence preserves the provider's own default.
*/
defaultEffort?: ReasoningEffortId
}
/**
* Raw streaming protocol emitted by adapters.
* Block indexes correlate interleaved deltas, and `block-end` carries the
@@ -201,6 +222,8 @@ export interface GenerateOptions {
/** Registered provider route selecting the adapter instance. */
provider: string
model: string
/** Adapter-owned reasoning effort selected for this exact model. */
reasoningEffort?: ReasoningEffortId
/**
* Ordered conversation messages, exactly as the provider sees them (after
* the `system` slot). A loop-built request assembles them as
@@ -6,6 +6,7 @@
import { describe, expect, it } from 'vitest'
import { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from '../src/call-config.ts'
import { ReasoningEffortId } from '../src/brand.ts'
import type { GenerateOptions } from '../src/types.ts'
describe('callConfigEquals', () => {
@@ -14,6 +15,11 @@ describe('callConfigEquals', () => {
expect(callConfigEquals(base, base)).toBe(true)
expect(callConfigEquals(base, { provider: 'x', model: 'm' })).toBe(false)
expect(callConfigEquals(base, { provider: 'p', model: 'x' })).toBe(false)
expect(callConfigEquals({ ...base, reasoningEffort: ReasoningEffortId('high') }, base)).toBe(false)
expect(callConfigEquals(
{ ...base, reasoningEffort: ReasoningEffortId('high') },
{ ...base, reasoningEffort: ReasoningEffortId('high') },
)).toBe(true)
expect(callConfigEquals({ ...base, temperature: 0.5 }, base)).toBe(false)
expect(callConfigEquals({ ...base, maxTokens: 1 }, { ...base, maxTokens: 2 })).toBe(false)
expect(callConfigEquals({ ...base, stop: ['a'] }, base)).toBe(false)
+126 -1
View File
@@ -11,9 +11,15 @@ import LlmService, {
LlmError,
llmFailureOf,
ProviderRequestId,
ReasoningEffortId,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import type { LlmModelContext, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm'
import type {
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
} from '@deepseek-ai/dsh-llm'
class ScriptedAdapter extends LlmAdapter {
constructor(private script: StreamChunk[]) {
@@ -49,6 +55,7 @@ class CatalogAdapter extends ScriptedAdapter {
private readonly provider: LlmProviderInfo,
private readonly models: readonly LlmModelInfo[],
private readonly contexts: Readonly<Record<string, LlmModelContext>> = {},
private readonly reasoning: Readonly<Record<string, LlmModelReasoningInfo>> = {},
) {
super(SCRIPT)
}
@@ -67,6 +74,13 @@ class CatalogAdapter extends ScriptedAdapter {
): Promise<LlmModelContext | undefined> {
return Promise.resolve(this.contexts[model])
}
override resolveModelReasoning(
_provider: string,
model: string,
): Promise<LlmModelReasoningInfo | undefined> {
return Promise.resolve(this.reasoning[model])
}
}
const SCRIPT: StreamChunk[] = [
@@ -674,6 +688,117 @@ describe('LlmService', () => {
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
})
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const source = {
efforts: [
{ id: ReasoningEffortId('standard'), name: 'Standard' },
{ id: ReasoningEffortId('ultra'), name: 'Ultra', description: 'Largest budget' },
],
defaultEffort: ReasoningEffortId('standard'),
}
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: source },
))
const resolved = await ctx.llm.resolveModelReasoning('route', 'model')
expect(resolved).toEqual(source)
source.efforts[0]!.name = 'mutated'
expect(resolved?.efforts[0]?.name).toBe('Standard')
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
provider: 'route',
model: 'model',
reasoningEffort: ReasoningEffortId('standard'),
})
const explicit = { provider: 'route', model: 'model', reasoningEffort: ReasoningEffortId('ultra') }
await expect(ctx.llm.resolveCallConfig(explicit)).resolves.toBe(explicit)
})
it.each([
[{ efforts: [] }, 'empty effort list'],
[{ efforts: [{ id: '', name: 'Empty' }] }, 'empty id'],
[{ efforts: [{ id: 'valid', name: '' }] }, 'empty name'],
[{ efforts: [{ id: 'valid', name: 'Valid', description: 1 }] }, 'non-string description'],
[{ efforts: [{ id: 'same', name: 'One' }, { id: 'same', name: 'Two' }] }, 'duplicate id'],
[{ efforts: [{ id: 'valid', name: 'Valid' }], defaultEffort: 'other' }, 'unknown default'],
] as const)('rejects invalid model reasoning metadata (%s: %s)', async (metadata, _label) => {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: metadata as unknown as LlmModelReasoningInfo },
))
await expect(ctx.llm.resolveModelReasoning('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
})
it('rejects unsupported reasoning efforts without clamping', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
ctx.llm.registerAdapter(['route'], new CatalogAdapter(
{ id: 'route', name: 'Route' },
[],
{},
{ model: { efforts: [{ id: ReasoningEffortId('ultra'), name: 'Ultra' }] } },
))
await expect(ctx.llm.resolveCallConfig({
provider: 'route',
model: 'model',
reasoningEffort: ReasoningEffortId('standard'),
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
await expect(ctx.llm.resolveCallConfig({
provider: 'route',
model: 'plain',
reasoningEffort: ReasoningEffortId('standard'),
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
})
it('resolves reasoning defaults at the final adapter boundary after routing middleware', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new class extends RecordingAdapter {
override resolveModelReasoning(
_provider: string,
_model: string,
): Promise<LlmModelReasoningInfo> {
return Promise.resolve({
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
defaultEffort: ReasoningEffortId('standard'),
})
}
}(SCRIPT)
ctx.llm.registerAdapter(['routed'], adapter)
const disposeRouting = ctx.on('llm/stream', (options, next) => {
options.provider = 'routed'
return next()
})
for await (const _chunk of ctx.llm.stream({
provider: 'initial',
model: 'model',
messages: [],
})) { /* drain */ }
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
disposeRouting()
const frozenRequest: GenerateOptions = Object.freeze({
provider: 'routed',
model: 'model',
messages: [],
})
for await (const _chunk of ctx.llm.stream(frozenRequest)) { /* drain */ }
expect(adapter.lastOptions?.reasoningEffort).toBe(ReasoningEffortId('standard'))
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
})
it.each([0, -1, 1.5, Number.NaN])(
'rejects invalid adapter model context %s',
async (contextWindow) => {