docs: structure model experience fields
This commit is contained in:
@@ -52,19 +52,31 @@ Unit suites run against a local `node:http` mock SSE server (no network). Real-A
|
||||
|
||||
### DeepSeek request
|
||||
|
||||
**What the model sees**: The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. On a prior assistant turn with tool calls, its reasoning content is passed back as required; reasoning from tool-call-free turns is omitted.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available.
|
||||
The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. On a prior assistant turn with tool calls, its reasoning content is passed back as required; reasoning from tool-call-free turns is omitted.
|
||||
|
||||
**KV Cache effect**: An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips.
|
||||
#### Token effect
|
||||
|
||||
Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips.
|
||||
|
||||
### DeepSeek response
|
||||
|
||||
**What the model sees**: Reasoning, text, and raw-string tool arguments are translated into harness chunks for the loop to log and assemble.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input.
|
||||
Reasoning, text, and raw-string tool arguments are translated into harness chunks for the loop to log and assemble.
|
||||
|
||||
**KV Cache effect**: Loop-retained response blocks append to the next request and preserve its earlier reusable prefix; dropped blocks have no later cache effect. Changing the provider or model selects a different cache domain.
|
||||
#### Token effect
|
||||
|
||||
Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Loop-retained response blocks append to the next request and preserve its earlier reusable prefix; dropped blocks have no later cache effect. Changing the provider or model selects a different cache domain.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
@@ -63,19 +63,31 @@ Unit tests use pi-ai catalog models redirected to local mock servers and cover p
|
||||
|
||||
### Provider request through pi-ai
|
||||
|
||||
**What the model sees**: The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. This package adds no prompt prose. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Provider tokenization governs exact input. Conversion adds no model-visible text; replay metadata may let a native API reuse provider-side state.
|
||||
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. This package adds no prompt prose. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
|
||||
**KV Cache effect**: Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference.
|
||||
#### Token effect
|
||||
|
||||
Provider tokenization governs exact input. Conversion adds no model-visible text; replay metadata may let a native API reuse provider-side state.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference.
|
||||
|
||||
### Provider response
|
||||
|
||||
**What the model sees**: pi-ai events become harness reasoning, text, tool-call, usage, and finish chunks. Parsed tool arguments cross the harness boundary as raw JSON strings.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Generated content affects later inputs only after the loop records it. pi-ai folds reasoning tokens into output usage when the provider does not report them separately.
|
||||
pi-ai events become harness reasoning, text, tool-call, usage, and finish chunks. Parsed tool arguments cross the harness boundary as raw JSON strings.
|
||||
|
||||
**KV Cache effect**: Recorded response content appends to the next request and does not invalidate its earlier reusable prefix. Unrecorded transport metadata and usage accounting do not affect cache identity.
|
||||
#### Token effect
|
||||
|
||||
Generated content affects later inputs only after the loop records it. pi-ai folds reasoning tokens into output usage when the provider does not report them separately.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Recorded response content appends to the next request and does not invalidate its earlier reusable prefix. Unrecorded transport metadata and usage accounting do not affect cache identity.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
@@ -58,7 +58,9 @@ Two adapters implement `LlmAdapter` on different internals: [`@deepseek-ai/dsh-l
|
||||
|
||||
None, as this adapter registry forwards an already assembled request without adding or changing any model-bound text, schema, or message.
|
||||
|
||||
**KV Cache effect**: Pass-through; the registry preserves the assembled request prefix, while the selected adapter and provider own cache reuse and routing boundaries.
|
||||
#### KV Cache effect
|
||||
|
||||
Pass-through; the registry preserves the assembled request prefix, while the selected adapter and provider own cache reuse and routing boundaries.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
@@ -42,7 +42,9 @@ Both plugins have usable defaults. A deployment with a different capacity config
|
||||
|
||||
Indirectly, through consumers such as `dsh-compact-basic`; the service itself adds no prompt, message, schema, tool, or model call.
|
||||
|
||||
**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
#### KV Cache effect
|
||||
|
||||
No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
Reference in New Issue
Block a user