fix(tools): complete the Literal parseability attribution and the soft-keyword positions

pyScalar's docstring named only the two code points CPython refuses
anywhere in source. A bare quote, a trailing odd backslash, and a bare
LF/CR break the Literal line just as fatally, and JSON.stringify is what
covers those too. The argument also leaned on an unstated coincidence:
every escape JSON.stringify can emit is a Python escape for the same
character, which is why the emitted text both parses and decodes back to
the declared value. Say both, and assert the second class.

"statement head" does not describe `case`, whose clause block is not a
statement. Split the positions three ways.

Add the mode 'both' by python assembly, pinning the mode-by-language
matrix rather than leaving it to the shared code path.
This commit is contained in:
Chinesezjc
2026-08-05 19:01:18 +08:00
parent 5afcf36ae1
commit dbefb2fa90
3 changed files with 43 additions and 13 deletions
+18 -9
View File
@@ -29,9 +29,10 @@ const IDENTIFIER = /^[A-Za-z_][A-Za-z0-9_]*$/
* every tool and field without collisions. * every tool and field without collisions.
* Soft keywords (``match``, ``case``, ``type``, ``_`` — the language * Soft keywords (``match``, ``case``, ``type``, ``_`` — the language
* reference's whole set) are deliberately ABSENT: each is special in exactly * reference's whole set) are deliberately ABSENT: each is special in exactly
* one syntactic position — a statement head, or a ``match`` pattern for ``_`` * one syntactic position — a statement head (``match``, ``type``), a ``match``
* — so ``match: str`` as a field and ``async def match(...)`` as a method are * statement's clause head (``case``), or a pattern (``_``) — so ``match: str``
* both legal, and including * as a field and ``async def match(...)`` as a method are both legal, and
* including
* them would needlessly degrade common search/regex tool fields to * them would needlessly degrade common search/regex tool fields to
* ``dict[str, Any]``. Underscore-leading names are handled separately, not * ``dict[str, Any]``. Underscore-leading names are handled separately, not
* here: a non-dunder ``__token`` name-mangles, a dunder present on * here: a non-dunder ``__token`` name-mangles, a dunder present on
@@ -262,13 +263,21 @@ function childClassName(base: string, segment: string): string {
* by a JS parser back into the same double. * by a JS parser back into the same double.
* *
* `JSON.stringify` is also what keeps this path's output parseable, and it is * `JSON.stringify` is also what keeps this path's output parseable, and it is
* the only thing that does: it escapes both code points CPython refuses in * the only thing that does. It covers both classes of hazard: the two code
* source — NUL among the C0 controls, and unpaired surrogates under ES2019 * points CPython refuses anywhere in source — NUL among the C0 controls, and
* well-formed stringification, which the engines range guarantees. The * unpaired surrogates under ES2019 well-formed stringification, which the
* engines range guarantees — and the ones that break this line in particular,
* a bare `"` closing the literal early, a trailing odd backslash eating the
* closing quote, and a bare LF/CR ending it before its terminator. The
* `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE} * `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE}
* because nothing quotes it. DEL and the C1 controls do reach a `Literal[...]` * because nothing quotes it, and folds newlines in {@link describe}.
* raw — legal but invisible, byte-for-byte as in the TS flavor; escaping them *
* is a both-flavors change. * That leans on a coincidence worth naming: every escape `JSON.stringify` can
* emit (`\"`, `\\`, `\b`, `\f`, `\n`, `\r`, `\t`, `\uXXXX`) is also a Python
* escape denoting the same character, so the emitted `Literal[...]` both
* parses and decodes back to the value the schema declared. DEL and the C1
* controls do reach it raw — legal but invisible, byte-for-byte as in the TS
* flavor; escaping them is a both-flavors change.
*/ */
function pyScalar(value: JsonSchemaScalar): string { function pyScalar(value: JsonSchemaScalar): string {
if (value === true) return 'True' if (value === true) return 'True'
@@ -350,6 +350,21 @@ describe('mode-aware wire contribution', () => {
expect(sdk?.text).toContain('top-level `await`') expect(sdk?.text).toContain('top-level `await`')
}) })
it("assembles under a python runtime in mode 'both' as well, SDK and schema together", async () => {
// `both` reaches the same wireSchemas/requireCodeRuntime/SDK-section code
// as `code`, so this pins the mode-by-language matrix rather than a
// separate path — including that `schemas()` under `both` projects the
// Python flavor instead of hitting the flavor-table guard.
const { ctx, systemPrompt } = await setup({ mode: 'both', runtime: { language: 'python' } })
registerEcho(ctx)
const assembly = await systemPrompt.assemble()
expect(assembly.sections.find(section => section.name === 'tools:sdk')?.text).toContain('class Tools(Protocol):')
const runCodeSchema = assembly.tools.find(tool => tool.name === RUN_CODE_NAME)
expect(runCodeSchema?.description).toContain('Execute a Python program')
// `both` keeps the native tools alongside run_code; `code` does not.
expect(assembly.tools.map(tool => tool.name)).toContain('echo')
})
it('emits a TypeScript-flavored run_code schema under a typescript runtime', async () => { it('emits a TypeScript-flavored run_code schema under a typescript runtime', async () => {
const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'typescript' } }) const { ctx, systemPrompt } = await setup({ mode: 'code', runtime: { language: 'typescript' } })
registerEcho(ctx) registerEcho(ctx)
+10 -4
View File
@@ -52,12 +52,18 @@ describe('jsonSchemaToPy', () => {
}) })
it('leans on JSON.stringify to keep a Literal parseable', () => { it('leans on JSON.stringify to keep a Literal parseable', () => {
// The two code points CPython refuses in source reach this path as well, // Nothing here escapes anything itself; `JSON.stringify` carries both
// and nothing here escapes them itself — `JSON.stringify` does, NUL as a // classes of hazard. The two code points CPython refuses anywhere in
// C0 control and a lone surrogate under ES2019 well-formed stringification. // source: NUL, and a lone surrogate under ES2019 well-formed
// Python decodes both escapes back to the value the schema declared. // stringification.
expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`) expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`)
expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`) expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`)
// And the ones that break this line in particular: a bare quote closing
// the literal early, a trailing backslash eating the closing quote, a bare
// newline ending it before its terminator. Every escape it emits is also a
// Python escape for the same character, so the value round-trips.
expect(jsonSchemaToPy({ type: 'string', const: 'say "hi"\n' })).toBe(String.raw`Literal["say \"hi\"\n"]`)
expect(jsonSchemaToPy({ type: 'string', const: 'ends\\' })).toBe(String.raw`Literal["ends\\"]`)
}) })
it('emits exact digits for a beyond-safe-range integer literal', () => { it('emits exact digits for a beyond-safe-range integer literal', () => {