Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

fix review finding: an impossible scripted permission click rejects the run

Browse Source 
A client-callback throw only becomes a JSON-RPC error RESPONSE to the
agent's session/request_permission — runScenario itself kept going, so a
tolerant agent could treat the error as a denial and the scenario would
pass, or worse, record: the impossible click baked into fixture and
golden, green on every replay. The mismatch is now captured as a harness
error while the agent is answered plain cancelled (a well-defined path
it cannot reinterpret), and the step loop rejects the run on it as soon
as the in-flight step settles. The spec asserts the rejection instead of
the agent-side error echo.
This commit is contained in:
kingwl
2026-07-08 02:46:23 +08:00
parent 9ab3a89cea
commit 1097fa3507
4 changed files with 29 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/support/acp-snapshot/tests/harness.spec.ts
+7 -7
View File
@@ -257,16 +257,16 @@ describe('runScenario', () => {
expect(result.rawStdout).toContain('permission:{\\"outcome\\":\\"selected\\",\\"optionId\\":\\"opt-reject\\"}')
})
it('fails loud on a scripted permission kind the agent never offered', { timeout: 20_000 }, async () => {
it('rejects the run on a scripted permission kind the agent never offered', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ permissionProbe: true })
// The fake bin offers allow_once/reject_once; scripting allow_always is a
// scenario bug. The client handler throws, the SDK surfaces it as a
// JSON-RPC error on the permission request, and the fake bin echoes the
// missing outcome as null.
const result = await runScenario(
// scenario bug. The agent is answered `cancelled` (it must not be able to
// absorb the bug as an error-means-denial), and the RUN fails: a callback
// throw would only reach the agent as a JSON-RPC error response, letting
// a tolerant agent carry on and the scenario pass — or record.
await expect(runScenario(
{ steps: [...boot, { op: 'prompt', text: 'impossible click' }], permissionAnswers: [{ kind: 'allow_always' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.rawStdout).toContain('permission:null')
)).rejects.toThrow(/allow_always not among the offered options \[allow_once, reject_once\]/)
})
})