Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

ci: gate static lane's cache restore under failover; fix runbook recovery steps

Browse Source 
Review round on the pivoted design:

- node-24 (static) kept an unconditional hosted pnpm cache restore while the
  coverage and consumers lanes skip it under failover. On the self-hosted VM
  that restore downloads ~180 MB into /home/runner, a path pnpm never reads
  there, adding latency and contention during an outage. Gate it with the same
  `vars.DSH_CI_FAILOVER != 'selfhosted'` condition so all three lanes match.

- Runbook switch step 2 said "Re-run failed jobs", but the documented
  indefinite-queue outage leaves jobs queued (not failed), which cannot be
  re-run in place and do not retarget on variable change. Correct both language
  sides to cancel the run and re-run all jobs, or push a new commit.

- The standby-lane comment still described the switch as a one-line runs-on
  change; it is now setting the admin-only DSH_CI_FAILOVER variable.
This commit is contained in:
Chinesezjc
2026-07-26 05:44:35 +08:00
parent 68e280ce4f
commit 498df1d8de
4 changed files with 9 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md
+1 -1
View File
@@ -19,7 +19,7 @@ Each of the three required Linux jobs resolves its runner pool through the `DSH_
### Switch (repo admin, ~1 minute, no merge)
1. Repository **Settings → Secrets and variables → Actions → Variables → New repository variable**: name `DSH_CI_FAILOVER`, value `selfhosted`.
2. Re-run the failed/queued required jobs (Re-run failed jobs on affected PRs, or let new pushes pick it up).
2. Retrigger the required jobs so they re-resolve their pool. Jobs already **queued** for the hosted labels do not retarget and cannot be re-run in place, so for the documented indefinite-queue outage, cancel the stuck run and re-run all jobs, or push a new commit; "Re-run failed jobs" only helps once a job has actually failed rather than queued.
3. That is the entire switch. Under failover the workflow also, automatically: halves `DSH_COVERAGE_MAX_WORKERS` to 12 and `DSH_SNAPSHOT_MAX_CONCURRENCY` to 16 (shared-VM contention bounds), and skips the hosted-path pnpm cache restores (the VM's persistent store serves warm installs).
### Capacity during failover