Files
Andrew Beveridge ☄️andClaude Opus 5.5 d98e0d6a5e fix(runner-manager): retry a repo once on transient GitHub errors before paging (v0.295.1) (#1262)
## Summary
The stalled-job scan logged one `RemoteDisconnected` from the GitHub API
at ERROR, and the error monitor posted it to #alerts. The next scheduler
tick rescans every repo anyway, so one dropped connection doesn't mean
the safety net failed.

- Network failures (`URLError`, `http.client` errors,
`OSError`/timeouts) and GitHub 5xx/429 now get **one immediate retry**
for that repo (logged as plain INFO).
- If the retry also fails, or the error is not transient (403/404 auth
or permission errors), it still logs `Skipping <repo> in stalled-job
scan` at **ERROR**, so a broken safety net still pages. #1252 depends on
that.
- Pulled the per-repo body into `_queued_self_hosted_jobs_in_repo` so it
can be retried.

## Testing
- `pytest infrastructure/functions/runner_manager -q`: 75 passed, 4 of
them new (a retried disconnect doesn't page, two transient failures in a
row page, a 403 pages with no retry, 5xx/429 classification).
- Local CodeRabbit review: no findings.

## Deploy
This is a Cloud Function deployed by Pulumi, so it isn't live after
merge until a targeted `pulumi up` runs on the runner-manager function
(needs `/gcp-admin`).

Alert:
https://discord.com/channels/1198691101943541770/1265517528608669707/1569441998656635012

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_012RYLVWJ7FwupZXGBp7cE5X

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 15:35:08 -05:00
..