fix(ci): bump GHA runner 2.334.0 → 2.336.0 (deprecated, wedging all self-hosted CI) (#902)

## Incident
All self-hosted CI jobs (`Backend - Unit Tests`, `Backend - Emulator
Integration Tests`, and any `build`/`gpu` jobs) have been **wedged in
`queued`** — ephemeral runner VMs boot, then self-terminate within ~1-2
min without ever picking up a job.

## Root cause (confirmed via live serial console)
GitHub **deprecated `actions/runner` v2.334.0**. Our ephemeral runners
register with JIT config (`run.sh --jitconfig`), which runs with
`disableUpdate` — so they **cannot self-update**. Serial console of a
dispatched runner VM:

```
√ Connected to GitHub
Current runner version: '2.334.0'
2026-08-12 ...Z: Listening for Jobs
An error occurred: Runner version v2.334.0 is deprecated and cannot receive messages.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
Shutdown scheduled for ...
```

The VM's `trap 'shutdown -h +1' EXIT` then terminates it, the
orphan-reaper deletes it, and the queued job is stranded. (The
immediate-TERMINATED reaping made this masquerade as Spot
preemption/capacity — it is not.)

This is a recurrence of the known runner-deprecation failure mode; the
provision script even documents it.

## Fix
Bump the pin `2.334.0 → 2.336.0` (current latest) in:
- `infrastructure/scripts/runner-image-provision.sh` (image bake — the
one the ephemeral `run.sh` actually uses)
- `infrastructure/compute/startup_scripts/github_runner.sh`,
`github_runner_gpu.sh` (baked startup scripts, kept consistent)

## Rollout
The **`Build GHA Runner Images`** workflow must rebuild the
`gha-runner-*` images so the ephemeral dispatcher (which selects the
newest non-deprecated image in each family) picks up the new binary.
I've dispatched a build for `general,build` from this branch. **`gpu` +
`gpu-windows` still need rebuilding too** (follow-up) to unblock GPU CI.

## Testing
Infra-only change; validated by the image build + a green self-hosted CI
run once new images are live.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated the GitHub Actions runner to version 2.336.0.
* Applied the update across standard, GPU, and provisioned runner
environments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude Opus 4.8 authored and GitHub committed 2026-08-12 15:00:21 -04:00
1 parent 5291d6ca78
commit f5d3bdf426
3 files changed
+3 -3

No files matched your search

@@ -165,7 +165,7 @@ usermod -aG docker runner
echo "Setting up GitHub Actions Runner..."
# Keep current with actions/runner releases — GitHub deprecates old versions
# (HTTP 403 on broker poll). See runner-image-provision.sh for the full note.
RUNNER_VERSION="2.334.0"
RUNNER_VERSION="2.336.0"
RUNNER_DIR="/home/runner/actions-runner"
mkdir -p $RUNNER_DIR
@@ -231,7 +231,7 @@ useradd -m -s /bin/bash runner 2>/dev/null || true
echo "Setting up GitHub Actions Runner..."
# Keep current with actions/runner releases — GitHub deprecates old versions
# (HTTP 403 on broker poll). See runner-image-provision.sh for the full note.
RUNNER_VERSION="2.334.0"
RUNNER_VERSION="2.336.0"
RUNNER_DIR="/home/runner/actions-runner"
mkdir -p $RUNNER_DIR
@@ -391,7 +391,7 @@ visudo -cf /etc/sudoers.d/runner
# registers, gets rejected, and self-halts, leaving CI jobs queued forever.
# When bumping, check https://github.com/actions/runner/releases/latest.
ck "phase: github actions runner binary"
RUNNER_VERSION="2.334.0"
RUNNER_VERSION="2.336.0"
RUNNER_DIR="/home/runner/actions-runner"
mkdir -p "$RUNNER_DIR"