Follow-up to #1220 (mobile Phase 6), from its whole-branch review.
## Problem
Cloud Run Job workers run inside `asyncio.run(...)`.
`JobManager._send_push_notification` (and the completion email / idle
reminder) used `loop.create_task(...)` and the worker returned
immediately, so `asyncio.run` cancelled the send. Prod logs: all 62
"Push notifications sent" lines in 14 days came from the API service,
none from worker jobs. Separately, worker jobs never had
`ENABLE_PUSH_NOTIFICATIONS` / VAPID keys, so even a delivered Web Push
would have been skipped.
## Customer impact check (emails)
Checked 2026-09-25 → 2026-10-25 (full log retention): **no customer
emails were lost.** 289 COMPLETE transitions → 11 deliberately
suppressed re-renders, 278 completion emails scheduled and 278 sent
(email_log has 297 completion records, all with Postmark MessageIDs;
Postmark outbound ≥ email_log). The email coroutines happen to do all
their I/O synchronously, so they finished in the task's first step
before `asyncio.run` cancelled; pushes hand off to a thread and were
cancelled. The drain removes that fragility for emails too. **Only
pushes were lost** (0 worker pushes in 14 days).
## Fix
- Notification tasks are tracked; every worker `main()` wraps its job in
`with_notification_drain` (bounded 10 s wait on our coroutines; provider
calls keep their own timeouts). The API server path stays
fire-and-forget.
- `failure_push_sent` is written only after the push was attempted;
in-flight id cleared via done-callback.
- Worker jobs (audio-download, audio-separation, bulk-search,
lyrics-transcription, video-encoding) get
`ENABLE_PUSH_NOTIFICATIONS=true` +
`VAPID_PUBLIC_KEY`/`VAPID_PRIVATE_KEY` (same secrets as the API).
CI-pinned jobs now `ignore_changes` their image (CI owns it), so a
targeted `pulumi up` changes env only.
- Expo messages carry `channelId: "jobs"` (the app's Android channel).
## Deploy
After merge, a targeted `pulumi up` from main on the 5 jobs. Preview
(read-only, from this branch): 5 updates, each only
+`ENABLE_PUSH_NOTIFICATIONS`/+`VAPID_*`, no removals, no image change,
283 unchanged.
**Behaviour change:** Web Push subscribers will start receiving the
worker-triggered pushes (video ready, needs review, failed) they were
always meant to get.
## Tests
6901 backend tests pass (emulator-integration file excluded: local env).
Tests reproduce the cancellation through each worker's real `main()`.
Per-task review + local CodeRabbit (no findings).
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01CZf1cM9vefnNw5djARRhyA
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>