Files
karaoke-gen/.github
Andrew Beveridge ☄️andClaude Fable 5 6514673fde perf: Cloud Run burst headroom — concurrency 32 + min-instances 2 (#1035)
## Summary

Final tuning from the 16-tab test diagnosis (#1033 fixed the edge rate
limit eating the requests; this fixes the remaining latency). With
nothing failing anymore, the warm 16-tab run still showed every tab at
~17–19s — server logs pinpointed why: the one warm instance answered
`correction-data` in **1.2s** while the three instances the burst
spawned answered in **15.8–18.8s** — Cloud Run holds requests during the
~16s app boot.

Concurrency 16 (set in #1028 while the event loop still had blocking
hazards) now overshoots: the loop is clean, one instance juggles dozens
of light requests fine, and forcing scale-out per burst just converts
load into cold-start waits.

## Change

`--concurrency 16 → 32` and `--min-instances 1 → 2`: 64 warm request
slots, so a 16-tab reload (~48 concurrent API calls) is absorbed with
zero cold starts. Still far below the original 80 that allowed the
2026-09-18 single-instance wedge.

## 💰 Cost note (Andrew)

The second min-instance idles at Cloud Run idle rates — roughly
**$15/month**. Single-flag revert if you'd rather not pay it; without it
the first burst after idle pays one ~16s cold-start wave for the
overflow beyond 32 concurrent slots.

## Follow-up worth considering (not in this PR)

App cold start is ~16s — heavy import graph (matplotlib, pydub, GCP
clients). Lazy-importing the heavy tails would benefit every scale-out.

## Testing

Deploy-flag-only change. Post-deploy: re-run
`scripts/load-test-review.py --tabs 16` — expect single-digit-second
tabs with no cold-start waves.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-22 18:26:41 -04:00
..
2024-09-17 12:18:12 -04:00
2025-12-11 03:49:32 -05:00