mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Summary Final tuning from the 16-tab test diagnosis (#1033 fixed the edge rate limit eating the requests; this fixes the remaining latency). With nothing failing anymore, the warm 16-tab run still showed every tab at ~17–19s — server logs pinpointed why: the one warm instance answered `correction-data` in **1.2s** while the three instances the burst spawned answered in **15.8–18.8s** — Cloud Run holds requests during the ~16s app boot. Concurrency 16 (set in #1028 while the event loop still had blocking hazards) now overshoots: the loop is clean, one instance juggles dozens of light requests fine, and forcing scale-out per burst just converts load into cold-start waits. ## Change `--concurrency 16 → 32` and `--min-instances 1 → 2`: 64 warm request slots, so a 16-tab reload (~48 concurrent API calls) is absorbed with zero cold starts. Still far below the original 80 that allowed the 2026-09-18 single-instance wedge. ## 💰 Cost note (Andrew) The second min-instance idles at Cloud Run idle rates — roughly **$15/month**. Single-flag revert if you'd rather not pay it; without it the first burst after idle pays one ~16s cold-start wave for the overflow beyond 32 concurrent slots. ## Follow-up worth considering (not in this PR) App cold start is ~16s — heavy import graph (matplotlib, pydub, GCP clients). Lazy-importing the heavy tails would benefit every scale-out. ## Testing Deploy-flag-only change. Post-deploy: re-run `scripts/load-test-review.py --tabs 16` — expect single-digit-second tabs with no cold-start waves. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>