mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
perf: Cloud Run burst headroom — concurrency 32 + min-instances 2 (#1035)
## Summary Final tuning from the 16-tab test diagnosis (#1033 fixed the edge rate limit eating the requests; this fixes the remaining latency). With nothing failing anymore, the warm 16-tab run still showed every tab at ~17–19s — server logs pinpointed why: the one warm instance answered `correction-data` in **1.2s** while the three instances the burst spawned answered in **15.8–18.8s** — Cloud Run holds requests during the ~16s app boot. Concurrency 16 (set in #1028 while the event loop still had blocking hazards) now overshoots: the loop is clean, one instance juggles dozens of light requests fine, and forcing scale-out per burst just converts load into cold-start waits. ## Change `--concurrency 16 → 32` and `--min-instances 1 → 2`: 64 warm request slots, so a 16-tab reload (~48 concurrent API calls) is absorbed with zero cold starts. Still far below the original 80 that allowed the 2026-09-18 single-instance wedge. ## 💰 Cost note (Andrew) The second min-instance idles at Cloud Run idle rates — roughly **$15/month**. Single-flag revert if you'd rather not pay it; without it the first burst after idle pays one ~16s cold-start wave for the overflow beyond 32 concurrent slots. ## Follow-up worth considering (not in this PR) App cold start is ~16s — heavy import graph (matplotlib, pydub, GCP clients). Lazy-importing the heavy tails would benefit every scale-out. ## Testing Deploy-flag-only change. Post-deploy: re-run `scripts/load-test-review.py --tabs 16` — expect single-digit-second tabs with no cold-start waves. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
438efb751d
commit
6514673fde
2 files changed
+12
-9
No files matched your search
@@ -1976,12 +1976,15 @@ jobs:
|
||||
echo "Version: $VERSION"
|
||||
echo "Previously-serving revision: ${PREV_REVISION:-<unknown>}"
|
||||
|
||||
# --concurrency 16 (was 80): the backend is one uvicorn process per
|
||||
# 2-vCPU instance; review-page loads are heavy (audio byte-serving,
|
||||
# waveform work). At 80 a 10-tab burst all landed on the single
|
||||
# min-instance and wedged it ("no available instance" aborts,
|
||||
# 2026-09-18 investigation). 16 makes bursts trigger scale-out while
|
||||
# staying comfortably above normal per-instance load.
|
||||
# --concurrency 32 (was 80 → 16 → 32): at 80 a 10-tab burst all
|
||||
# landed on the single min-instance and wedged its then-blocking
|
||||
# event loop ("no available instance" aborts, 2026-09-18). 16 fixed
|
||||
# that but overshot once the loop was made non-blocking: every burst
|
||||
# forced scale-out, and each overflow request waited out a ~16s app
|
||||
# cold start (2026-09-22 16-tab test: warm instance served in 1.2s,
|
||||
# freshly-spawned ones in 16-19s). 32 + --min-instances 2 gives 64
|
||||
# warm slots so a 16-tab reload is absorbed without cold starts,
|
||||
# while staying far below the wedge-era 80.
|
||||
gcloud run deploy karaoke-backend \
|
||||
--image $IMAGE \
|
||||
--no-traffic \
|
||||
@@ -1995,9 +1998,9 @@ jobs:
|
||||
--cpu-throttling \
|
||||
--execution-environment gen2 \
|
||||
--timeout 1800 \
|
||||
--concurrency 16 \
|
||||
--concurrency 32 \
|
||||
--max-instances 20 \
|
||||
--min-instances 1 \
|
||||
--min-instances 2 \
|
||||
--vpc-connector projects/nomadkaraoke/locations/us-central1/connectors/cloud-run-connector \
|
||||
--vpc-egress private-ranges-only \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,DEFAULT_PRIVATE_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-NonPublished,DEFAULT_PRIVATE_BRAND_PREFIX=NOMADNP,USE_GCE_ENCODING=true,USE_GCE_PREVIEW_ENCODING=true,ENABLE_PUSH_NOTIFICATIONS=true,USE_CLOUD_RUN_JOBS_FOR_VIDEO=true,USE_CLOUD_RUN_JOBS_FOR_RENDER=true,EMAIL_PROVIDER=postmark,AUTO_CORRECT_MODEL=claude-opus-4-8,AUTO_CORRECT_COMPARE_MODELS=claude-opus-4-8;gemini-3.1-pro-preview,AUTO_CORRECT_PROACTIVE_ENABLED=true,AUTO_APPROVAL_BACKING_KEEP_ENABLED=true,COMMUNITY_DAILY_PICK_ENABLED=true,EDGE_AUTH_MODE=enforce,COMMIT_SHA=${{ steps.build-meta.outputs.sha }},PR_NUMBER=${{ steps.build-meta.outputs.pr_number }},PR_TITLE=${{ steps.build-meta.outputs.pr_title }}" \
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "karaoke-gen"
|
||||
version = "0.234.0"
|
||||
version = "0.234.1"
|
||||
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
|
||||
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
|
||||
license = "MIT"
|
||||
|
||||
Reference in new issue
Block a user