mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Summary - Implements robust deployment pattern for GCE encoding worker to eliminate recurring version mismatch issues - Fixed wheel path (`karaoke_gen-current.whl`) eliminates version sorting bugs - Self-updating startup script downloaded from GCS on every service start - Version manifest with strict CI verification (deployment fails on mismatch) ## Problem The encoding worker had recurring version issues because: - `startup.sh` was baked into the Packer image (required rebuild + VM recreation) - Wheel selection used version sorting across 51 files - CI only warned on version mismatch, didn't fail - Multiple deployment vectors (CI, Pulumi, manual SSH) caused drift ## Solution 1. **Fixed wheel path**: CI uploads to `karaoke_gen-current.whl` (no sorting needed) 2. **Self-updating startup**: `bootstrap.sh` downloads `startup.sh` from GCS on every start 3. **Version manifest**: CI writes expected version, service verifies before starting 4. **Strict CI verification**: Deployment fails if version doesn't match (was just warning) ## Changes - **New**: `infrastructure/encoding-worker/` with `bootstrap.sh`, `startup.sh`, README - **Modified**: CI workflow for fixed paths and strict verification - **Modified**: Packer `provision.sh` for bootstrap pattern - **Updated**: `LESSONS-LEARNED.md` with immutable deployment pattern - **Updated**: `docs/README.md` and `docs/ARCHITECTURE.md` ## Test plan - [x] Shell scripts validated (`bash -n`) - [x] Backend tests pass (1703 passed) - [x] CodeRabbit review passed - [ ] After merge: Rebuild Packer image - [ ] After merge: Recreate VM to use new image - [ ] After merge: Verify first deployment uses new pattern ## Post-Merge Steps Required 1. Rebuild Packer image: `packer build infrastructure/packer/encoding-worker.pkr.hcl` 2. Recreate VM: Delete encoding-worker, run `pulumi up` 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
43 lines
1.4 KiB
Bash
43 lines
1.4 KiB
Bash
#!/bin/bash
|
|
# Encoding Worker Bootstrap Script
|
|
#
|
|
# This is the ONLY script baked into the Packer image.
|
|
# It downloads and executes the real startup script from GCS.
|
|
#
|
|
# Why this pattern?
|
|
# - startup.sh logic can be updated without rebuilding the image
|
|
# - All code changes go through CI → GCS → service restart
|
|
# - No version sorting bugs (fixed path for wheel)
|
|
# - Strict version verification before starting
|
|
#
|
|
# This script should NEVER need modification. All logic lives in
|
|
# the GCS-hosted startup.sh which is managed by CI.
|
|
|
|
set -e
|
|
|
|
BUCKET="gs://karaoke-gen-storage-nomadkaraoke"
|
|
WORKER_DIR="/opt/encoding-worker"
|
|
LOG_FILE="/var/log/encoding-worker-bootstrap.log"
|
|
|
|
exec > >(tee -a "$LOG_FILE") 2>&1
|
|
|
|
echo "=== Bootstrap: $(date) ==="
|
|
echo "Downloading latest startup script from GCS..."
|
|
|
|
# Download the CI-managed startup script
|
|
if ! gsutil cp "${BUCKET}/encoding-worker/startup.sh" "${WORKER_DIR}/startup-latest.sh" 2>&1; then
|
|
echo "ERROR: Failed to download startup.sh from GCS"
|
|
echo "Falling back to local startup.sh if available..."
|
|
if [ -f "${WORKER_DIR}/startup-fallback.sh" ]; then
|
|
cp "${WORKER_DIR}/startup-fallback.sh" "${WORKER_DIR}/startup-latest.sh"
|
|
else
|
|
echo "FATAL: No fallback startup script available"
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
chmod +x "${WORKER_DIR}/startup-latest.sh"
|
|
|
|
echo "=== Bootstrap: Executing downloaded startup script ==="
|
|
exec "${WORKER_DIR}/startup-latest.sh"
|