mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
feat: Upgrade GCE encoding worker to Python 3.13 (#221)
## Summary Upgrades the GCE encoding worker VM from Debian's default Python 3.11 to Python 3.13 built from source, aligning it with CI and Cloud Run Python version. ## Changes - Build Python 3.13.1 from source with `--enable-optimizations --with-lto` for best performance - Install to `/opt/python313` on the Debian 12 VM - Create virtual environment using Python 3.13 instead of system Python 3.11 - Add build dependencies (build-essential, libssl-dev, zlib1g-dev, etc.) - Remove `python3-pip` and `python3-venv` apt packages (no longer needed) - Update docs/README.md with recent changes entry ## Testing - [x] Pulumi preview shows only encoding-worker VM replacement (expected) - [x] Backend tests pass (`make test` - 1006 passed) - [x] Frontend tests pass (`npm run test:all` - 65 passed) - [x] Python and bash syntax validation pass ## Review - [x] Local CodeRabbit review completed - [x] No actionable issues found ## Notes - The VM will be **replaced** (not updated in-place) when this deploys - Python build from source takes ~5-10 minutes during VM startup - Brief encoding downtime expected during deployment @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
bd26d742a2
commit
07cde54b3f
2 files changed
+23
-4
No files matched your search
@@ -36,6 +36,8 @@
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- **GCE Encoding Worker Python 3.13** (2026-01-06): Upgraded GCE encoding worker from Debian's Python 3.11 to Python 3.13 built from source. Uses dedicated virtual environment at `/opt/encoding-worker/venv`, removing the need for `--break-system-packages`. Aligns the encoding worker with CI/Cloud Run Python version.
|
||||
|
||||
- **GCE Encoding Unified with LocalEncodingService** (2026-01-06): GCE encoding worker now uses the same `LocalEncodingService` as the local CLI, eliminating duplicated encoding logic. Output files now have proper names (`Artist - Title (Final Karaoke Lossless 4k).mp4` instead of `output_4k_lossless.mp4`) and include title/end screen concatenation. Wheel deployed to GCS; worker installs at job start for hot updates without VM restart. Removed all fallback logic - single code path for consistent output across CLI, Cloud Run, and GCE. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#unify-encoding-logic-with-gcs-wheel-deployment).
|
||||
|
||||
- **Cloud Distribution Matches Local CLI** (2026-01-06): Fixed cloud distribution (Dropbox uploads) to match local CLI output structure. The `lyrics/` folder now includes: proper filenames (`(Lyrics Corrections).json` instead of `(Karaoke).json`), all reference lyrics files (`(Lyrics Genius).txt`, `(Lyrics Spotify).txt`, etc.), `(With Vocals).mkv`, and `previews/` subfolder with preview ASS files. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#cloud-distribution-must-match-local-cli-output-structure).
|
||||
|
||||
@@ -1209,9 +1209,12 @@ set -e
|
||||
exec > >(tee /var/log/encoding-worker-startup.log) 2>&1
|
||||
echo "Starting encoding worker setup at $(date)"
|
||||
|
||||
# Install dependencies
|
||||
# Install dependencies (including Python build dependencies)
|
||||
apt-get update
|
||||
apt-get install -y python3-pip python3-venv docker.io curl git xz-utils
|
||||
apt-get install -y docker.io curl git xz-utils \\
|
||||
build-essential libssl-dev zlib1g-dev libbz2-dev \\
|
||||
libreadline-dev libsqlite3-dev libncursesw5-dev \\
|
||||
tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
|
||||
|
||||
# Install fonts for ASS subtitle rendering (libass uses fontconfig)
|
||||
# - fonts-noto: Noto Sans (default karaoke font)
|
||||
@@ -1234,12 +1237,26 @@ chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe
|
||||
rm -rf /tmp/ffmpeg*
|
||||
ffmpeg -version
|
||||
|
||||
# Build and install Python 3.13 from source
|
||||
echo "Building Python 3.13 from source..."
|
||||
PYTHON_VERSION="3.13.1"
|
||||
cd /tmp
|
||||
curl -L "https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz" -o python.tar.xz
|
||||
tar -xf python.tar.xz
|
||||
cd "Python-${PYTHON_VERSION}"
|
||||
./configure --prefix=/opt/python313 --enable-optimizations --with-lto
|
||||
make -j$(nproc)
|
||||
make install
|
||||
rm -rf /tmp/python.tar.xz /tmp/Python-${PYTHON_VERSION}
|
||||
/opt/python313/bin/python3.13 --version
|
||||
echo "Python 3.13 installed at /opt/python313"
|
||||
|
||||
# Create working directory
|
||||
mkdir -p /opt/encoding-worker
|
||||
cd /opt/encoding-worker
|
||||
|
||||
# Create Python virtual environment
|
||||
python3 -m venv venv
|
||||
# Create Python virtual environment using Python 3.13
|
||||
/opt/python313/bin/python3.13 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install Python dependencies
|
||||
|
||||
Reference in new issue
Block a user