Files
karaoke-gen/Makefile
T
Andrew Beveridge ☄️andClaude Opus 4.7 8c530f5ade feat: syllable-aware custom lyrics with validate-and-repair loop (#741)
## Summary

Enhances the Custom Lyrics LLM mode with syllable-count validation and
per-line repair, plus user-facing settings (`allow_reword`,
`allow_omit`, `fixed_line_count`, `strictness`) and a per-line preview
UI showing target/actual syllables and severity badges.

- **Backend**: New `GenerationSettings` model, validator computing
per-line `min_delta` against the existing `SyllablesMatchHandler`
engine, repair loop (max 3 iterations, plateau-aware on the full
`(violations, total_delta)` score tuple), proportional timing
redistribution for variable-line-count outputs, structured per-iteration
logging with worst-3-line breadcrumbs.
- **Frontend**: Collapsible `CustomLyricsSettings` panel (3 toggles +
5-button strictness segmented control with ARIA group semantics +
contradiction warning), `CustomLyricsPreview` with severity badges +
iterations badge + stop-reason banner, wired into existing
`CustomLyricsMode` modal. Snake-case API response shape
(`line_metadata`, `iterations_used`, `stop_reason`, `settings_applied`,
`new_segment_timing`).
- **i18n**: 33 new keys across 33 locales (auto-translated via Gemini
pipeline).
- **Eval harness**: `backend/eval/custom_lyrics/` with disk-backed
Gemini cache (record-and-replay), per-line + per-fixture + corpus
scorers, CLI runner (`make eval-custom-lyrics`), and 4 fixtures (Year 5
Stars from production + 3 handcrafted public-domain).
- **Production E2E**: Updated existing test for the new preview UI + 3
new tests for settings panel toggles.

## Risk / Deferred Validation

The eval harness is built and ready, but **Phase 8 baseline capture
(v1-ships criteria validation) was deferred** — we ship and test in
prod. If the Year-5-Stars fixture doesn't hit `pct_pass_at_2 >= 75%`
post-merge, the spec lists fallback paths (chunked generation, per-line
repair); the eval can be run anytime via `make eval-custom-lyrics`.

Repair-loop logging is structured and rich so any prod failure is
diagnosable from Cloud Logging without re-running locally — log
breadcrumbs include per-iteration violation count, score deltas, and the
worst 3 failing lines with their target/candidate text + syllable
arrays.

## Test plan

- [x] Backend unit tests: 77 passing
- [x] Frontend unit tests: 32 passing across CustomLyricsSettings,
CustomLyricsPreview, CustomLyricsMode, customLyrics API client,
segmentsFromLines
- [x] TypeScript compiles cleanly for the touched files
- [x] CodeRabbit local review run; real bugs in timing.py (median +
zero-syllable) and validator.py (segment-duration hardening) fixed
inline
- [ ] **Post-merge**: real Gemini run against Year-5-Stars production
job, check Cloud Logging for `custom_lyrics_repair_iter_complete`
events, verify `pct_pass_at_2` matches expectations
- [ ] **Post-merge**: run prod E2E suite once admin token is plumbed in
CI (or manually)

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:26:09 -04:00

140 lines
5.4 KiB
Makefile

# Makefile for karaoke-gen project
# Run `make help` to see available commands
.PHONY: help install install-backend install-frontend build-frontend dev-install test test-unit test-backend test-e2e test-frontend test-all lint clean emulators-start emulators-stop eval-custom-lyrics
# Per-stage timeout in seconds (default: 10 minutes)
# Prevents hangs from stale dev servers, broken emulators, etc.
# Override with: make test TEST_TIMEOUT=300
TEST_TIMEOUT ?= 600
# Default target
help:
@echo "Available commands:"
@echo " make test - Run ALL tests (backend + frontend, installs deps automatically)"
@echo " make test-backend - Run backend tests only (unit + emulator)"
@echo " make test-frontend - Run frontend unit tests (Jest)"
@echo " make test-frontend-e2e - Run frontend E2E tests (Playwright, requires dev server)"
@echo " make test-unit - Run unit tests only (karaoke_gen package)"
@echo " make test-e2e - Run emulator tests with auto-start/stop"
@echo " make install - Install all dependencies (backend + frontend)"
@echo " make build-frontend - Build frontend and copy to Python package"
@echo " make dev-install - Build frontend + pip install -e . (for local CLI testing)"
@echo " make lint - Run linter checks"
@echo " make emulators-start - Start GCP emulators for local development"
@echo " make emulators-stop - Stop GCP emulators"
@echo ""
@echo "Before committing, run: make test"
# Install dependencies (only if needed)
install-backend:
@if [ ! -d "$$(poetry env info --path 2>/dev/null)" ] || ! poetry run python -c "import fastapi" 2>/dev/null; then \
echo "=== Installing backend dependencies ==="; \
poetry install; \
fi
install-frontend:
@if [ ! -d "frontend/node_modules" ]; then \
echo "=== Installing frontend dependencies ==="; \
cd frontend && npm install; \
fi
install: install-backend install-frontend
# Build frontend and copy to Python package
build-frontend: install-frontend
@echo "=== Building frontend ==="
cd frontend && npm run build
@echo "=== Copying build to Python package ==="
rm -rf karaoke_gen/nextjs_frontend/out
mkdir -p karaoke_gen/nextjs_frontend/out
cp -r frontend/out/* karaoke_gen/nextjs_frontend/out/
@echo "✅ Frontend built and copied to karaoke_gen/nextjs_frontend/out/"
# Build frontend + install package in editable mode for local CLI testing
# After running this, `karaoke-gen` CLI will use the freshly built frontend.
dev-install: build-frontend
@echo "=== Installing package in editable mode ==="
pip install -e .
@echo ""
@echo "✅ Ready! You can now run karaoke-gen with the updated frontend."
@echo " Example: karaoke-gen input.flac \"Artist\" \"Title\""
# Run unit tests for karaoke_gen package
test-unit: install-backend
@echo "=== Running karaoke_gen unit tests ==="
timeout $(TEST_TIMEOUT) poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing --cov-fail-under=69
# Run backend unit tests (excludes emulator tests)
test-backend-unit: install-backend
@echo "=== Running backend unit tests ==="
timeout $(TEST_TIMEOUT) poetry run pytest backend/tests/ --ignore=backend/tests/emulator -v
# Run E2E integration tests with emulators
test-e2e: install-backend
@echo "=== Running E2E integration tests with emulators ==="
@./scripts/start-emulators.sh || (echo "Failed to start emulators" && exit 1)
@export FIRESTORE_EMULATOR_HOST=127.0.0.1:8080 && \
export STORAGE_EMULATOR_HOST=http://127.0.0.1:4443 && \
export GOOGLE_CLOUD_PROJECT=test-project && \
export GCS_BUCKET_NAME=test-bucket && \
export ADMIN_TOKENS=test-admin-token && \
export ENVIRONMENT=test && \
timeout $(TEST_TIMEOUT) poetry run pytest backend/tests/emulator/ -v; \
TEST_RESULT=$$?; \
./scripts/stop-emulators.sh; \
exit $$TEST_RESULT
# Run all backend tests (unit + emulator)
test-backend: test-unit test-backend-unit test-e2e
@echo ""
@echo "✅ Backend tests passed!"
# Run frontend unit tests (Jest only - E2E requires manual setup)
test-frontend: install-frontend
@echo "=== Running frontend unit tests ==="
cd frontend && timeout $(TEST_TIMEOUT) npm run test:unit
# Run frontend E2E tests (requires dev server - run manually, not in make test)
test-frontend-e2e: install-frontend
@echo "=== Running frontend E2E tests ==="
cd frontend && timeout $(TEST_TIMEOUT) npm run test:e2e
# Run ALL tests (backend + frontend) - use this before committing!
test: test-backend test-frontend
@echo ""
@echo "✅ All tests passed! Ready to commit."
# Alias for test
test-all: test
# Lint checks
lint:
@echo "=== Running lint checks ==="
poetry run ruff check karaoke_gen/ backend/
poetry run ruff format --check karaoke_gen/ backend/
# Start emulators for local development
emulators-start:
@echo "=== Starting GCP emulators ==="
./scripts/start-emulators.sh
# Stop emulators
emulators-stop:
@echo "=== Stopping GCP emulators ==="
./scripts/stop-emulators.sh
# Clean up temporary files
clean:
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find . -type d -name ".pytest_cache" -exec rm -rf {} + 2>/dev/null || true
find . -type d -name ".coverage" -exec rm -rf {} + 2>/dev/null || true
find . -type f -name "*.pyc" -delete 2>/dev/null || true
rm -rf htmlcov/ .coverage coverage.xml 2>/dev/null || true
@echo "Cleaned up temporary files"
.PHONY: eval-custom-lyrics
eval-custom-lyrics:
cd backend && poetry run python -m backend.eval.custom_lyrics.run