mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Summary Ships the **fully-automated review framework**: jobs now default to `review_mode="auto"` — when the auto-approval scorer is fully confident on BOTH axes, the combined review screens are skipped entirely and the job renders straight through. Everything else falls back to human review exactly as before. **The confident class (deliberately narrow to start, ~5% of jobs expected):** - **Lyrics**: synced reference present, ≥90-99% anchor coverage, and every gap word covered by the pre-generated AI auto-correct suggestions (which the executor applies server-side — a faithful port of the review UI's on-load auto-apply, including conflict-group resolution) — OR zero gaps outright. - **Backing vocals**: non-subjective only — no audible backing content → clean instrumental. - **Never-auto gates**: vocalization sections (da-da-dun/woo-woo — P3), phantom/absurd-duration lines (P8), too-few-words, no/unsynced reference, errored backing analysis. **Calibrated against a 20-job human-review corpus** (all recent admin reviews replayed + annotated): safety 100% (every job that truly needed a human stays in review), lyrics-auto 3/20, overall-auto 1/20. ## How it works - `screens_worker` (pre-AWAITING_REVIEW) and `audio_worker` (post-stems, for the audio-lags-lyrics ordering) both call `maybe_auto_complete_review()`. Every job gets a verdict recorded in `processing_metadata.auto_approval` (shadow data keeps accumulating for calibration); enforcement additionally requires: flag on + `review_mode="auto"` + not made-for-you + no existing/custom instrumental + audio complete. - Enforce path mirrors a human "Complete Review": apply cached AI suggestions → sanity checks → `corrections_updated.json` → `instrumental_selection="clean"` → clear progress keys → `GENERATING_SCREENS→REVIEW_COMPLETE` (new legal edge; no review token or "ready for review" notification is ever issued) → trigger render worker. - **Fail-safe everywhere**: stale suggestions, duplicate-word artifacts (P1 self-conflict signature), state changed mid-enforce (human opened editor), any exception → normal human review. Auto-approval can never fail a job. - Kill switch: `AUTO_APPROVAL_ENFORCE_ENABLED=false` (scoring continues). Per-job: `review_mode="always_review"` (admin PATCH, validated). - Excluded from enforcement (still shadow-scored): made-for-you orders (human QC is part of that product), custom-instrumental jobs. ## Also in this PR (sessions 1-3 of the initiative) - **Review replay tool** (admin-only, prod-inert): `?replay=1` reopens the real lyrics/instrumental review UIs read-only for completed jobs — queue nav, Post-AI/Final toggle, edit-log action panel, dev-only audio byte-proxy (env-gated `REVIEW_AUDIO_PROXY`, path-anchored). This is what produced the calibration corpus. - `auto_approval` service package: scorer (anchor/gap/AI-coverage signals + P3/P8 gate detectors), apply engine, executor, lyrics-diff, offline capture script. ## Testing - 100+ new unit tests across scorer/apply/executor/replay routes; full backend suite 4000+ green locally (8 pre-existing emulator failures reproduce identically on pristine `main` — local-env only, green in CI). Frontend: 1196 Jest tests green. - Scorer validated read-only against the 20 real prod jobs in the private calibration corpus. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
50 lines
2.1 KiB
Bash
Executable File
50 lines
2.1 KiB
Bash
Executable File
#!/bin/bash
|
|
# Run the backend locally in REPLAY mode against REAL production data (read-only),
|
|
# so you can re-open the lyrics-review + instrumental-selection UIs for COMPLETED
|
|
# jobs with full audio/backing-vocals playback.
|
|
#
|
|
# Uses your own gcloud ADC (must be `gcloud auth application-default login` as an
|
|
# account that can READ Firestore `jobs` + the GCS bucket). No writes are made.
|
|
# REVIEW_AUDIO_PROXY=1 makes the backend stream review audio bytes instead of GCS
|
|
# signed URLs (local user ADC can't sign) — nothing customer-facing changes.
|
|
#
|
|
# Usage:
|
|
# ./scripts/run-replay-local.sh
|
|
# then in another terminal: (cd frontend && npm run dev)
|
|
#
|
|
# See docs/REPLAY.md for the full walkthrough + the job list.
|
|
|
|
set -e
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
|
cd "$PROJECT_ROOT"
|
|
|
|
export GOOGLE_CLOUD_PROJECT="${GOOGLE_CLOUD_PROJECT:-nomadkaraoke}"
|
|
export GCS_BUCKET_NAME="${GCS_BUCKET_NAME:-karaoke-gen-storage-nomadkaraoke}"
|
|
export FIRESTORE_COLLECTION="${FIRESTORE_COLLECTION:-jobs}" # REAL prod jobs (not jobs-dev)
|
|
export ENVIRONMENT="development"
|
|
export EDGE_AUTH_MODE="off"
|
|
export REVIEW_AUDIO_PROXY="1" # stream bytes, no signing
|
|
export ADMIN_TOKENS="${ADMIN_TOKENS:-replay-local-token}" # matches the frontend token below
|
|
export PORT="${PORT:-8000}"
|
|
|
|
cat <<EOF
|
|
|
|
🎬 Replay backend starting on http://127.0.0.1:$PORT (REAL prod data, read-only)
|
|
|
|
Next steps:
|
|
1. In another terminal: cd frontend && npm run dev # http://localhost:3000
|
|
2. Open http://localhost:3000 in your browser, then in DevTools console run:
|
|
localStorage.setItem('karaoke_access_token', '$ADMIN_TOKENS')
|
|
(then reload — you'll be signed in as admin@nomadkaraoke.com)
|
|
3. Open a job in replay (LYRICS review):
|
|
http://localhost:3000/en/app/jobs?baseApiUrl=http://127.0.0.1:$PORT&replay=1#/<JOB_ID>/review
|
|
…or the INSTRUMENTAL screen:
|
|
http://localhost:3000/en/app/jobs?baseApiUrl=http://127.0.0.1:$PORT&replay=1#/<JOB_ID>/instrumental
|
|
|
|
Job list + walkthrough: docs/REPLAY.md
|
|
|
|
EOF
|
|
|
|
exec poetry run uvicorn backend.main:app --host 127.0.0.1 --port "$PORT"
|