Files
karaoke-gen/docs/REPLAY.md
T
Andrew Beveridge ☄️andClaude Opus 5.5 d63c9781e1 feat(replay): sync-triage verdict bar + vocal waveforms for stems-purged jobs (v0.292.0) (#1254)
## Summary
Andrew found the standalone triage page unintuitive, with no way to scan
visually. So the **real review UI in replay mode** becomes the triage
tool for the word-timing reference set (full-auto v2): anchors, gaps,
waveforms and audio he already knows.

## Changes
- **`ReplaySyncVerdict` bar** (replay only, admin): Perfectly synced /
Not perfect, reason chips, notes, Save / **Save & next job** through the
existing `queue` URL param.
  - It shows the saved verdict and flags unsaved changes.
- It saves via `POST /api/review/{id}/sync-reference`, the same store as
the review-modal checkbox from #1242, which now also accepts `reasons`
(trimmed, max 10).
- **Vocal waveform for old jobs:** storage retention purges stems after
30 days, so replaying old jobs showed no waveform. `_review_vocals_path`
now falls back to the backfilled `training/lead_vocals/{job_id}.flac`,
but only for stems-purged jobs. It's used by the vocals stream and
vocals-peaks.
- Docs: REPLAY.md § Sync triage.

## Testing
- Backend: reasons stored trimmed and capped; waveform fallback
(registered stem preferred, purged → training copy, unpurged never
checks storage). `test_routes_review` + `test_derived_vocals` +
sync-reference tests: 143 pass.
- Frontend: `ReplaySyncVerdict.test.tsx` (save & next, reasons + notes,
saved state, unsaved flag, disabled until chosen): 5 pass. tsc shows no
new errors.
- **Manual:** local replay (read-only against prod data) on a Feb-2026
stems-purged job (83414c76). The verdict bar renders, and Waveforms mode
shows the vocal waveform via the fallback (vocals-peaks 200).
- CodeRabbit ran locally; its one finding (unsaved state) is fixed.

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_012RYLVVVPsvPbXFhRYY5VSJ

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 21:33:16 +00:00

5.0 KiB

Review Replay (read-only re-open of completed jobs)

Re-open the real lyrics-review and instrumental-selection UIs for a job that's already completed, with full audio playback + backing-vocals previews (streamed from GCS), plus a side panel showing your ordered actions (AI-accept / AI-reject / manual / timing) from the stored edit log.

Built for the fully-automated-review recording program — see docs/archive/2026-08-25-full-auto-review-design.md.

What it is

  • Backend GET /api/review/{job_id}/correction-data?replay=true (admin/owner only): serves the same payload for a job in any status — skips the AWAITING_REVIEW/IN_REVIEW gate and the status-transition side-effect, and attaches the reviewer's edit_log. Purely read-only.
  • Frontend ?replay=1 on the review/instrumental route: bypasses the state gate (admins), renders the existing UI with isReadOnly (no submit / no mutations), and shows the Replay action log panel.
  • REVIEW_AUDIO_PROXY=1 (local dev only): the backend streams review audio bytes instead of GCS signed URLs, because local user ADC can't sign. Inert in prod.

Run it locally (against real prod data, read-only)

# Terminal 1 — backend (needs `gcloud auth application-default login` with read
# access to Firestore `jobs` + the GCS bucket; makes NO writes):
./scripts/run-replay-local.sh

# Terminal 2 — frontend:
cd frontend && npm run dev        # http://localhost:3000

Then in the browser:

  1. Open http://localhost:3000, open DevTools console, and sign in as admin:
    localStorage.setItem('karaoke_access_token', 'replay-local-token')  // matches ADMIN_TOKENS
    
    Reload — you're now admin@nomadkaraoke.com.
  2. Open a job in replay:
    • Lyrics review: http://localhost:3000/en/app/jobs?baseApiUrl=http://127.0.0.1:8000&replay=1#/<JOB_ID>/review
    • Instrumental screen: http://localhost:3000/en/app/jobs?baseApiUrl=http://127.0.0.1:8000&replay=1#/<JOB_ID>/instrumental

Audio, vocals waveform, and backing-vocals previews all play (proxied through the local backend). The submit/mutation controls are hidden; nothing can be changed.

Editable replay (&edit=1) — dev only

Append &edit=1 to the replay URL to make the review UI editable against the real job's data + audio (e.g. to exercise Tap To Sync / segment editing locally):

http://localhost:3000/en/app/jobs?baseApiUrl=http://127.0.0.1:8000&replay=1&edit=1#/<JOB_ID>/review

The header reads "Replay (editable · dev)". This flag is guarded by NODE_ENV !== 'production' — it is ignored in production builds, where replay stays strictly read-only. Edits are local-only (the replay backend makes no writes).

Using it for the recording sessions

Work through the job list in the private corpus (~/Projects/nomadkaraoke/docs/automation-corpus/). For each job, with the replay UI open, narrate the two things we're capturing (the agent writes them into the corpus record):

  1. Lyrics — what I corrected and why (per manual edit / AI rejection: why the AI was wrong, how you knew the right answer, whether a deterministic check could catch it). Timings too: which timing adjustments you make, and what signal (e.g. the separated vocal audio) could detect them automatically.
  2. Backing vocals — what I heard and how I decided (clean vs. with-backing, why; what would make it a clear yes/no vs. a judgement call).

The action-log panel groups your edits as AI ✓ / AI ✗ / manual / timing so it's easy to talk through what the AI got right, what you overrode, and what's left.

Notes / limitations

  • Fidelity is "final state + ordered action log", not a per-keystroke visual replay (no intermediate full-state snapshots are stored). The final lyrics are shown with AI-corrected words tagged (ai_corrected, original_text, timing_estimated).
  • If a job had its outputs deleted (admin "delete outputs"), stems may be gone and audio won't load — the lyrics/action-log still work.

Sync triage: building the word-timing reference set (full-auto v2)

Replay doubles as the triage UI for the perfectly synced reference set: the jobs whose reviewed word timing is trusted for aligner training and evaluation. In production, open replay as an admin with a queue of job ids:

https://gen.nomadkaraoke.com/en/app/jobs?replay=1&queue=<id1>,<id2>,...#/<id1>/review

A Reference set bar under the replay nav records the verdict:

  • Perfectly synced: a confident spot-check is enough (anchors, waveforms, a listen to anything suspicious).
  • Not perfect: tick reason chips, add optional notes.
  • Save & next job saves and moves to the next queued job.

Verdicts go to state_data.sync_reference through POST /api/review/{job_id}/sync-reference (admin only), the same store as the review modal's "perfectly synced" checkbox.

For jobs whose stems storage retention purged, the vocal waveform falls back to the backfilled training copy training/lead_vocals/{job_id}.flac (_review_vocals_path).