mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
feat: emit + push original-vocals guide on NOMAD render (write-path) (#875)
## Summary
Makes the kjbox **"Original Vocals"** sing-along guide
**self-maintaining**. When karaoke-gen finalises a new public NOMAD
master, it now automatically emits an aligned original-vocals guide and
pushes it to GCS — no re-run of the kjbox retro-fit pipeline.
**Key insight:** at render time karaoke-gen already has the isolated
`mixed_vocals` stem (a byproduct of the karaoke instrumental) and knows
the exact intro offset (the job's style `intro_video_duration`). So the
guide is simply `silence[intro] + vocals`, capped to the master's
duration — **no correlation, no review, no alignment measurement**.
## Changes
- `backend/services/original_vocals_guide.py` (new): pure ffmpeg builder
(`adelay` + `-f flac`, atomic `.part` rename) + `probe_duration`.
Best-effort, never raises.
- `nomad_master_mirror`: `push_vocals_guide` +
`delete_vocals_guides_by_brand` (siblings of the 720p methods);
brand-recycle cleanup now clears guides too.
- `video_worker_orchestrator`: `_emit_original_vocals_guide` pulls the
`mixed_vocals` stem from `jobs/{id}/stems/vocals_clean.flac`, resolves
the **actual** intro duration (long-intro styles honoured; `0`
preserved), caps to the master, registers it in `output_files`.
- `gdrive_service`: pushes the guide beside the 720p master (same
Nomad-brand gating + best-effort/distribution-warning contract).
- `config`: `nomad_vocals_guide_gcs_prefix` (`files/Nomad
Karaoke/vocals-padded`).
## Behaviour
- **Public NOMAD only** (`is_nomad_public_brand`, skips `NOMADNP`).
Shares the `nomad_master_fast_sync_enabled` gate.
- **Inert** until the kjbox device pulls the new prefix into
`NOMAD-vocals-padded/` (separate kjbox PR) — guides just accumulate in
GCS. Existing 1,459 device guides untouched.
- Every step best-effort: a guide failure is a distribution warning,
never fatal.
## Testing
- Builder unit tests (synthetic tone+silence): offset, duration cap,
valid flac, best-effort `None` paths.
- Mirror push/delete + gating; gdrive + orchestrator wiring; intro=0 /
long-intro resolution.
- Full backend suite: **3711 passed** (excl. emulator). CodeRabbit: 1
finding (intro-0 truthiness) fixed.
Design spec:
`docs/superpowers/specs/2026-07-09-original-vocals-writepath-design.md`
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
6f86b3d901
commit
b8e67e6ba6
12 files changed
+816
-4
No files matched your search
@@ -27,6 +27,10 @@ class Settings(BaseSettings):
|
||||
divebar_files_bucket: str = os.getenv("DIVEBAR_FILES_BUCKET", "nomadkaraoke-divebar-files")
|
||||
nomad_master_gcs_prefix: str = os.getenv("NOMAD_MASTER_GCS_PREFIX", "files/Nomad Karaoke/MP4-720p")
|
||||
nomad_master_fast_sync_enabled: bool = os.getenv("NOMAD_MASTER_FAST_SYNC_ENABLED", "true").lower() == "true"
|
||||
# Original-vocals guide: a padded isolated-vocals file (silence[intro] + vocals) pushed to
|
||||
# a sibling prefix in the same Divebar bucket, which kjbox mirrors into NOMAD-vocals-padded/
|
||||
# to power the "Original Vocals" sing-along slider. Same brand gating as the master push.
|
||||
nomad_vocals_guide_gcs_prefix: str = os.getenv("NOMAD_VOCALS_GUIDE_GCS_PREFIX", "files/Nomad Karaoke/vocals-padded")
|
||||
|
||||
# Audio Separator API (for GPU processing)
|
||||
audio_separator_api_url: Optional[str] = os.getenv("AUDIO_SEPARATOR_API_URL")
|
||||
|
||||
@@ -335,6 +335,18 @@ class GoogleDriveService:
|
||||
if fast_sync_warning and warnings is not None:
|
||||
warnings.append(fast_sync_warning)
|
||||
|
||||
# Push the padded original-vocals guide (silence[intro] + mixed_vocals) to the
|
||||
# vocals prefix in the same Divebar mirror, so kjbox can layer it under the master
|
||||
# at the "Original Vocals" slider. Same Nomad-brand gating + best-effort contract
|
||||
# as the 720p push; the guide was pre-built by the orchestrator when available.
|
||||
guide_path = output_files.get("original_vocals_guide")
|
||||
if guide_path and os.path.exists(guide_path):
|
||||
guide_warning = self._fast_sync_vocals_guide(
|
||||
brand_code, guide_path, f"{filename_base}.flac"
|
||||
)
|
||||
if guide_warning and warnings is not None:
|
||||
warnings.append(guide_warning)
|
||||
|
||||
# Upload CDG ZIP to CDG/
|
||||
cdg_zip_path = output_files.get("final_karaoke_cdg_zip")
|
||||
if cdg_zip_path and os.path.exists(cdg_zip_path):
|
||||
@@ -386,6 +398,38 @@ class GoogleDriveService:
|
||||
logger.warning(f"Nomad master fast-sync skipped (unexpected error): {e}")
|
||||
return f"Nomad master fast-sync errored for {filename}: {e}"
|
||||
|
||||
def _fast_sync_vocals_guide(
|
||||
self, brand_code: str, local_path: str, filename: str
|
||||
) -> Optional[str]:
|
||||
"""Best-effort push of a freshly-built original-vocals guide to the Divebar GCS
|
||||
mirror's vocals prefix so kjbox can pull it into ``NOMAD-vocals-padded/``.
|
||||
|
||||
No-op unless the fast-sync is enabled and this is a public Nomad release
|
||||
(``NOMAD-####``, excluding ``NOMADNP``). Never raises — nothing downstream
|
||||
depends on the guide. Returns a human-readable warning on failure (surfaced via
|
||||
``distribution_warnings`` / the admin alert), or ``None`` on success or skip.
|
||||
"""
|
||||
try:
|
||||
from backend.config import settings
|
||||
from backend.services.nomad_master_mirror import (
|
||||
NomadMasterMirror,
|
||||
is_nomad_public_brand,
|
||||
)
|
||||
|
||||
if not settings.nomad_master_fast_sync_enabled:
|
||||
return None
|
||||
if not is_nomad_public_brand(brand_code):
|
||||
return None
|
||||
|
||||
if NomadMasterMirror().push_vocals_guide(local_path, filename):
|
||||
return None
|
||||
return (
|
||||
f"Original-vocals guide sync to the GCS mirror failed for {filename}"
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the pipeline
|
||||
logger.warning(f"Original-vocals guide sync skipped (unexpected error): {e}")
|
||||
return f"Original-vocals guide sync errored for {filename}: {e}"
|
||||
|
||||
def delete_file(self, file_id: str) -> bool:
|
||||
"""
|
||||
Delete a file from Google Drive.
|
||||
|
||||
@@ -43,6 +43,10 @@ class NomadMasterMirror:
|
||||
prefix = settings.nomad_master_gcs_prefix.strip("/")
|
||||
return f"{prefix}/{filename}"
|
||||
|
||||
def _vocals_blob_name(self, filename: str) -> str:
|
||||
prefix = settings.nomad_vocals_guide_gcs_prefix.strip("/")
|
||||
return f"{prefix}/{filename}"
|
||||
|
||||
def push_720p(self, local_path: str, filename: str) -> bool:
|
||||
"""Upload a 720p master to the divebar GCS mirror. Best-effort; returns success."""
|
||||
blob_name = self._blob_name(filename)
|
||||
@@ -77,6 +81,51 @@ class NomadMasterMirror:
|
||||
logger.warning("Nomad master mirror delete failed for %s: %s", blob_name, e)
|
||||
return False
|
||||
|
||||
def push_vocals_guide(self, local_path: str, filename: str) -> bool:
|
||||
"""Upload a padded original-vocals guide to the divebar GCS mirror's vocals
|
||||
prefix (which kjbox pulls into ``NOMAD-vocals-padded/``). Best-effort."""
|
||||
blob_name = self._vocals_blob_name(filename)
|
||||
try:
|
||||
blob = self._bucket.blob(blob_name)
|
||||
blob.upload_from_filename(local_path)
|
||||
logger.info(
|
||||
"Pushed original-vocals guide to gs://%s/%s",
|
||||
settings.divebar_files_bucket,
|
||||
blob_name,
|
||||
)
|
||||
return True
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the pipeline
|
||||
logger.warning("Vocals-guide push failed for %s: %s", blob_name, e)
|
||||
return False
|
||||
|
||||
def delete_vocals_guides_by_brand(self, brand_code: str) -> int:
|
||||
"""Delete ALL original-vocals guide objects for a Nomad release, matched by the
|
||||
unique ``{brand_code} - `` filename prefix (covers renames). Best-effort; returns
|
||||
the count deleted. Refuses a non-Nomad / bare brand code (same guardrail as the
|
||||
master prefix-delete) so a broad prefix can never wipe unrelated objects."""
|
||||
if not is_nomad_public_brand(brand_code):
|
||||
return 0
|
||||
suffix = brand_code.split("-", 1)[1] if "-" in brand_code else ""
|
||||
if not suffix.strip():
|
||||
return 0
|
||||
prefix = f"{settings.nomad_vocals_guide_gcs_prefix.strip('/')}/{brand_code} - "
|
||||
deleted = 0
|
||||
try:
|
||||
for blob in self._bucket.list_blobs(prefix=prefix):
|
||||
try:
|
||||
blob.delete()
|
||||
deleted += 1
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("Failed to delete vocals-guide object %s: %s", blob.name, e)
|
||||
if deleted:
|
||||
logger.info(
|
||||
"Removed %d original-vocals guide object(s) from gs://%s/%s*",
|
||||
deleted, settings.divebar_files_bucket, prefix,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the pipeline
|
||||
logger.warning("Vocals-guide mirror prefix-delete failed for %s: %s", prefix, e)
|
||||
return deleted
|
||||
|
||||
def delete_masters_by_brand(self, brand_code: str) -> int:
|
||||
"""Delete ALL 720p master objects for a Nomad release, matched by the unique
|
||||
``{brand_code} - `` filename prefix.
|
||||
@@ -125,7 +174,13 @@ def cleanup_nomad_masters(brand_code: Optional[str]) -> int:
|
||||
return 0
|
||||
if not is_nomad_public_brand(brand_code):
|
||||
return 0
|
||||
return NomadMasterMirror().delete_masters_by_brand(brand_code)
|
||||
mirror = NomadMasterMirror()
|
||||
deleted = mirror.delete_masters_by_brand(brand_code)
|
||||
# A recycled brand must not keep serving its old original-vocals guide, so remove
|
||||
# the GCS guide(s) too. (Note: with the device's additive-only vocals sync this
|
||||
# won't auto-remove an already-downloaded device copy — a documented limitation.)
|
||||
mirror.delete_vocals_guides_by_brand(brand_code)
|
||||
return deleted
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the caller
|
||||
logger.warning("Nomad master mirror cleanup skipped: %s", e)
|
||||
return 0
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Build the padded original-vocals guide that feeds kjbox's "Original Vocals" slider.
|
||||
|
||||
At render time karaoke-gen already has the isolated ``mixed_vocals`` stem (a byproduct
|
||||
of making the karaoke instrumental) and knows the exact intro offset (the master is
|
||||
``[silent title-card intro] + song + [tail]`` and the intro length is the job's style
|
||||
``intro_video_duration``). So the guide is simply ``silence[intro] + vocals``, capped to
|
||||
the master's duration — no cross-correlation, no human review, no alignment measurement.
|
||||
This is byte-compatible with what the kjbox retro-fit ``pad_vocals.sh`` produced.
|
||||
|
||||
Every function here is best-effort: it returns ``None`` / ``0`` on any failure and never
|
||||
raises, so a guide problem can never break the distribution pipeline.
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def probe_duration(path: str, ffprobe_path: str = "ffprobe") -> Optional[float]:
|
||||
"""Return a media file's duration in seconds, or ``None`` if it can't be measured."""
|
||||
try:
|
||||
if not path or not os.path.isfile(path):
|
||||
return None
|
||||
result = subprocess.run(
|
||||
[
|
||||
ffprobe_path, "-v", "error", "-show_entries", "format=duration",
|
||||
"-of", "default=noprint_wrappers=1:nokey=1", path,
|
||||
],
|
||||
capture_output=True, text=True, timeout=60,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
return float(result.stdout.strip())
|
||||
except Exception as e: # noqa: BLE001 - never fatal
|
||||
logger.warning("Could not probe duration for %s: %s", path, e)
|
||||
return None
|
||||
|
||||
|
||||
def build_original_vocals_guide(
|
||||
mixed_vocals_path: str,
|
||||
intro_seconds: float,
|
||||
dest_path: str,
|
||||
master_duration: Optional[float] = None,
|
||||
ffmpeg_path: str = "ffmpeg",
|
||||
) -> Optional[str]:
|
||||
"""Emit ``silence[intro] + mixed_vocals`` as a FLAC guide aligned to the master.
|
||||
|
||||
Args:
|
||||
mixed_vocals_path: The isolated vocals stem (all vocals: lead + backing).
|
||||
intro_seconds: The master's silent title-card length (job's ``intro_video_duration``).
|
||||
dest_path: Where to write the guide (``.flac``).
|
||||
master_duration: If given, the output is capped to this length (``-t``) so the
|
||||
guide can never bleed past the master. Omit to leave the vocals' own length.
|
||||
ffmpeg_path: ffmpeg binary.
|
||||
|
||||
Returns:
|
||||
``dest_path`` on success, else ``None`` (missing input / ffmpeg failure). Never raises.
|
||||
"""
|
||||
try:
|
||||
if not mixed_vocals_path or not os.path.isfile(mixed_vocals_path):
|
||||
logger.warning("Vocals guide: missing mixed-vocals stem %s", mixed_vocals_path)
|
||||
return None
|
||||
if intro_seconds is None or intro_seconds < 0:
|
||||
logger.warning("Vocals guide: invalid intro_seconds %r", intro_seconds)
|
||||
return None
|
||||
|
||||
delay_ms = int(round(float(intro_seconds) * 1000))
|
||||
os.makedirs(os.path.dirname(dest_path) or ".", exist_ok=True)
|
||||
tmp_path = dest_path + ".part"
|
||||
|
||||
cmd = [
|
||||
ffmpeg_path, "-y", "-hide_banner", "-nostats", "-loglevel", "error",
|
||||
"-i", mixed_vocals_path,
|
||||
"-af", f"adelay={delay_ms}:all=1",
|
||||
]
|
||||
if master_duration and master_duration > 0:
|
||||
cmd += ["-t", f"{float(master_duration):.3f}"]
|
||||
# ``-f flac`` is mandatory: the ``.part`` extension would otherwise make the
|
||||
# muxer guess the container from the extension and fail.
|
||||
cmd += ["-c:a", "flac", "-f", "flac", tmp_path]
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=600)
|
||||
if result.returncode != 0 or not os.path.isfile(tmp_path):
|
||||
logger.warning(
|
||||
"Vocals guide: ffmpeg failed (rc=%s) for %s: %s",
|
||||
result.returncode, mixed_vocals_path, (result.stderr or "")[-200:].strip(),
|
||||
)
|
||||
_quiet_remove(tmp_path)
|
||||
return None
|
||||
|
||||
os.replace(tmp_path, dest_path)
|
||||
logger.info("Built original-vocals guide: %s", dest_path)
|
||||
return dest_path
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the pipeline
|
||||
logger.warning("Vocals guide build failed for %s: %s", mixed_vocals_path, e)
|
||||
_quiet_remove(dest_path + ".part")
|
||||
return None
|
||||
|
||||
|
||||
def _quiet_remove(path: str) -> None:
|
||||
try:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
pass
|
||||
@@ -524,7 +524,97 @@ class TestUploadToPublicShare:
|
||||
# ...and the failure is surfaced as a distribution warning (admin alert),
|
||||
# not silently swallowed.
|
||||
assert any("fast-sync" in w for w in warnings)
|
||||
|
||||
|
||||
@patch("backend.services.gdrive_service.get_settings")
|
||||
def test_public_share_pushes_vocals_guide_for_nomad(self, mock_get_settings, tmp_path):
|
||||
"""A pre-built original-vocals guide is pushed to the vocals prefix with the exact
|
||||
"{brand_code} - {base_name}.flac" object name (matches the master's naming)."""
|
||||
from backend.services.gdrive_service import GoogleDriveService
|
||||
|
||||
mock_settings = Mock()
|
||||
mock_settings.get_secret.return_value = json.dumps({
|
||||
"refresh_token": "token", "client_id": "id", "client_secret": "secret",
|
||||
})
|
||||
mock_get_settings.return_value = mock_settings
|
||||
|
||||
mp4_720_file = tmp_path / "output_720p.mp4"
|
||||
mp4_720_file.write_bytes(b"720p video")
|
||||
guide_file = tmp_path / "original_vocals_guide.flac"
|
||||
guide_file.write_bytes(b"guide flac")
|
||||
|
||||
service = GoogleDriveService()
|
||||
with patch.object(service, "get_or_create_folder", return_value="f"), \
|
||||
patch.object(service, "upload_file", return_value="id"), \
|
||||
patch("backend.services.nomad_master_mirror.NomadMasterMirror") as mock_mirror_cls:
|
||||
mock_mirror_cls.return_value.push_vocals_guide.return_value = True
|
||||
service.upload_to_public_share(
|
||||
root_folder_id="root-123",
|
||||
brand_code="NOMAD-1163",
|
||||
base_name="Artist - Title",
|
||||
output_files={
|
||||
"final_karaoke_lossy_720p_mp4": str(mp4_720_file),
|
||||
"original_vocals_guide": str(guide_file),
|
||||
},
|
||||
)
|
||||
mock_mirror_cls.return_value.push_vocals_guide.assert_called_once_with(
|
||||
str(guide_file), "NOMAD-1163 - Artist - Title.flac"
|
||||
)
|
||||
|
||||
@patch("backend.services.gdrive_service.get_settings")
|
||||
def test_public_share_no_vocals_guide_for_non_nomad(self, mock_get_settings, tmp_path):
|
||||
"""Non-Nomad / NOMADNP brands must never push a guide to the public mirror."""
|
||||
from backend.services.gdrive_service import GoogleDriveService
|
||||
|
||||
mock_settings = Mock()
|
||||
mock_settings.get_secret.return_value = json.dumps({
|
||||
"refresh_token": "token", "client_id": "id", "client_secret": "secret",
|
||||
})
|
||||
mock_get_settings.return_value = mock_settings
|
||||
|
||||
guide_file = tmp_path / "original_vocals_guide.flac"
|
||||
guide_file.write_bytes(b"guide flac")
|
||||
|
||||
service = GoogleDriveService()
|
||||
with patch.object(service, "get_or_create_folder", return_value="f"), \
|
||||
patch.object(service, "upload_file", return_value="id"), \
|
||||
patch("backend.services.nomad_master_mirror.NomadMasterMirror") as mock_mirror_cls:
|
||||
service.upload_to_public_share(
|
||||
root_folder_id="root-123",
|
||||
brand_code="NOMADNP-1163",
|
||||
base_name="Artist - Title",
|
||||
output_files={"original_vocals_guide": str(guide_file)},
|
||||
)
|
||||
mock_mirror_cls.assert_not_called()
|
||||
|
||||
@patch("backend.services.gdrive_service.get_settings")
|
||||
def test_public_share_vocals_guide_failure_is_non_fatal(self, mock_get_settings, tmp_path):
|
||||
"""A guide push failure surfaces a warning but never breaks the Drive upload."""
|
||||
from backend.services.gdrive_service import GoogleDriveService
|
||||
|
||||
mock_settings = Mock()
|
||||
mock_settings.get_secret.return_value = json.dumps({
|
||||
"refresh_token": "token", "client_id": "id", "client_secret": "secret",
|
||||
})
|
||||
mock_get_settings.return_value = mock_settings
|
||||
|
||||
guide_file = tmp_path / "original_vocals_guide.flac"
|
||||
guide_file.write_bytes(b"guide flac")
|
||||
|
||||
service = GoogleDriveService()
|
||||
warnings: list = []
|
||||
with patch.object(service, "get_or_create_folder", return_value="f"), \
|
||||
patch.object(service, "upload_file", return_value="id"), \
|
||||
patch("backend.services.nomad_master_mirror.NomadMasterMirror") as mock_mirror_cls:
|
||||
mock_mirror_cls.return_value.push_vocals_guide.return_value = False
|
||||
service.upload_to_public_share(
|
||||
root_folder_id="root-123",
|
||||
brand_code="NOMAD-1163",
|
||||
base_name="Artist - Title",
|
||||
output_files={"original_vocals_guide": str(guide_file)},
|
||||
warnings=warnings,
|
||||
)
|
||||
assert any("guide" in w.lower() for w in warnings)
|
||||
|
||||
@patch("backend.services.gdrive_service.get_settings")
|
||||
def test_upload_to_public_share_skips_missing_files(
|
||||
self, mock_get_settings, tmp_path
|
||||
|
||||
@@ -134,16 +134,69 @@ def test_delete_masters_by_brand_is_non_fatal_on_list_error():
|
||||
assert mirror.delete_masters_by_brand("NOMAD-1500") == 0
|
||||
|
||||
|
||||
# --- vocals guide push / delete (parallel to the master methods) ---
|
||||
|
||||
def test_push_vocals_guide_uses_vocals_prefix_and_uploads():
|
||||
mirror, bucket, blob = _mirror_with_mock()
|
||||
|
||||
ok = mirror.push_vocals_guide(
|
||||
"/tmp/out/NOMAD-1500 - Rush - The Spirit of Radio.flac",
|
||||
"NOMAD-1500 - Rush - The Spirit of Radio.flac",
|
||||
)
|
||||
|
||||
assert ok is True
|
||||
bucket.blob.assert_called_once_with(
|
||||
"files/Nomad Karaoke/vocals-padded/NOMAD-1500 - Rush - The Spirit of Radio.flac"
|
||||
)
|
||||
blob.upload_from_filename.assert_called_once_with(
|
||||
"/tmp/out/NOMAD-1500 - Rush - The Spirit of Radio.flac"
|
||||
)
|
||||
|
||||
|
||||
def test_push_vocals_guide_is_non_fatal_on_error():
|
||||
mirror, _bucket, blob = _mirror_with_mock()
|
||||
blob.upload_from_filename.side_effect = RuntimeError("boom")
|
||||
assert mirror.push_vocals_guide("/tmp/x.flac", "NOMAD-1 - A - B.flac") is False
|
||||
|
||||
|
||||
def test_delete_vocals_guides_by_brand_deletes_all_matching_by_prefix():
|
||||
mirror, bucket, blobs = _mirror_with_blobs([
|
||||
"files/Nomad Karaoke/vocals-padded/NOMAD-1500 - Rush - Spirit of Radio.flac",
|
||||
"files/Nomad Karaoke/vocals-padded/NOMAD-1500 - Rush - The Spirit of Radio.flac",
|
||||
])
|
||||
n = mirror.delete_vocals_guides_by_brand("NOMAD-1500")
|
||||
assert n == 2
|
||||
bucket.list_blobs.assert_called_once_with(
|
||||
prefix="files/Nomad Karaoke/vocals-padded/NOMAD-1500 - "
|
||||
)
|
||||
for b in blobs:
|
||||
b.delete.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("brand", ["NOMADNP-1500", "VOCALSTAR-1", "", None, "NOMAD"])
|
||||
def test_delete_vocals_guides_by_brand_refuses_non_nomad_or_bare(brand):
|
||||
mirror, bucket, _blobs = _mirror_with_blobs([])
|
||||
assert mirror.delete_vocals_guides_by_brand(brand) == 0
|
||||
bucket.list_blobs.assert_not_called()
|
||||
|
||||
|
||||
def test_delete_vocals_guides_by_brand_is_non_fatal_on_list_error():
|
||||
mirror, bucket, _blobs = _mirror_with_blobs([])
|
||||
bucket.list_blobs.side_effect = RuntimeError("gcs down")
|
||||
assert mirror.delete_vocals_guides_by_brand("NOMAD-1500") == 0
|
||||
|
||||
|
||||
def test_cleanup_nomad_masters_delegates_for_nomad(monkeypatch):
|
||||
from backend.services import nomad_master_mirror as mod
|
||||
|
||||
called = {}
|
||||
inst = MagicMock()
|
||||
inst.delete_masters_by_brand.return_value = 3
|
||||
monkeypatch.setattr(mod, "NomadMasterMirror", lambda: inst)
|
||||
|
||||
assert mod.cleanup_nomad_masters("NOMAD-1500") == 3
|
||||
inst.delete_masters_by_brand.assert_called_once_with("NOMAD-1500")
|
||||
# Brand recycle must also clear the release's original-vocals guide(s).
|
||||
inst.delete_vocals_guides_by_brand.assert_called_once_with("NOMAD-1500")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("brand", ["NOMADNP-1", "OTHER-1", "", None])
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Unit tests for the padded original-vocals guide builder."""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from backend.services.original_vocals_guide import (
|
||||
build_original_vocals_guide,
|
||||
probe_duration,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
shutil.which("ffmpeg") is None or shutil.which("ffprobe") is None,
|
||||
reason="ffmpeg/ffprobe not available",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tone(tmp_path):
|
||||
"""A 4-second 440Hz tone standing in for the mixed-vocals stem."""
|
||||
src = tmp_path / "vocals.flac"
|
||||
subprocess.run(
|
||||
["ffmpeg", "-y", "-loglevel", "error", "-f", "lavfi",
|
||||
"-i", "sine=frequency=440:duration=4", "-c:a", "flac", str(src)],
|
||||
check=True,
|
||||
)
|
||||
return str(src)
|
||||
|
||||
|
||||
def _first_sound_onset(path):
|
||||
"""Seconds until audio first exceeds the silence threshold (leading-silence length)."""
|
||||
out = subprocess.run(
|
||||
["ffmpeg", "-hide_banner", "-i", path, "-af",
|
||||
"silencedetect=noise=-50dB:d=0.1", "-f", "null", "-"],
|
||||
capture_output=True, text=True,
|
||||
).stderr
|
||||
# A guide with N seconds of leading silence reports "silence_end: N".
|
||||
for line in out.splitlines():
|
||||
if "silence_end:" in line:
|
||||
return float(line.split("silence_end:")[1].split("|")[0].strip())
|
||||
return 0.0
|
||||
|
||||
|
||||
def test_prepends_intro_silence_and_reports_expected_duration(tmp_path, tone):
|
||||
dest = str(tmp_path / "NOMAD-1500 - A - B.flac")
|
||||
out = build_original_vocals_guide(tone, intro_seconds=5, dest_path=dest)
|
||||
|
||||
assert out == dest
|
||||
assert os.path.isfile(dest)
|
||||
# silence[5] + 4s tone ≈ 9s total, with the tone starting ~5s in.
|
||||
total = probe_duration(dest)
|
||||
assert total == pytest.approx(9.0, abs=0.3)
|
||||
assert _first_sound_onset(dest) == pytest.approx(5.0, abs=0.3)
|
||||
|
||||
|
||||
def test_respects_non_default_intro(tmp_path, tone):
|
||||
dest = str(tmp_path / "guide.flac")
|
||||
build_original_vocals_guide(tone, intro_seconds=10, dest_path=dest)
|
||||
# Proves we honour the job's actual intro, not a hardcoded 5s.
|
||||
assert _first_sound_onset(dest) == pytest.approx(10.0, abs=0.3)
|
||||
|
||||
|
||||
def test_caps_to_master_duration(tmp_path, tone):
|
||||
dest = str(tmp_path / "guide.flac")
|
||||
# silence[5] + 4s tone would be 9s; cap to 7s.
|
||||
build_original_vocals_guide(tone, intro_seconds=5, dest_path=dest, master_duration=7.0)
|
||||
assert probe_duration(dest) == pytest.approx(7.0, abs=0.3)
|
||||
|
||||
|
||||
def test_output_is_valid_flac(tmp_path, tone):
|
||||
dest = str(tmp_path / "guide.flac")
|
||||
build_original_vocals_guide(tone, intro_seconds=1, dest_path=dest)
|
||||
codec = subprocess.run(
|
||||
["ffprobe", "-v", "error", "-select_streams", "a:0",
|
||||
"-show_entries", "stream=codec_name", "-of",
|
||||
"default=noprint_wrappers=1:nokey=1", dest],
|
||||
capture_output=True, text=True,
|
||||
).stdout.strip()
|
||||
assert codec == "flac"
|
||||
|
||||
|
||||
def test_no_part_file_left_behind(tmp_path, tone):
|
||||
dest = str(tmp_path / "guide.flac")
|
||||
build_original_vocals_guide(tone, intro_seconds=1, dest_path=dest)
|
||||
assert not os.path.exists(dest + ".part")
|
||||
|
||||
|
||||
def test_missing_input_returns_none(tmp_path):
|
||||
out = build_original_vocals_guide(
|
||||
str(tmp_path / "nope.flac"), intro_seconds=5, dest_path=str(tmp_path / "g.flac")
|
||||
)
|
||||
assert out is None
|
||||
assert not os.path.exists(tmp_path / "g.flac")
|
||||
|
||||
|
||||
def test_negative_intro_returns_none(tmp_path, tone):
|
||||
assert build_original_vocals_guide(tone, intro_seconds=-1, dest_path=str(tmp_path / "g.flac")) is None
|
||||
|
||||
|
||||
def test_ffmpeg_failure_is_non_fatal(tmp_path, tone):
|
||||
# A bogus ffmpeg binary must not raise; just returns None.
|
||||
out = build_original_vocals_guide(
|
||||
tone, intro_seconds=5, dest_path=str(tmp_path / "g.flac"),
|
||||
ffmpeg_path="/nonexistent/ffmpeg",
|
||||
)
|
||||
assert out is None
|
||||
assert not os.path.exists(str(tmp_path / "g.flac") + ".part")
|
||||
|
||||
|
||||
def test_probe_duration_missing_file_returns_none(tmp_path):
|
||||
assert probe_duration(str(tmp_path / "nope.flac")) is None
|
||||
@@ -2049,3 +2049,86 @@ class TestStageOriginalAudio:
|
||||
# Must not raise
|
||||
orch._stage_original_audio_for_upload()
|
||||
assert any("Original audio" in w for w in orch.result.distribution_warnings)
|
||||
|
||||
|
||||
class TestOriginalVocalsGuideEmit:
|
||||
"""The write-path emit of the padded original-vocals guide at finalize."""
|
||||
|
||||
def _orch(self, brand="NOMAD-1500", video_720p=None, job_manager=None):
|
||||
config = OrchestratorConfig(
|
||||
job_id="job-1", artist="A", title="B",
|
||||
title_video_path="/t.mov", karaoke_video_path="/k.mov",
|
||||
instrumental_audio_path="/a.flac",
|
||||
)
|
||||
orch = VideoWorkerOrchestrator(config, job_manager=job_manager, storage=MagicMock())
|
||||
orch.result.brand_code = brand
|
||||
orch.result.final_video_720p = video_720p
|
||||
return orch
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_emit_skips_non_nomad(self, tmp_path):
|
||||
v = tmp_path / "v.mp4"; v.write_bytes(b"x")
|
||||
orch = self._orch(brand="VOCALSTAR-1", video_720p=str(v))
|
||||
assert await orch._emit_original_vocals_guide("VOCALSTAR-1") is None
|
||||
orch.storage.download_file.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_emit_none_when_no_720p(self):
|
||||
orch = self._orch(brand="NOMAD-1500", video_720p=None)
|
||||
assert await orch._emit_original_vocals_guide("NOMAD-1500") is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_emit_builds_guide_for_nomad(self, tmp_path):
|
||||
v = tmp_path / "v.mp4"; v.write_bytes(b"video")
|
||||
orch = self._orch(brand="NOMAD-1500", video_720p=str(v), job_manager=None)
|
||||
with patch("backend.services.original_vocals_guide.build_original_vocals_guide",
|
||||
return_value="/tmp/guide.flac") as mb, \
|
||||
patch("backend.services.original_vocals_guide.probe_duration", return_value=200.0):
|
||||
out = await orch._emit_original_vocals_guide("NOMAD-1500")
|
||||
assert out == "/tmp/guide.flac"
|
||||
# The mixed-vocals stem is pulled from the job's GCS stems path.
|
||||
assert orch.storage.download_file.call_args[0][0] == "jobs/job-1/stems/vocals_clean.flac"
|
||||
# Built with the resolved intro (5s fallback, no job_manager) capped to master dur.
|
||||
assert mb.call_args[0][1] == 5.0
|
||||
assert mb.call_args.kwargs.get("master_duration") == 200.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_emit_none_when_stem_download_fails(self, tmp_path):
|
||||
v = tmp_path / "v.mp4"; v.write_bytes(b"video")
|
||||
orch = self._orch(brand="NOMAD-1500", video_720p=str(v))
|
||||
orch.storage.download_file.side_effect = RuntimeError("gone")
|
||||
assert await orch._emit_original_vocals_guide("NOMAD-1500") is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_intro_seconds_defaults_to_5_without_job_manager(self):
|
||||
orch = self._orch(job_manager=None)
|
||||
assert await orch._resolve_intro_seconds() == 5.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_intro_seconds_preserves_explicit_zero(self):
|
||||
# A style with no title card (video_duration=0) must NOT collapse to the 5s
|
||||
# default — that would push the guide's vocals 5s late.
|
||||
orch = self._orch(job_manager=MagicMock())
|
||||
orch.job_manager.get_job.return_value = MagicMock()
|
||||
fake_style = MagicMock()
|
||||
fake_style.get_intro_format.return_value = {"video_duration": 0}
|
||||
|
||||
async def fake_load(*a, **k):
|
||||
return fake_style
|
||||
|
||||
with patch("backend.workers.style_helper.load_style_config", side_effect=fake_load):
|
||||
assert await orch._resolve_intro_seconds() == 0.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_intro_seconds_honors_long_intro_style(self):
|
||||
# A long-intro style (e.g. 8s) must be read, not assumed to be 5s.
|
||||
orch = self._orch(job_manager=MagicMock())
|
||||
orch.job_manager.get_job.return_value = MagicMock()
|
||||
fake_style = MagicMock()
|
||||
fake_style.get_intro_format.return_value = {"video_duration": 8}
|
||||
|
||||
async def fake_load(*a, **k):
|
||||
return fake_style
|
||||
|
||||
with patch("backend.workers.style_helper.load_style_config", side_effect=fake_load):
|
||||
assert await orch._resolve_intro_seconds() == 8.0
|
||||
@@ -871,6 +871,13 @@ class VideoWorkerOrchestrator:
|
||||
'final_karaoke_cdg_zip': self.result.final_karaoke_cdg_zip,
|
||||
}
|
||||
|
||||
# Emit the padded original-vocals guide for public NOMAD releases so kjbox can
|
||||
# layer it under the master at the "Original Vocals" slider. Best-effort: a
|
||||
# failure just omits the guide (the upload layer pushes it if present).
|
||||
guide_path = await self._emit_original_vocals_guide(brand_code)
|
||||
if guide_path:
|
||||
output_files['original_vocals_guide'] = guide_path
|
||||
|
||||
uploaded = gdrive.upload_to_public_share(
|
||||
root_folder_id=self.config.gdrive_folder_id,
|
||||
brand_code=brand_code,
|
||||
@@ -887,6 +894,78 @@ class VideoWorkerOrchestrator:
|
||||
self.result.distribution_warnings.append(f"Google Drive upload failed: {e}")
|
||||
# Don't fail the pipeline - GDrive is optional
|
||||
|
||||
async def _emit_original_vocals_guide(self, brand_code) -> Optional[str]:
|
||||
"""Build the padded original-vocals guide (silence[intro] + mixed_vocals, capped to
|
||||
the master's duration) for a public NOMAD release, and return its local path.
|
||||
|
||||
The mixed-vocals stem is a byproduct of the karaoke separation (uploaded to
|
||||
``jobs/{job_id}/stems/vocals_clean.flac``) and the intro offset is the job's own
|
||||
style ``intro_video_duration`` — so no correlation/review is needed. Best-effort:
|
||||
returns ``None`` (and never raises) for non-NOMAD brands, a missing stem, or any
|
||||
build failure — the guide simply won't be distributed.
|
||||
"""
|
||||
try:
|
||||
from backend.services.nomad_master_mirror import is_nomad_public_brand
|
||||
if not is_nomad_public_brand(brand_code):
|
||||
return None
|
||||
master_720p = self.result.final_video_720p
|
||||
if not master_720p or not os.path.exists(master_720p):
|
||||
return None
|
||||
if not self.storage:
|
||||
return None
|
||||
|
||||
import tempfile
|
||||
from backend.services.original_vocals_guide import (
|
||||
build_original_vocals_guide,
|
||||
probe_duration,
|
||||
)
|
||||
|
||||
work_dir = tempfile.mkdtemp(prefix="vocals_guide_")
|
||||
stem_gcs = f"jobs/{self.config.job_id}/stems/vocals_clean.flac"
|
||||
local_stem = os.path.join(work_dir, "vocals_clean.flac")
|
||||
try:
|
||||
self.storage.download_file(stem_gcs, local_stem)
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.job_log.warning(
|
||||
f"Original-vocals guide: mixed-vocals stem unavailable ({stem_gcs}): {e}"
|
||||
)
|
||||
return None
|
||||
|
||||
intro_seconds = await self._resolve_intro_seconds()
|
||||
master_duration = probe_duration(master_720p)
|
||||
dest = os.path.join(work_dir, "original_vocals_guide.flac")
|
||||
guide = build_original_vocals_guide(
|
||||
local_stem, intro_seconds, dest, master_duration=master_duration
|
||||
)
|
||||
if guide:
|
||||
self.job_log.info(
|
||||
f"Built original-vocals guide for {brand_code} (intro={intro_seconds}s)"
|
||||
)
|
||||
return guide
|
||||
except Exception as e: # noqa: BLE001 - never fatal to the pipeline
|
||||
self.job_log.warning(f"Original-vocals guide emit skipped: {e}")
|
||||
return None
|
||||
|
||||
async def _resolve_intro_seconds(self) -> float:
|
||||
"""The master's silent title-card length = the job's style intro ``video_duration``
|
||||
(default 5s). Read the *actual* value — long-intro styles exist. Falls back to 5s
|
||||
(``get_intro_format`` raises when no theme is loaded — that's the fallback path)."""
|
||||
try:
|
||||
from backend.workers.style_helper import load_style_config
|
||||
import tempfile
|
||||
|
||||
job = self.job_manager.get_job(self.config.job_id) if self.job_manager else None
|
||||
if job is None:
|
||||
return 5.0
|
||||
style = await load_style_config(job, self.storage, tempfile.mkdtemp(prefix="style_"))
|
||||
# Preserve an intentional 0 (a style with no title card); only None/absent
|
||||
# falls back to the 5s default. A truthiness check would misalign the guide.
|
||||
intro = style.get_intro_format().get("video_duration", 5)
|
||||
return float(intro) if intro is not None else 5.0
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.job_log.warning(f"Original-vocals guide: intro-duration lookup fell back to 5s: {e}")
|
||||
return 5.0
|
||||
|
||||
async def _run_notifications(self):
|
||||
"""Run the notifications stage (Discord)."""
|
||||
self.job_log.info("Starting notifications stage")
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
# Plan: Original-Vocals guide write-path
|
||||
|
||||
**Created:** 2026-07-09
|
||||
**Branch:** feat/sess-20260709-0013-vocals-writepath
|
||||
**Spec:** `docs/superpowers/specs/2026-07-09-original-vocals-writepath-design.md`
|
||||
**Status:** In progress (autonomous)
|
||||
|
||||
## Verified code map (server render flow)
|
||||
|
||||
| Fact | Location |
|
||||
|--|--|
|
||||
| `mixed_vocals` stem uploaded to GCS | `backend/workers/audio_worker.py:510` → `jobs/{job_id}/stems/vocals_clean.flac` |
|
||||
| Finalize/upload hook (has brand_code, base_name, local 720p) | `backend/workers/video_worker_orchestrator.py:852 _upload_to_gdrive` |
|
||||
| 720p master (local at finalize) | `self.result.final_video_720p` (orch:870) |
|
||||
| Existing 720p GCS push seam | `backend/services/gdrive_service.py:315-336` (`_fast_sync_nomad_master` → `NomadMasterMirror.push_720p`) |
|
||||
| Intro seconds from style | `karaoke_gen/style_loader.py:688 get_video_durations(style_params)` → intro (default 5) |
|
||||
| Style load | `backend/workers/style_helper.py StyleHelper(job, storage, temp_dir).load()` (orch has `self.storage`, `self.job_manager`) |
|
||||
| GCS helpers | `backend/services/storage_service.py` (`download_file`, `delete`) |
|
||||
| Brand-recycle / master cleanup seam | `backend/services/nomad_master_mirror.py cleanup_nomad_masters()` |
|
||||
| Settings | `backend/config.py` (`divebar_files_bucket`, `nomad_master_gcs_prefix`, `nomad_master_fast_sync_enabled`, `gcs_bucket_name`) |
|
||||
|
||||
## Implementation steps (karaoke-gen)
|
||||
|
||||
1. [ ] `backend/config.py`: add `nomad_vocals_guide_gcs_prefix` (default `"files/Nomad Karaoke/vocals-padded"`).
|
||||
2. [ ] `backend/services/original_vocals_guide.py` (NEW, pure + testable):
|
||||
`build_original_vocals_guide(mixed_vocals_path, intro_seconds, master_duration, dest_path, ffmpeg_path="ffmpeg") -> str | None`
|
||||
— ffmpeg `-i vocals -af "adelay={ms}:all=1" -t {master_duration} -c:a flac -f flac {dest}.part` → atomic replace. Returns dest or None (missing input / ffmpeg fail → None, never raises).
|
||||
3. [ ] `backend/services/nomad_master_mirror.py`: add `push_vocals_guide(local_path, filename)` + `delete_vocals_guides_by_brand(brand_code)` (siblings of the 720p methods, keyed on `nomad_vocals_guide_gcs_prefix`). Extend `cleanup_nomad_masters` (or add sibling call) to also delete guides.
|
||||
4. [ ] `video_worker_orchestrator.py _upload_to_gdrive`: before building `output_files`, call new `await self._emit_original_vocals_guide(brand_code, safe_base_name)` which (only for `is_nomad_public_brand`): loads job+style→intro secs, downloads `vocals_clean.flac`, ffprobes 720p→master_dur, builds guide named `{brand} - {base}.flac`, returns path. Register as `output_files["original_vocals_guide"]`. Best-effort; failures → `self.result.distribution_warnings`.
|
||||
5. [ ] `gdrive_service.py _upload_public_share_files`: after the 720p fast-sync block, push `output_files["original_vocals_guide"]` via a new `_fast_sync_vocals_guide(brand_code, path, filename)` (gated `is_nomad_public_brand` + `nomad_master_fast_sync_enabled`, best-effort, warning on failure).
|
||||
6. [ ] Tests: `test_original_vocals_guide.py` (builder: offset/cap/`-f flac`/None paths), extend `test_nomad_master_mirror.py` (push/delete vocals prefix + gating), wiring test (guide registered + pushed + warned).
|
||||
7. [ ] Version bump `pyproject.toml`.
|
||||
|
||||
## Implementation steps (kjbox — separate worktree, PR only)
|
||||
|
||||
8. [ ] `kj-controller/config.py`: add `vocals_sync_enabled` (False), `vocals_sync_source` (`gs://nomadkaraoke-divebar-files/files/Nomad Karaoke/vocals-padded/`), `vocals_sync_dest` (""→`{download_folder}/NOMAD-vocals-padded`), `vocals_sync_delete_removed` (False).
|
||||
9. [ ] `kj-controller/scripts/sync_masters.py main()`: after master `run_sync`, build a vocals config view (map `vocals_sync_*`→`master_sync_*` keys, `delete_removed=False`, no rescan poke) and call `run_sync` again when `vocals_sync_enabled`. `run_sync` unchanged.
|
||||
10. [ ] Tests: extend `test_sync_masters.py` — both syncs run, vocals additive-only, isolation on vocals failure.
|
||||
|
||||
## Testing
|
||||
- karaoke-gen: `make test` (or targeted pytest for new files) — unit for builder (synthetic tone+silence), mirror (mocked storage), wiring.
|
||||
- kjbox: `cd kj-controller && pytest tests/unit/test_sync_masters.py`.
|
||||
|
||||
## Release
|
||||
- karaoke-gen: full ship to `main` (inert change).
|
||||
- kjbox: PR only; **not merged/deployed** (live device — Andrew's maintenance window).
|
||||
|
||||
## Rollback
|
||||
- karaoke-gen: revert PR; `nomad_master_fast_sync_enabled=false` also disables guide push.
|
||||
- kjbox: `vocals_sync_enabled=false` default → inert; not deployed.
|
||||
@@ -0,0 +1,136 @@
|
||||
# Original-Vocals Guide write-path — Design Spec
|
||||
|
||||
**Status:** Approved (autonomous session 2026-07-09; brainstormed with Andrew before handoff)
|
||||
**Repos:** karaoke-gen (emit + push + cleanup) · kjbox (device pull)
|
||||
**Related:** kjbox `docs/ORIGINAL-VOCALS.md`, `docs/archive/2026-07-06-original-vocals-guide-design.md`; workspace `docs/archive/2026-07-09-karaoke-gen-vocals-writepath-handoff.md`
|
||||
|
||||
## Goal
|
||||
|
||||
Make the "Original Vocals" sing-along guide **self-maintaining**. When karaoke-gen
|
||||
finalises a new public NOMAD master, it automatically emits an aligned original-vocals
|
||||
guide and pushes it to GCS; the kjbox device pulls it into `NOMAD-vocals-padded/`, so the
|
||||
feature "just works" for new tracks with no re-run of the kjbox retro-fit pipeline.
|
||||
|
||||
## Key insight (why this is simpler than the retro-fit)
|
||||
|
||||
At render time karaoke-gen **already knows everything** the retro-fit had to *measure*:
|
||||
|
||||
- The **isolated vocals stem** is a byproduct of making the karaoke instrumental
|
||||
(`karaoke_gen/audio_processor.py` separation → `mixed_vocals`).
|
||||
- The **exact intro offset** is the job's own style value — the master is
|
||||
`[silent title-card intro] + song + [tail]`, and the intro length is
|
||||
`intro_video_duration` from the job's resolved style (default 5s, but read the
|
||||
*actual* value, never hardcode 5).
|
||||
|
||||
So the guide is simply `silence[intro] + mixed_vocals`, capped to the master's duration.
|
||||
**No correlation, no human review, no `align_offsets`/`align_decisions`, no "exclude"** —
|
||||
the vocals ARE the true original and the alignment is known, so every new public NOMAD
|
||||
render gets a valid guide for free. This is byte-compatible with what the retro-fit
|
||||
`pad_vocals.sh` produced (`adelay` + flac), but built for free instead of measured.
|
||||
|
||||
## Decisions
|
||||
|
||||
| # | Decision | Choice | Rationale |
|
||||
|--|--|--|--|
|
||||
| Scope | One spec, both repos | ✅ | Agree the GCS prefix/naming contract once. |
|
||||
| Vocals stem | `mixed_vocals` (lead + backing) | ✅ | Always produced (Stage 1); matches retro-fit "main vocal, backing left in"; fullest sing-along reference. |
|
||||
| Emit shape | Padded (`silence[intro]+vocals`), not raw | ✅ | Zero device compute; offset is known at render. |
|
||||
| Offset | Job's actual `intro_video_duration` | ✅ | Not hardcoded 5s — long-intro styles exist. |
|
||||
| Length | Cap to 720p master duration (`-t`) | ✅ | Defensive: guide can never bleed past the master. |
|
||||
| GCS layout | New sibling prefix `files/Nomad Karaoke/vocals-padded/` | ✅ | Parallel to masters `MP4-720p/`; device maps it to `NOMAD-vocals-padded/`. |
|
||||
| Device pull | Extend existing 5-min timer to `run_sync` twice | ✅ | `run_sync` unchanged (never raises → can't touch masters mirror); no new systemd unit on the live box. |
|
||||
| Sync mode | **Additive-only** (`vocals_sync_delete_removed=False`) | ✅ | The device already holds 1,459 retro-fit guides absent from the (initially near-empty) GCS prefix; a reconcile would try to delete them. Additive-only removes that risk entirely. |
|
||||
| Privates | Gate on `is_nomad_public_brand` | ✅ | Skip `NOMADNP`. |
|
||||
| Cleanup | Delete GCS guide on brand recycle | ✅ | Parallel to `cleanup_nomad_masters`. |
|
||||
|
||||
### Known limitation (documented, out of scope)
|
||||
|
||||
With additive-only sync, a **recycled** public NOMAD brand code (re-rendered as a
|
||||
different song) leaves its **old** guide on the device. The play-time resolver globs
|
||||
`NOMAD-#### - *` and takes `sorted()[0]`, so it could serve the stale vocal under the new
|
||||
song. Public-NOMAD brand recycling is rare, so this is accepted for now. Future fix:
|
||||
one-off backfill of the 1,459 existing device guides → the GCS prefix, then flip the
|
||||
vocals sync to reconcile-enabled so deletes propagate cleanly. Until then, GCS-side
|
||||
cleanup (`delete_vocals_guides_by_brand`) keeps GCS correct but won't auto-remove a stale
|
||||
device copy.
|
||||
|
||||
## Architecture / data flow
|
||||
|
||||
```
|
||||
karaoke-gen finalize (server flow, backend/services/gdrive_service.py)
|
||||
has: mixed_vocals stem + job intro_video_duration + freshly built 720p master
|
||||
├─ build_original_vocals_guide(): ffmpeg -af "adelay={intro_ms}:all=1" -t {master_dur}
|
||||
│ -c:a flac -f flac → "{brand} - {base_name}.flac" (atomic .part rename)
|
||||
│ registered as output_files["original_vocals_guide"]
|
||||
└─ _upload_public_share_files():
|
||||
push_720p() → gs://…/MP4-720p/{brand} - name.mp4 (existing)
|
||||
push_vocals_guide() → gs://…/vocals-padded/{brand} - name.flac (NEW, best-effort)
|
||||
|
||||
kjbox device (existing 5-min timer: kj-controller/scripts/sync_masters.py :: main())
|
||||
├─ run_sync(master config) → NOMAD-720p/ (existing, unchanged)
|
||||
└─ run_sync(vocals config) → NOMAD-vocals-padded/ (NEW, additive-only, no /rescan poke)
|
||||
|
||||
play time (unchanged): routes.py::_resolve_vocals_guide() globs "NOMAD-#### - *"
|
||||
in NOMAD-vocals-padded/ → "Original Vocals" slider works
|
||||
```
|
||||
|
||||
## Components & interfaces
|
||||
|
||||
### karaoke-gen
|
||||
|
||||
**`build_original_vocals_guide(mixed_vocals_path, intro_seconds, master_duration, dest_path, ffmpeg=...) -> str | None`**
|
||||
(new helper; lives next to the audio/finalize code that has the separation result + style)
|
||||
- ffmpeg: `-i mixed_vocals -af "adelay={intro_seconds*1000}:all=1" -t {master_duration} -c:a flac -f flac {dest}.part`, then atomic `os.replace` to `{dest}`.
|
||||
- `-f flac` is mandatory (the `.part` extension would otherwise confuse the muxer — carried-over gotcha).
|
||||
- Returns dest path on success, `None` on any failure (best-effort; never raises to the pipeline). Missing inputs → `None`.
|
||||
- Emits `{brand_code} - {safe_base_name}.flac` (mirrors master naming so the device's brand-prefix glob resolves it).
|
||||
|
||||
**`NomadMasterMirror.push_vocals_guide(local_path, filename) -> bool`** (sibling of `push_720p`)
|
||||
- Uploads to `gs://{divebar_files_bucket}/{nomad_vocals_guide_gcs_prefix}/{filename}`.
|
||||
- New setting `nomad_vocals_guide_gcs_prefix` (default `"files/Nomad Karaoke/vocals-padded"`).
|
||||
- Best-effort; returns success.
|
||||
|
||||
**`NomadMasterMirror.delete_vocals_guides_by_brand(brand_code) -> int`** (sibling of `delete_masters_by_brand`)
|
||||
- Deletes `{vocals_prefix}/{brand_code} - *` (requires full `NOMAD-####`, refuses bare/non-Nomad — same guardrail as the master delete).
|
||||
|
||||
**Wiring:**
|
||||
- `_fast_sync_nomad_master`-adjacent (or a new `_fast_sync_vocals_guide`) in `gdrive_service.py` calls `push_vocals_guide` right after the 720p push, gated by `nomad_master_fast_sync_enabled` + `is_nomad_public_brand`, surfacing failure as a distribution warning.
|
||||
- `cleanup_nomad_masters(brand_code)` gains a sibling call (or is extended) to also delete guides, so every brand-recycle / job-delete path that already cleans masters cleans guides too.
|
||||
|
||||
### kjbox
|
||||
|
||||
**`sync_masters.py :: main()`** — after the existing master `run_sync`, call `run_sync` again with a vocals config view:
|
||||
- `source = config["vocals_sync_source"]` (default `gs://nomadkaraoke-divebar-files/files/Nomad Karaoke/vocals-padded/`)
|
||||
- `dest = config["vocals_sync_dest"]` or derived `{download_folder}/NOMAD-vocals-padded`
|
||||
- `vocals_sync_enabled` (default False until the device is configured), `vocals_sync_delete_removed=False` (additive-only), and skip the `/rescan` poke (guides are glob-resolved at play time, never indexed).
|
||||
- `run_sync` itself is **unchanged**; the vocals invocation passes a config dict with the `master_sync_*` keys populated from the `vocals_sync_*` values (thin adapter in `main()`), so the isolation/never-raise contract is preserved verbatim.
|
||||
- New config keys added to `config.py` defaults.
|
||||
|
||||
## Error handling & safety
|
||||
|
||||
- **Never fatal.** Emit + push are best-effort; the Drive upload and render already succeeded. A guide failure = a `distribution_warnings` entry (admin alert), nothing more. The nightly Drive→GCS VM remains the masters backfill net (guides are new-renders-only; no VM path yet, acceptable).
|
||||
- **Masters mirror untouched.** kjbox `run_sync` is not modified; a vocals-sync error returns an error dict and cannot break the masters sync in the same `main()` run.
|
||||
- **Additive-only** protects the existing device guides (see Decisions).
|
||||
- **Gating** on `is_nomad_public_brand` keeps `NOMADNP` privates out of the public GCS mirror.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
**karaoke-gen (pytest):**
|
||||
- Unit `build_original_vocals_guide`: synthetic tone+silence input → assert output offset (leading silence ≈ intro), duration capped to master, flac muxed via `-f flac`; missing input → `None`; ffmpeg failure → `None` (no raise).
|
||||
- Unit `push_vocals_guide` / `delete_vocals_guides_by_brand`: mocked `storage.Client` — correct blob prefix, brand gating, best-effort return, bare-brand refusal. Mirror the existing `nomad_master_mirror` tests.
|
||||
- Unit wiring: guide push is gated + surfaced as a warning like the master push; non-Nomad / disabled → no-op.
|
||||
|
||||
**kjbox (pytest, `test_sync_masters.py`):**
|
||||
- `main()` invokes `run_sync` for both master and vocals configs.
|
||||
- Vocals invocation is additive-only (`delete_removed=False`) and skips `/rescan`.
|
||||
- A vocals `run_sync` failure does not fail the masters sync (isolation).
|
||||
|
||||
## Rollback
|
||||
|
||||
- karaoke-gen: revert the PR; the new GCS prefix simply stops being written. Nothing consumes it, so no user-facing effect either way. `nomad_master_fast_sync_enabled=false` also disables the guide push (shared gate).
|
||||
- kjbox: `vocals_sync_enabled=false` (default) makes the device change inert; not merged/deployed autonomously — left for a maintenance window.
|
||||
|
||||
## Release plan
|
||||
|
||||
- **karaoke-gen:** full ship to `main` (main-based; Cloud Run deploys on merge). Change is inert until kjbox lands, so safe.
|
||||
- **kjbox:** implement + test + PR only. **Not merged/deployed** in this session — live production device; merge+deploy is Andrew's call during a maintenance window (kjbox `CLAUDE.md`).
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "karaoke-gen"
|
||||
version = "0.190.4"
|
||||
version = "0.190.5"
|
||||
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
|
||||
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
|
||||
license = "MIT"
|
||||
|
||||
Reference in new issue
Block a user