## Summary Fixes the deployment failure from PR #297 by making lyrics_transcriber a proper subpackage of karaoke_gen instead of a separate vendored library. ### Changes - Move `libs/lyrics_transcriber/` to `karaoke_gen/lyrics_transcriber/` - Update all imports from `from lyrics_transcriber.X` to `from karaoke_gen.lyrics_transcriber.X` - Simplify Dockerfile (no separate `libs/` COPY needed) - Update pyproject.toml to remove separate package entry - Update test mocks and logger names ### Why this is better - Cleaner project structure - one package containing everything - Simpler Dockerfile - just COPY karaoke_gen/ - No confusion about vendored vs internal code ## Test plan - [x] `poetry install` succeeds - [x] All imports work correctly - [x] All 2614 tests pass - [ ] Backend deployment succeeds @coderabbitai ignore Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
13 KiB
Countdown Padding Feature
This document explains how the countdown padding feature works across the karaoke-gen system, including both the standalone CLI tool and the cloud backend.
Overview
When a song's vocals start within the first 3 seconds, karaoke singers don't have enough time to prepare. The countdown padding feature addresses this by:
- Adding 3 seconds of silence to the start of the audio
- Shifting all lyrics timestamps forward by 3 seconds
- Adding a "3... 2... 1..." countdown segment (displayed at 0.1s - 2.9s)
This gives singers a visual countdown before the lyrics begin.
Technical Details
CountdownProcessor (lyrics-transcriber)
The core countdown logic lives in the lyrics-transcriber library:
File: lyrics_transcriber/output/countdown_processor.py
Constants:
COUNTDOWN_THRESHOLD_SECONDS = 3.0- Trigger countdown if first word starts within this timeCOUNTDOWN_PADDING_SECONDS = 3.0- Amount of silence to addCOUNTDOWN_START_TIME = 0.1- When countdown text starts displayingCOUNTDOWN_END_TIME = 2.9- When countdown text endsCOUNTDOWN_TEXT = "3... 2... 1..."- The countdown text displayed
Key Methods:
process(correction_result, audio_filepath)- Main entry point. Returns(modified_result, padded_audio_path, padding_added, padding_seconds)_needs_countdown(correction_result)- Checks if first word starts within 3 seconds_create_padded_audio(audio_filepath)- Uses ffmpeg to prepend silence_add_countdown_to_result(correction_result)- Shifts timestamps and adds countdown segmenthas_countdown(correction_result)- Detects if countdown already exists (for loading saved corrections)
FFmpeg Command Used:
ffmpeg -y -hide_banner -loglevel error \
-f lavfi -t 3.0 -i anullsrc=channel_layout=stereo:sample_rate=44100 \
-i input_audio.flac \
-filter_complex "[0:a][1:a]concat=n=2:v=0:a=1[out]" \
-map "[out]" \
-c:a flac \
output_padded.flac
This prepends 3 seconds of stereo silence (44.1kHz) and uses FLAC to preserve audio quality.
Workflow: Standalone CLI (karaoke-gen)
When using the karaoke-gen CLI tool locally:
┌─────────────────────────────────────────────────────────────────────┐
│ STANDALONE CLI WORKFLOW │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ 1. LyricsTranscriber.process() │
│ └─> add_countdown=True (default) │
│ └─> CountdownProcessor adds countdown + pads vocals audio │
│ └─> Returns: padded_audio_filepath, countdown_padding_added │
│ │
│ 2. Audio Separation │
│ └─> Creates instrumental files (original timing) │
│ │
│ 3. Apply Padding to Instrumentals │
│ └─> AudioProcessor.apply_countdown_padding_to_instrumentals() │
│ └─> Pads all instrumental variants to match vocals │
│ └─> Creates files with "(Padded)" suffix │
│ │
│ 4. Final Video Generation │
│ └─> Uses padded vocals + padded instrumental │
│ └─> Everything is synchronized │
│ │
└─────────────────────────────────────────────────────────────────────┘
Key Files:
karaoke_gen/lyrics_processor.py- Calls LyricsTranscriber, captures padding infokaraoke_gen/audio_processor.py-apply_countdown_padding_to_instrumentals()andpad_audio_file()karaoke_gen/karaoke_gen.py- Orchestrates the workflow, applies padding after separation
Instrumental File Naming:
- Original:
Artist - Song (Instrumental model.ckpt).flac - Padded:
Artist - Song (Instrumental model.ckpt) (Padded).flac
Workflow: Cloud Backend
The cloud backend has a two-phase workflow with human review in between. This requires special handling to avoid sync issues.
┌─────────────────────────────────────────────────────────────────────┐
│ CLOUD BACKEND WORKFLOW │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ PHASE 1: Lyrics Worker (lyrics_worker.py) │
│ ───────────────────────────────────────── │
│ └─> LyricsTranscriber.process() │
│ └─> add_countdown=FALSE <── Deferred to Phase 2 │
│ └─> Corrections saved WITHOUT countdown │
│ └─> Original audio timing preserved │
│ │
│ HUMAN REVIEW (Review UI) │
│ ──────────────────────── │
│ └─> User reviews/edits lyrics │
│ └─> Audio playback uses original (un-padded) audio │
│ └─> Timestamps match audio perfectly │
│ │
│ PHASE 2: Render Video Worker (render_video_worker.py) │
│ ───────────────────────────────────────────────────── │
│ └─> CountdownProcessor.process() called HERE │
│ └─> Adds countdown segment + shifts timestamps │
│ └─> Creates padded audio for video rendering │
│ └─> Final video has countdown + synced audio │
│ │
└─────────────────────────────────────────────────────────────────────┘
Why Defer Countdown to Phase 2?
Previously, countdown was added during Phase 1 (lyrics transcription). This caused a problem:
- Lyrics corrections were saved with countdown segment + shifted timestamps
- The padded audio file was created in a temp directory that was deleted
- Review UI served the original (un-padded) audio
- Result: 3-second sync mismatch between lyrics and audio in review UI
By deferring countdown to Phase 2:
- Review UI shows original timing (audio and lyrics are perfectly synced)
- Users can accurately review and edit lyrics
- Countdown is added just before final video rendering
Key Files:
karaoke_gen/lyrics_processor.py:405- Setsadd_countdown=not is_serverlessbackend/workers/render_video_worker.py:188-200- Adds countdown usingCountdownProcessor.process()
Serverless Detection:
is_serverless = (
os.getenv("MODAL_TASK_ID") is not None or
os.getenv("MODAL_FUNCTION_NAME") is not None or
os.path.exists("/.modal")
)
Finalization Step (karaoke_finalise)
The KaraokeFinalise class handles creating the final karaoke video with instrumental audio.
File: karaoke_gen/karaoke_finalise/karaoke_finalise.py
It includes a safety net in remux_with_instrumental():
- If
countdown_padding_secondsis set but instrumental doesn't have "(Padded)" in the name - It automatically pads the instrumental before remuxing
- This ensures vocals and instrumental stay synchronized
Detecting Existing Countdown
When loading saved corrections (e.g., from a JSON file), you can detect if countdown was applied:
countdown_processor = CountdownProcessor(cache_dir=temp_dir)
has_countdown = countdown_processor.has_countdown(correction_result)
# Or check for countdown text in first segment
first_segment = correction_result.corrected_segments[0]
is_countdown = first_segment.text == "3... 2... 1..."
You can also detect from LRC files:
# LyricsProcessor._detect_countdown_padding_from_lrc()
# Checks for "3... 2... 1..." text or first timestamp >= 2.5s
Configuration
Disabling Countdown
To disable countdown for a specific transcription:
from karaoke_gen.lyrics_transcriber.core.config import OutputConfig
output_config = OutputConfig(
# ... other options
add_countdown=False, # Disable countdown
)
Padding Amount
The padding amount is currently fixed at 3.0 seconds. To change this, you would need to modify the constants in CountdownProcessor.
Troubleshooting
Issue: Countdown appears but audio is out of sync
Cause: Instrumental audio wasn't padded to match vocals.
Solution:
- Check that
countdown_padding_addedwas captured from transcription - Verify
apply_countdown_padding_to_instrumentals()was called - Look for "(Padded)" in instrumental filenames
Issue: Review UI shows 3-second offset
Cause: Countdown was added during transcription instead of rendering phase.
Solution:
- This was fixed in v0.76.1
- Ensure
add_countdown=Falsefor serverless mode - Countdown should only be added in
render_video_worker.py
Issue: No countdown even though song starts immediately
Cause: First word timestamp might be exactly at or after 3.0 seconds.
Check:
- Look at the first segment's first word start_time
- Countdown triggers only if
first_word_start < 3.0
Testing
Unit Tests
Countdown padding tests are in tests/unit/test_countdown_padding.py:
python -m pytest tests/unit/test_countdown_padding.py -v
Tests cover:
pad_audio_file()success, failure, and timeoutapply_countdown_padding_to_instrumentals()for clean and combined instrumentals- Integration with
LyricsProcessorandKaraokeGen - Custom instrumental padding
- Idempotency (skipping already-padded files)
Data Flow Summary
┌─────────────────────────────────────────────────────────────────────┐
│ DATA FLOW │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ Input: Song with vocals starting at 1.5s │
│ │
│ After Countdown Processing: │
│ ───────────────────────────── │
│ • Audio: 3s silence + original audio │
│ • First segment: "3... 2... 1..." at 0.1s-2.9s │
│ • Original first word: moved from 1.5s → 4.5s │
│ • All subsequent timestamps: shifted +3.0s │
│ │
│ Result: Singer sees countdown, has time to prepare │
│ │
└─────────────────────────────────────────────────────────────────────┘
Version History
- v0.76.1 - Fixed cloud backend sync issue by deferring countdown to render phase
- v0.70.x - Initial countdown padding implementation integrated from lyrics-transcriber