mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
fix: reference lyrics truncated at 3000 chars (blank lines counted) (v0.285.2) (#1225)
## Summary - `BaseLyricsProvider.MAX_LYRICS_LENGTH` (3000) was measured on raw text. Genius RapidAPI emits `\n\n` per line, so a normal ~4 min song (job 258ba59b, 4,256 chars) was truncated after verse 2. Lrclib/Musixmatch hit the same cap (2,975 byte files). - Blank lines no longer count toward the cap; cap raised to 15000 (screenplays/articles are 50k+). ## Testing - New regressions: double-spaced 70-line song and ~8k-char rap song keep every line; screenplay tests still truncate. - `tests/unit/test_base_lyrics_provider.py` 17 pass. 9 failures in test_lyrics.py / test_agentic_gemini_provider.py are pre-existing on main (env). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * Lyrics can now contain up to 15,000 characters before truncation, up from 3,000. * Blank lines no longer count toward the lyrics character limit, helping preserve longer lyrics with extra spacing. * **Documentation** * Added guidance about the updated lyrics limit and a caution for rerunning lyrics actions on in-progress jobs, where existing working data may be replaced. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
cd1fa3feaf
commit
5c30fde9b4
4 files changed
+41
-7
No files matched your search
@@ -2488,3 +2488,15 @@ and unattended audio is chosen only on an exact artist+title match else the owne
|
||||
Lessons: a "fallback to first item" in an unattended path hides a broken ranker — fail to a
|
||||
human instead; and a denormalised claim (who owns this) needs releasing wherever the thing that
|
||||
justified it (the vote) can change.
|
||||
|
||||
## Reference-lyrics length cap counted blank lines (2026-10-27, v0.285.2)
|
||||
|
||||
`BaseLyricsProvider.MAX_LYRICS_LENGTH` (3000, a guard against screenplays/articles) was measured on
|
||||
raw text. The Genius RapidAPI path turns every `<br>\n` into `\n\n`, so an ordinary ~4-minute song
|
||||
measured ~4,300 chars and was silently cut after the second verse (job 258ba59b; Lrclib/Musixmatch
|
||||
were cut the same way at 2,975 bytes). The only symptom was a WARNING log and a review screen with no
|
||||
reference for the last verse, so auto-correction quietly did nothing there. Fix: blank lines no longer
|
||||
count and the cap is 15,000. Lesson: a safety cap must be measured on the same representation as the
|
||||
threat it guards against, and a cap that truncates silently should be generous or fail loudly. Never
|
||||
repair an affected in-progress job with `add-lyrics`/`search-lyrics`: the frontend replaces its whole
|
||||
working data with the server's pre-edit rerun, discarding the reviewer's edits.
|
||||
@@ -30,8 +30,11 @@ class BaseLyricsProvider(ABC):
|
||||
|
||||
# Lyrics text beyond this length are almost certainly not real song lyrics
|
||||
# (e.g. screenplays, articles, or other non-lyrics content from web scraping).
|
||||
# Typical songs are 1000-4000 chars; 3000 is a safe truncation point.
|
||||
MAX_LYRICS_LENGTH = 3000
|
||||
# Measured on the non-blank lines only (some providers double-space every line,
|
||||
# which used to push ordinary 4-minute songs over the old 3000 limit and silently
|
||||
# drop their last verses). Long rap songs reach ~10k chars; screenplays and
|
||||
# articles are 50k+, so 15000 still catches them.
|
||||
MAX_LYRICS_LENGTH = 15000
|
||||
|
||||
def __init__(self, config: LyricsProviderConfig, logger: Optional[logging.Logger] = None):
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
@@ -129,7 +132,9 @@ class BaseLyricsProvider(ABC):
|
||||
Returns:
|
||||
List of LyricsSegment objects with unique IDs and Word objects
|
||||
"""
|
||||
# Guard against abnormally long text (screenplays, articles, etc.)
|
||||
# Guard against abnormally long text (screenplays, articles, etc.).
|
||||
# Blank lines are dropped below anyway, so don't count them toward the limit.
|
||||
text = "\n".join(line for line in text.split("\n") if line.strip())
|
||||
if len(text) > self.MAX_LYRICS_LENGTH:
|
||||
self.logger.warning(
|
||||
f"{self.get_name()}: Lyrics text is {len(text):,} chars, truncating to ~{self.MAX_LYRICS_LENGTH:,} "
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "karaoke-gen"
|
||||
version = "0.285.1"
|
||||
version = "0.285.2"
|
||||
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
|
||||
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
|
||||
license = "MIT"
|
||||
|
||||
@@ -45,7 +45,7 @@ class TestCreateSegmentsWithWords:
|
||||
def test_long_text_truncated(self):
|
||||
provider = self._make_provider()
|
||||
# Create text well over the limit (simulating a screenplay)
|
||||
lines = [f"Line number {i} of the screenplay" for i in range(500)]
|
||||
lines = [f"Line number {i} of the screenplay" for i in range(2000)]
|
||||
text = "\n".join(lines)
|
||||
assert len(text) > BaseLyricsProvider.MAX_LYRICS_LENGTH
|
||||
|
||||
@@ -54,9 +54,26 @@ class TestCreateSegmentsWithWords:
|
||||
total_text = "\n".join(s.text for s in segments)
|
||||
assert len(total_text) <= BaseLyricsProvider.MAX_LYRICS_LENGTH
|
||||
|
||||
def test_double_spaced_song_not_truncated(self):
|
||||
"""Regression: Genius RapidAPI doubles every newline; a real ~4k-char song lost its last verse."""
|
||||
provider = self._make_provider()
|
||||
lines = [f"This is lyric line number {i} of a real song" for i in range(70)]
|
||||
text = "\n\n".join(lines)
|
||||
assert len(text) > 3000
|
||||
segments = provider._create_segments_with_words(text)
|
||||
assert len(segments) == 70
|
||||
assert segments[-1].text == lines[-1]
|
||||
|
||||
def test_long_rap_song_not_truncated(self):
|
||||
provider = self._make_provider()
|
||||
lines = [f"Verse line {i} with some rap words in it ok" for i in range(200)]
|
||||
text = "\n".join(lines)
|
||||
assert 8000 < len(text) < BaseLyricsProvider.MAX_LYRICS_LENGTH
|
||||
assert len(provider._create_segments_with_words(text)) == 200
|
||||
|
||||
def test_truncation_preserves_complete_lines(self):
|
||||
provider = self._make_provider()
|
||||
lines = [f"Line {i}: some lyrics text here" for i in range(200)]
|
||||
lines = [f"Line {i}: some lyrics text here" for i in range(1000)]
|
||||
text = "\n".join(lines)
|
||||
|
||||
segments = provider._create_segments_with_words(text)
|
||||
@@ -66,7 +83,7 @@ class TestCreateSegmentsWithWords:
|
||||
|
||||
def test_truncation_logs_warning(self, caplog):
|
||||
provider = self._make_provider()
|
||||
text = "x\n" * 5000 # Way over limit
|
||||
text = "x\n" * 20000 # Way over limit
|
||||
with caplog.at_level(logging.WARNING):
|
||||
provider._create_segments_with_words(text)
|
||||
assert "truncating" in caplog.text.lower()
|
||||
|
||||
Reference in new issue
Block a user