fix: reference lyrics truncated at 3000 chars (blank lines counted) (v0.285.2) (#1225)

## Summary
- `BaseLyricsProvider.MAX_LYRICS_LENGTH` (3000) was measured on raw
text. Genius RapidAPI emits `\n\n` per line, so a normal ~4 min song
(job 258ba59b, 4,256 chars) was truncated after verse 2.
Lrclib/Musixmatch hit the same cap (2,975 byte files).
- Blank lines no longer count toward the cap; cap raised to 15000
(screenplays/articles are 50k+).

## Testing
- New regressions: double-spaced 70-line song and ~8k-char rap song keep
every line; screenplay tests still truncate.
- `tests/unit/test_base_lyrics_provider.py` 17 pass. 9 failures in
test_lyrics.py / test_agentic_gemini_provider.py are pre-existing on
main (env).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Lyrics can now contain up to 15,000 characters before truncation, up
from 3,000.
* Blank lines no longer count toward the lyrics character limit, helping
preserve longer lyrics with extra spacing.

* **Documentation**
* Added guidance about the updated lyrics limit and a caution for
rerunning lyrics actions on in-progress jobs, where existing working
data may be replaced.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude Sonnet 5.5 authored and GitHub committed 2026-10-27 15:17:05 -04:00
1 parent cd1fa3feaf
commit 5c30fde9b4
4 files changed
+41 -7

No files matched your search

+12
View File
@@ -2488,3 +2488,15 @@ and unattended audio is chosen only on an exact artist+title match else the owne
Lessons: a "fallback to first item" in an unattended path hides a broken ranker — fail to a
human instead; and a denormalised claim (who owns this) needs releasing wherever the thing that
justified it (the vote) can change.
## Reference-lyrics length cap counted blank lines (2026-10-27, v0.285.2)
`BaseLyricsProvider.MAX_LYRICS_LENGTH` (3000, a guard against screenplays/articles) was measured on
raw text. The Genius RapidAPI path turns every `<br>\n` into `\n\n`, so an ordinary ~4-minute song
measured ~4,300 chars and was silently cut after the second verse (job 258ba59b; Lrclib/Musixmatch
were cut the same way at 2,975 bytes). The only symptom was a WARNING log and a review screen with no
reference for the last verse, so auto-correction quietly did nothing there. Fix: blank lines no longer
count and the cap is 15,000. Lesson: a safety cap must be measured on the same representation as the
threat it guards against, and a cap that truncates silently should be generous or fail loudly. Never
repair an affected in-progress job with `add-lyrics`/`search-lyrics`: the frontend replaces its whole
working data with the server's pre-edit rerun, discarding the reviewer's edits.
@@ -30,8 +30,11 @@ class BaseLyricsProvider(ABC):
# Lyrics text beyond this length are almost certainly not real song lyrics
# (e.g. screenplays, articles, or other non-lyrics content from web scraping).
# Typical songs are 1000-4000 chars; 3000 is a safe truncation point.
MAX_LYRICS_LENGTH = 3000
# Measured on the non-blank lines only (some providers double-space every line,
# which used to push ordinary 4-minute songs over the old 3000 limit and silently
# drop their last verses). Long rap songs reach ~10k chars; screenplays and
# articles are 50k+, so 15000 still catches them.
MAX_LYRICS_LENGTH = 15000
def __init__(self, config: LyricsProviderConfig, logger: Optional[logging.Logger] = None):
self.logger = logger or logging.getLogger(__name__)
@@ -129,7 +132,9 @@ class BaseLyricsProvider(ABC):
Returns:
List of LyricsSegment objects with unique IDs and Word objects
"""
# Guard against abnormally long text (screenplays, articles, etc.)
# Guard against abnormally long text (screenplays, articles, etc.).
# Blank lines are dropped below anyway, so don't count them toward the limit.
text = "\n".join(line for line in text.split("\n") if line.strip())
if len(text) > self.MAX_LYRICS_LENGTH:
self.logger.warning(
f"{self.get_name()}: Lyrics text is {len(text):,} chars, truncating to ~{self.MAX_LYRICS_LENGTH:,} "
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.285.1"
version = "0.285.2"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
+20 -3
View File
@@ -45,7 +45,7 @@ class TestCreateSegmentsWithWords:
def test_long_text_truncated(self):
provider = self._make_provider()
# Create text well over the limit (simulating a screenplay)
lines = [f"Line number {i} of the screenplay" for i in range(500)]
lines = [f"Line number {i} of the screenplay" for i in range(2000)]
text = "\n".join(lines)
assert len(text) > BaseLyricsProvider.MAX_LYRICS_LENGTH
@@ -54,9 +54,26 @@ class TestCreateSegmentsWithWords:
total_text = "\n".join(s.text for s in segments)
assert len(total_text) <= BaseLyricsProvider.MAX_LYRICS_LENGTH
def test_double_spaced_song_not_truncated(self):
"""Regression: Genius RapidAPI doubles every newline; a real ~4k-char song lost its last verse."""
provider = self._make_provider()
lines = [f"This is lyric line number {i} of a real song" for i in range(70)]
text = "\n\n".join(lines)
assert len(text) > 3000
segments = provider._create_segments_with_words(text)
assert len(segments) == 70
assert segments[-1].text == lines[-1]
def test_long_rap_song_not_truncated(self):
provider = self._make_provider()
lines = [f"Verse line {i} with some rap words in it ok" for i in range(200)]
text = "\n".join(lines)
assert 8000 < len(text) < BaseLyricsProvider.MAX_LYRICS_LENGTH
assert len(provider._create_segments_with_words(text)) == 200
def test_truncation_preserves_complete_lines(self):
provider = self._make_provider()
lines = [f"Line {i}: some lyrics text here" for i in range(200)]
lines = [f"Line {i}: some lyrics text here" for i in range(1000)]
text = "\n".join(lines)
segments = provider._create_segments_with_words(text)
@@ -66,7 +83,7 @@ class TestCreateSegmentsWithWords:
def test_truncation_logs_warning(self, caplog):
provider = self._make_provider()
text = "x\n" * 5000 # Way over limit
text = "x\n" * 20000 # Way over limit
with caplog.at_level(logging.WARNING):
provider._create_segments_with_words(text)
assert "truncating" in caplog.text.lower()