mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
fix(ass): split lyric lines wider than the frame (overlapping lyrics, NOMAD-1779) v0.295.1 (#1263)
## Summary NOMAD-1779 (job `58555663`) shipped with two lyric lines drawn on top of each other near the end of the song (YouTube comment: "the lyrics get kinda messed up towards the end"). **Cause:** lines are split by character count (`max_line_length`, 40 for Nomad). Wide-glyph lines like "Mm- mm- mm, mm- …" pass that but measured ~3890px on the 3840px frame. libass wraps an over-wide event by itself; rows are exactly one line-height apart, so the wrapped half landed on the next row's lyrics. **Fix:** `SubtitlesGenerator.generate_ass` now measures each line in the style font, scaled the way libass sizes it (existing `LyricsLine._measure_text` / `ass_font_scale`), and splits any line wider than 94% of the frame at a word boundary (`SegmentResizer.split_to_width`, prefers phrase-ending punctuation, keeps word timing, shares translations). Applies to every ASS render (main, portrait, theme previews). Lines that fit are untouched. **Scope:** of 309 completed branded jobs since 2026-09-10, only 1779 had an overlap; 4 more (1759, 1738, 1766, NOMADNP-0264) had a line wrapping into an empty row. ## Testing - New `tests/unit/lyrics_transcriber/output/test_line_width_split.py`: split logic (fits / splits / phrase break / balance / single long word / timing / translation / singer), ASS generation, and a **real libass render** asserting each lyric event draws as exactly one row. Verified the render test fails with the fix disabled. - Regenerated 1779's ASS from its `corrections_updated.json` with the real theme font and rendered 2:58 with ffmpeg/libass: old ASS reproduces the overlap exactly, new one renders 4 clean lines (widest 3502px). - Locally: 1968 unit + 7231 backend tests pass. Failures were environmental only (shared conda env lacks `flacfetch`: 6 unit + 2 collection errors; `test_emulator_integration.py` fails identically on `origin/main` locally). - CodeRabbit CLI (local, committed): no findings. After deploy: admin re-render of 58555663. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012RYLVW1AnhNWu2qGLaNvxu --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d98e0d6a5e
commit
da204bd52e
4 files changed
+250
-1
No files matched your search
@@ -2540,3 +2540,23 @@ exactly like the old path. Kill switch: `AUTO_APPROVAL_AUDIO_HOLD_ENABLED`.
|
||||
Lesson: an async "second chance" that only runs when no human has touched the
|
||||
job will lose that race to a fast human. Hold the human-facing state until the
|
||||
automatic decision is final, rather than racing it.
|
||||
|
||||
## Character limits don't stop libass wrapping a line into the row below (2026-11-09, v0.295.1)
|
||||
|
||||
NOMAD-1779 shipped with two lyric lines drawn on top of each other near the end. The lines
|
||||
("Mm- mm- mm, mm- …") passed the 40-character `max_line_length` but are made of wide glyphs:
|
||||
~3890px on the 3840px frame. libass wraps an over-wide event by itself, and since rows are
|
||||
exactly one line-height apart, the wrapped half lands on the next row's lyrics.
|
||||
|
||||
- `SubtitlesGenerator.generate_ass` now measures every line in the style font (via
|
||||
`LyricsLine._measure_text`, which scales fonts the way libass does, see `ass_font_scale`)
|
||||
and splits any line wider than `MAX_LINE_WIDTH_FRACTION` (94%) of the frame at a word
|
||||
boundary (`SegmentResizer.split_to_width`, prefers phrase-ending punctuation). It applies to
|
||||
every ASS render: main, portrait, theme previews.
|
||||
- PIL font size ≠ libass font size: libass sizes by ascender+descender, so Avenir Next Bold
|
||||
at 250 draws ~0.73× as wide as `ImageFont.truetype(path, 250)`. Measure through
|
||||
`_measure_text`, never raw PIL.
|
||||
- Scope at the time: 1 of 309 completed jobs since 2026-09-10 had an overlap; 4 more had a
|
||||
line that wrapped into an empty row.
|
||||
- `tests/unit/lyrics_transcriber/output/test_line_width_split.py` renders with real libass and
|
||||
asserts each lyric event draws exactly one row.
|
||||
@@ -1,7 +1,7 @@
|
||||
import logging
|
||||
import re
|
||||
import unicodedata
|
||||
from typing import List, Optional
|
||||
from typing import Callable, List, Optional
|
||||
|
||||
from karaoke_gen.lyrics_transcriber.types import LyricsSegment, Word
|
||||
from karaoke_gen.lyrics_transcriber.utils.word_utils import WordUtils
|
||||
@@ -257,6 +257,67 @@ class SegmentResizer:
|
||||
self._log_output_segments(resized_segments)
|
||||
return resized_segments
|
||||
|
||||
# Punctuation that ends a phrase: the preferred place to break an over-wide line
|
||||
_PHRASE_END = (",", ".", ";", ":", "!", "?", ")", "—", "-")
|
||||
|
||||
def split_to_width(
|
||||
self,
|
||||
segments: List[LyricsSegment],
|
||||
measure: Callable[[str], float],
|
||||
max_width: float,
|
||||
) -> List[LyricsSegment]:
|
||||
"""Split any segment whose rendered width exceeds ``max_width`` pixels.
|
||||
|
||||
``max_line_length`` is a character budget, which wide-glyph lines can beat:
|
||||
"Mm- mm- mm, mm- mm- mm" is short in characters but wider than the frame.
|
||||
libass then wraps the line itself and the second half lands on the row
|
||||
below, drawing over that row's lyrics. ``measure`` returns a line's pixel
|
||||
width as the renderer will draw it; lines that fit pass through unchanged.
|
||||
|
||||
A single word too wide for the frame can't be split and is kept as is.
|
||||
"""
|
||||
out: List[LyricsSegment] = []
|
||||
for segment in segments:
|
||||
out.extend(self._split_segment_to_width(segment, measure, max_width))
|
||||
return out
|
||||
|
||||
@staticmethod
|
||||
def _words_text(words: List[Word]) -> str:
|
||||
return " ".join(w.text.replace("\n", "").strip() for w in words)
|
||||
|
||||
def _split_segment_to_width(
|
||||
self, segment: LyricsSegment, measure: Callable[[str], float], max_width: float
|
||||
) -> List[LyricsSegment]:
|
||||
words = segment.words
|
||||
if len(words) < 2 or measure(self._words_text(words)) <= max_width:
|
||||
return [segment]
|
||||
|
||||
# Pick the break: prefer one where both halves fit, then a phrase-ending
|
||||
# word before the break, then the most balanced halves.
|
||||
best_key, best_i = None, 1
|
||||
for i in range(1, len(words)):
|
||||
left = measure(self._words_text(words[:i]))
|
||||
right = measure(self._words_text(words[i:]))
|
||||
fits = left <= max_width and right <= max_width
|
||||
phrase_break = words[i - 1].text.strip().endswith(self._PHRASE_END)
|
||||
key = (not fits, not phrase_break, abs(left - right))
|
||||
if best_key is None or key < best_key:
|
||||
best_key, best_i = key, i
|
||||
|
||||
self.logger.warning(
|
||||
f"Line too wide for the frame ({measure(self._words_text(words)):.0f}px > {max_width:.0f}px), "
|
||||
f"splitting at word {best_i} of {len(words)}"
|
||||
)
|
||||
pieces = []
|
||||
for part in (words[:best_i], words[best_i:]):
|
||||
piece = self._create_segment_from_words(self._words_text(part), part, singer=segment.singer)
|
||||
pieces.extend(self._split_segment_to_width(piece, measure, max_width))
|
||||
if segment.translation:
|
||||
shares = split_text_proportionally(segment.translation, [self._display_width(p.text) for p in pieces])
|
||||
for piece, share in zip(pieces, shares):
|
||||
piece.translation = share or None
|
||||
return pieces
|
||||
|
||||
def _clean_text(self, text: str) -> str:
|
||||
"""Clean text by removing newlines and extra whitespace.
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ from karaoke_gen.lyrics_transcriber.output.ass.section_detector import SectionDe
|
||||
from karaoke_gen.lyrics_transcriber.output.ass.config import ScreenConfig, chord_layout, translation_layout
|
||||
from karaoke_gen.lyrics_transcriber.output.ass.chords import ChordChange, assign_chords_to_segments, clean_chords
|
||||
from karaoke_gen.lyrics_transcriber.output.ass.event import Event
|
||||
from karaoke_gen.lyrics_transcriber.output.segment_resizer import SegmentResizer
|
||||
|
||||
|
||||
# Translation rows render at this fraction of the lyric colour's opacity, so they read as
|
||||
@@ -29,6 +30,9 @@ CHORD_BOX_RIGHT = 0.955
|
||||
CHORD_BOX_TOP = 0.045
|
||||
CHORD_BOX_SIZE_RATIO = 1.25 # box chord size / chord-row size
|
||||
CHORD_BOX_NEXT_RATIO = 0.55 # "next" chord size / current chord size
|
||||
# A lyric line may use at most this fraction of the frame width; wider lines are split
|
||||
# (libass would otherwise wrap them into the row below). Leaves room for the outline.
|
||||
MAX_LINE_WIDTH_FRACTION = 0.94
|
||||
|
||||
|
||||
def _ass_colour(color) -> str:
|
||||
@@ -180,6 +184,7 @@ class SubtitlesGenerator:
|
||||
output_path = self._get_output_path(f"{output_prefix} (Karaoke)", "ass")
|
||||
|
||||
try:
|
||||
segments = self._fit_segments_to_width(segments)
|
||||
self.logger.debug(f"Processing {len(segments)} segments")
|
||||
song_duration = self._get_audio_duration(audio_filepath, segments)
|
||||
|
||||
@@ -207,6 +212,36 @@ class SubtitlesGenerator:
|
||||
self.logger.error(f"Failed to generate ASS file: {str(e)}", exc_info=True)
|
||||
raise
|
||||
|
||||
def _fit_segments_to_width(self, segments: List[LyricsSegment]) -> List[LyricsSegment]:
|
||||
"""Split lines that would render wider than the frame.
|
||||
|
||||
Lines are sized by character count upstream, but glyph widths vary a lot
|
||||
("mm- mm- mm" is far wider per character than "it is in"). A line wider
|
||||
than the frame is wrapped by libass onto a second line one row lower,
|
||||
which draws over the next row's lyrics (NOMAD-1779). Measure each line in
|
||||
the real style font, the way libass sizes it, and split the ones that
|
||||
don't fit at a word boundary.
|
||||
"""
|
||||
from karaoke_gen.lyrics_transcriber.output.ass.style import build_karaoke_styles
|
||||
|
||||
karaoke_styles = self.styles.get("karaoke", {})
|
||||
if not segments or not karaoke_styles.get("font"):
|
||||
return segments
|
||||
style = build_karaoke_styles(karaoke_styles, singers=[1], solo=True)[0]
|
||||
style.Fontsize = self.font_size
|
||||
if not style.Fontpath or not os.path.exists(style.Fontpath):
|
||||
# Measuring with PIL's default font would under-measure; leave lines as is.
|
||||
self.logger.warning(f"Font {style.Fontpath!r} not found; skipping line width check")
|
||||
return segments
|
||||
|
||||
measurer = LyricsLine(segment=segments[0], screen_config=self.config, logger=self.logger)
|
||||
|
||||
def measure(text: str) -> float:
|
||||
return measurer._measure_text(measurer._apply_case_transform(text), style)[0]
|
||||
|
||||
max_width = self.config.video_width * MAX_LINE_WIDTH_FRACTION
|
||||
return SegmentResizer(logger=self.logger).split_to_width(segments, measure, max_width)
|
||||
|
||||
def _translation_config(self, base: ScreenConfig) -> ScreenConfig:
|
||||
"""Copy of ``base`` with taller line slots that fit a translation row under each line."""
|
||||
layout = translation_layout(self.font_size, base.line_height, self.styles.get("karaoke", {}))
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""Lines too wide for the frame must be split before libass wraps them.
|
||||
|
||||
Bug (NOMAD-1779, job 58555663): the character-count limit (``max_line_length``) let
|
||||
through lines like "Mm- mm- mm, mm- call me when you're ..." that are short in
|
||||
characters but made of wide glyphs. At 4K they measured ~3890px on a 3840px frame,
|
||||
so libass wrapped each one onto a second physical line, one row lower, on top of the
|
||||
next lyric line: two lines of lyrics drawn over each other.
|
||||
|
||||
``SubtitlesGenerator`` now measures every line in the real style font (scaled as libass
|
||||
scales it) and splits any line wider than ``MAX_LINE_WIDTH_FRACTION`` of the frame.
|
||||
"""
|
||||
import logging
|
||||
import re
|
||||
|
||||
import pytest
|
||||
|
||||
from karaoke_gen.lyrics_transcriber.output.segment_resizer import SegmentResizer
|
||||
from karaoke_gen.lyrics_transcriber.output.subtitles import MAX_LINE_WIDTH_FRACTION
|
||||
from karaoke_gen.lyrics_transcriber.types import LyricsSegment, Word
|
||||
|
||||
from .ass_render_harness import (
|
||||
FONT_SIZE,
|
||||
WIDTH,
|
||||
ffmpeg_has_libass,
|
||||
generate_ass,
|
||||
lyric_lines,
|
||||
make_segment,
|
||||
render_frames,
|
||||
)
|
||||
|
||||
|
||||
def _seg(words, translation=None):
|
||||
ws = [Word(id=f"w{i}", text=t, start_time=i * 1.0, end_time=i * 1.0 + 0.8) for i, t in enumerate(words)]
|
||||
return LyricsSegment(
|
||||
id="s", text=" ".join(words), words=ws, start_time=0.0, end_time=ws[-1].end_time, translation=translation
|
||||
)
|
||||
|
||||
|
||||
def _measure(text: str) -> float:
|
||||
"""10px per character: easy to reason about."""
|
||||
return len(text) * 10.0
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def resizer():
|
||||
return SegmentResizer(logger=logging.getLogger("test"))
|
||||
|
||||
|
||||
class TestSplitToWidth:
|
||||
def test_line_that_fits_is_unchanged(self, resizer):
|
||||
seg = _seg(["short", "line"])
|
||||
assert resizer.split_to_width([seg], _measure, 200) == [seg]
|
||||
|
||||
def test_too_wide_line_is_split_and_every_piece_fits(self, resizer):
|
||||
seg = _seg(["mm-"] * 12) # 47 chars = 470px
|
||||
out = resizer.split_to_width([seg], _measure, 200)
|
||||
assert len(out) > 1
|
||||
assert all(_measure(s.text) <= 200 for s in out)
|
||||
|
||||
def test_words_and_timing_preserved_in_order(self, resizer):
|
||||
seg = _seg([f"word{i}" for i in range(10)])
|
||||
out = resizer.split_to_width([seg], _measure, 250)
|
||||
assert [w for s in out for w in s.words] == seg.words
|
||||
for s in out:
|
||||
assert s.start_time == s.words[0].start_time
|
||||
assert s.end_time == s.words[-1].end_time
|
||||
assert s.text == " ".join(w.text for w in s.words)
|
||||
|
||||
def test_prefers_breaking_after_a_comma(self, resizer):
|
||||
# Balanced split would be after word 3; the comma after word 2 still fits.
|
||||
seg = _seg(["aaaa", "bbbb,", "cccc", "dddd", "eeee", "ffff"])
|
||||
out = resizer.split_to_width([seg], _measure, 200)
|
||||
assert [s.text for s in out] == ["aaaa bbbb,", "cccc dddd eeee ffff"]
|
||||
|
||||
def test_balanced_split_without_punctuation(self, resizer):
|
||||
seg = _seg(["aaaa", "bbbb", "cccc", "dddd"]) # 19 chars
|
||||
out = resizer.split_to_width([seg], _measure, 150)
|
||||
assert [s.text for s in out] == ["aaaa bbbb", "cccc dddd"]
|
||||
|
||||
def test_single_word_wider_than_frame_is_kept(self, resizer):
|
||||
seg = _seg(["supercalifragilistic"])
|
||||
assert resizer.split_to_width([seg], _measure, 50) == [seg]
|
||||
|
||||
def test_translation_is_shared_between_pieces(self, resizer):
|
||||
seg = _seg(["aaaa", "bbbb", "cccc", "dddd"], translation="uno dos tres cuatro")
|
||||
out = resizer.split_to_width([seg], _measure, 150)
|
||||
assert len(out) == 2
|
||||
assert " ".join(s.translation for s in out) == "uno dos tres cuatro"
|
||||
|
||||
def test_singer_carried_to_pieces(self, resizer):
|
||||
seg = _seg(["aaaa", "bbbb", "cccc", "dddd"])
|
||||
seg.singer = 2
|
||||
assert {s.singer for s in resizer.split_to_width([seg], _measure, 150)} == {2}
|
||||
|
||||
|
||||
# A hum line made of wide glyphs, ~1475px on the 1280px test frame: well clear of the split
|
||||
# threshold on every platform (font layout differs slightly between macOS and Linux).
|
||||
WIDE_HUM = ["Mm-", "mm-", "mm,"] * 4
|
||||
|
||||
|
||||
def _lyric_events(ass_path):
|
||||
with open(ass_path, encoding="utf-8-sig") as f:
|
||||
return [l for l in f if l.startswith("Dialogue") and "\\pos(" in l]
|
||||
|
||||
|
||||
def test_generated_ass_splits_wide_hum_line(tmp_path):
|
||||
ass = generate_ass(tmp_path, [make_segment(WIDE_HUM, start=4.0)])
|
||||
events = _lyric_events(ass)
|
||||
assert len(events) >= 2
|
||||
texts = [re.sub(r"\{[^}]*\}", "", e.split(",", 5)[5]).strip() for e in events]
|
||||
assert " ".join(texts) == " ".join(WIDE_HUM)
|
||||
|
||||
|
||||
def test_generated_ass_leaves_normal_lines_alone(tmp_path):
|
||||
ass = generate_ass(tmp_path, [make_segment(["it", "is", "in", "a", "line"], start=4.0)])
|
||||
assert len(_lyric_events(ass)) == 1
|
||||
|
||||
|
||||
@pytest.mark.skipif(not ffmpeg_has_libass(), reason="ffmpeg with libass not available")
|
||||
def test_every_line_renders_as_one_physical_line(tmp_path):
|
||||
"""Rendered with real libass: a line wider than the frame must not wrap into a second row.
|
||||
|
||||
Unsplit, this line is ~1475px on the 1280px test frame and libass draws its one
|
||||
event as two rows (the second one lands where the next lyric line goes).
|
||||
"""
|
||||
ass = generate_ass(tmp_path, [make_segment(WIDE_HUM, start=4.0)])
|
||||
events = _lyric_events(ass)
|
||||
frame = render_frames(ass, [5.0]).frames[5.0]
|
||||
lines = lyric_lines(frame)
|
||||
assert len(lines) == len(events), f"{len(events)} lyric events rendered as {len(lines)} rows"
|
||||
for line in lines:
|
||||
assert line.bottom - line.top < FONT_SIZE * 1.5 # one row of text, not two
|
||||
assert line.right - line.left <= WIDTH * MAX_LINE_WIDTH_FRACTION
|
||||
Reference in new issue
Block a user