mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
Translated lyrics: don't split Japanese/Chinese translations mid-word (v0.260.1) (#1102)
## Summary Follow-up to #1101, found while verifying it on prod (job ca2e881a, English song → Japanese): when a long lyric line is split across rows, its translation is shared between the pieces, and unspaced scripts were cut at an arbitrary character — e.g. 無|駄 ("waste") across two rows, worse in portrait. `split_text_proportionally` now scores each split point for unspaced scripts: free after punctuation (、。!?…), cheap after a Japanese particle (を は が に…) or before a kanji/katakana word start following hiragana, a full "part" of cost for a mid-word cut. It also never leaves a piece empty when there are enough characters. Before → after (real prod line): `ねえ、おかしいでしょ、無` / `駄な時間を省いてあげる` → `ねえ、おかしいでしょ、` / `無駄な時間を省いてあげる`. Thai can still split inside a word (no word-boundary data without a dictionary). ## Testing - New unit tests for the prod case and a particle-boundary case; all 201 lyrics output tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved how translated lyrics are split across rows for unspaced text. Splits now favor punctuation and natural language boundaries, helping avoid cutting through words where possible. * Thai text may still be split within words. * **Chores** * Updated the project version to 0.260.1. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f8d0febcf5
commit
bc1fc774eb
5 files changed
+71
-9
No files matched your search
@@ -57,6 +57,7 @@
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- **Translated lyrics: Japanese/Chinese translations no longer split mid-word** (2026-10-01, v0.260.1): when a long lyric line is split across rows, its translation is shared between them. Unspaced scripts were cut at an arbitrary character, so prod job ca2e881a showed 無|駄 ("waste") across two rows. `split_text_proportionally` now prefers punctuation, then a break after a Japanese particle or before a kanji/katakana word start, and only cuts mid-word as a last resort; it no longer leaves a piece empty when there are enough characters. Thai can still split inside a word (no word-boundary data).
|
||||
- **Translated lyrics for language learners** (2026-10-01, v0.260.0): an "Add translated lyrics" option at job creation (any job, free for now) with a language picker (33 UI locales) and a live preview of the real video frame (`GET /api/themes/translation-preview`). After review, the render worker translates the final lyrics line by line with Gemini (`backend/services/lyrics_translation.py`, GCS-cached so re-renders are free) into `lyrics/translations.json`; the landscape and portrait videos draw each translation in smaller unsung-colour text beneath its line, 3 lines per screen (`ass/config.py:translation_layout`, `LyricsLine._create_translation_event`). Translations are their own ASS events with auto paragraph direction, so mixed scripts work (a Hebrew song keeps its right-to-left highlight with an English translation beneath, and vice versa); long translations wrap/scale to fit; a line split by the resizer shares its translation across the pieces. Not editable; a translation failure renders without it (`state_data.lyrics_translation`). See [archive/2026-10-01-translated-lyrics-plan.md](archive/2026-10-01-translated-lyrics-plan.md).
|
||||
- **Re-render works for tenants that archive to Dropbox** (2026-09-30, v0.254.1): randy-vild jobs get a brand code and a Dropbox archive folder (`RVILD-0001 - …`), so v0.253.0's "never distributed" guard rejected the very jobs the button was for. A re-render now reuses the job's brand code and overwrites the same folder instead of allocating a new code. YouTube/GDrive-published jobs are still rejected.
|
||||
- **Tenant "Re-render" with the current theme + edited tenant tracks no longer lose their downloads** (2026-09-30, v0.253.0): Finished portal tracks have a **Re-render** button (`POST /api/jobs/{id}/rerender`). It re-applies the tenant's current theme and rebuilds the screens, lyrics video, encodes and CDG/TXT from the already-reviewed lyrics, with no review step (`backend/services/theme_rerender_service.py`, same mechanism as the private→public visibility change). The encoding worker caches finished jobs by id in memory, so a re-run submitted while the VM was still up could get the previous render back. Render and encode now submit as `{job_id}_g{worker_generation}` (`encoding_worker_job_id`). Separately, `outputs_deleted_at` (set by Edit) was only cleared when the re-render uploaded to YouTube/Dropbox/GDrive, so edited tenant jobs re-rendered but hid every download (job 2579a1ea). Every successful render now clears it.
|
||||
|
||||
@@ -15,6 +15,9 @@ import { URLS } from '../helpers/constants';
|
||||
|
||||
const API_URL = URLS.production.api;
|
||||
|
||||
// API-only: the prod config's always-on trace hangs on the large image responses
|
||||
test.use({ trace: 'off', video: 'off', screenshot: 'off' });
|
||||
|
||||
test.describe('Translated lyrics preview', () => {
|
||||
for (const language of ['es', 'he', 'ja']) {
|
||||
test(`renders a preview frame for ${language}`, async ({ request }) => {
|
||||
|
||||
@@ -72,10 +72,45 @@ def grapheme_clusters(text: str) -> List[str]:
|
||||
return clusters
|
||||
|
||||
|
||||
_BREAK_AFTER = set("、。,,..!!??;;::・…))」』】〕〉》 ")
|
||||
# Japanese particles: a break right after one usually falls between phrases
|
||||
_JA_PARTICLES = set("をはがにでともへの")
|
||||
|
||||
|
||||
def _char_script(ch: str) -> str:
|
||||
if "\u3040" <= ch <= "\u309f":
|
||||
return "hiragana"
|
||||
if "\u30a0" <= ch <= "\u30ff" or "\u31f0" <= ch <= "\u31ff":
|
||||
return "katakana"
|
||||
if "\u4e00" <= ch <= "\u9fff" or "\u3400" <= ch <= "\u4dbf":
|
||||
return "han"
|
||||
return "other"
|
||||
|
||||
|
||||
def _unspaced_break_penalty(tokens: List[str], end: int, unit: float) -> float:
|
||||
"""Cost (in characters) of splitting unspaced text before ``tokens[end]``.
|
||||
|
||||
Free after punctuation; cheap after a Japanese particle or where hiragana gives way
|
||||
to kanji/katakana (usually a new word); otherwise a mid-word cut, avoided unless
|
||||
nothing else fits.
|
||||
"""
|
||||
if end <= 0 or end >= len(tokens):
|
||||
return 0.0
|
||||
prev, nxt = tokens[end - 1][-1], tokens[end][0]
|
||||
if prev in _BREAK_AFTER:
|
||||
return 0.0
|
||||
if _char_script(prev) == "hiragana" and _char_script(nxt) in ("han", "katakana"):
|
||||
return 0.25 * unit
|
||||
if prev in _JA_PARTICLES and _char_script(nxt) != "other":
|
||||
return 0.25 * unit
|
||||
return unit
|
||||
|
||||
|
||||
def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
|
||||
"""Split ``text`` into ``len(weights)`` consecutive parts sized by ``weights``.
|
||||
|
||||
Splits at spaces; unspaced scripts (CJK, Thai) split between characters.
|
||||
Splits at spaces. Unspaced scripts (CJK, Thai) split between grapheme clusters,
|
||||
preferring punctuation and likely word starts over cutting a word in half.
|
||||
Parts can be empty when there are fewer tokens than weights.
|
||||
"""
|
||||
n = len(weights)
|
||||
@@ -83,11 +118,14 @@ def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
|
||||
return [text]
|
||||
tokens = text.split()
|
||||
joiner = " "
|
||||
unspaced = False
|
||||
if len(tokens) < n and is_unspaced_script(text):
|
||||
tokens, joiner = grapheme_clusters(text), ""
|
||||
tokens, joiner, unspaced = grapheme_clusters(text), "", True
|
||||
total_weight = sum(max(0.0, w) for w in weights) or float(n)
|
||||
lengths = [len(t) for t in tokens]
|
||||
total_len = sum(lengths) or 1
|
||||
# A mid-word cut costs as much as moving the split by a whole average part
|
||||
unit = total_len / n
|
||||
parts: List[str] = []
|
||||
start = 0
|
||||
cumulative_weight = 0.0
|
||||
@@ -97,14 +135,23 @@ def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
|
||||
break
|
||||
cumulative_weight += max(0.0, w) if sum(weights) > 0 else 1.0
|
||||
target = total_len * cumulative_weight / total_weight
|
||||
# Smallest end index whose cumulative length is closest to the target
|
||||
best_end, best_diff, running = start, None, sum(lengths[:start])
|
||||
for end in range(start, len(tokens) + 1):
|
||||
# End index whose cumulative length is closest to the target (plus break cost)
|
||||
best_end, best_cost, running = start, None, sum(lengths[:start])
|
||||
# Leave at least one token for this part and each remaining one, when there are enough
|
||||
remaining_parts = n - 1 - i
|
||||
lo, hi = start, len(tokens)
|
||||
if len(tokens) - start >= remaining_parts + 1:
|
||||
lo, hi = start + 1, len(tokens) - remaining_parts
|
||||
for end in range(start, hi + 1):
|
||||
if end > start:
|
||||
running += lengths[end - 1]
|
||||
diff = abs(running - target)
|
||||
if best_diff is None or diff < best_diff:
|
||||
best_end, best_diff = end, diff
|
||||
if end < lo:
|
||||
continue
|
||||
cost = abs(running - target)
|
||||
if unspaced:
|
||||
cost += _unspaced_break_penalty(tokens, end, unit)
|
||||
if best_cost is None or cost < best_cost:
|
||||
best_end, best_cost = end, cost
|
||||
parts.append(joiner.join(tokens[start:best_end]))
|
||||
start = best_end
|
||||
return parts
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "karaoke-gen"
|
||||
version = "0.260.0"
|
||||
version = "0.260.1"
|
||||
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
|
||||
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
|
||||
license = "MIT"
|
||||
|
||||
@@ -300,3 +300,14 @@ def test_instrumental_card_position_unchanged_by_translations(tmp_path):
|
||||
plain = [seg("first line", 1), seg("after the break", 40)]
|
||||
translated = [seg("first line", 1, translation="primera"), seg("after the break", 40, translation="después")]
|
||||
assert instrumental_y(plain) == instrumental_y(translated)
|
||||
|
||||
|
||||
def test_japanese_split_prefers_punctuation_over_mid_word():
|
||||
# Seen in prod (job ca2e881a): 無駄 ("waste") was cut in half across two rows
|
||||
parts = split_text_proportionally("ねえ、おかしいでしょ、無駄な時間を省いてあげる", [22, 32])
|
||||
assert parts == ["ねえ、おかしいでしょ、", "無駄な時間を省いてあげる"]
|
||||
|
||||
|
||||
def test_japanese_split_falls_back_to_a_phrase_boundary():
|
||||
parts = split_text_proportionally("自分の生活をしようとしてるのに", [1, 1])
|
||||
assert parts == ["自分の生活を", "しようとしてるのに"] # after the particle を, not inside しよう
|
||||
Reference in new issue
Block a user