Translated lyrics: don't split Japanese/Chinese translations mid-word (v0.260.1) (#1102)

## Summary
Follow-up to #1101, found while verifying it on prod (job ca2e881a,
English song → Japanese): when a long lyric line is split across rows,
its translation is shared between the pieces, and unspaced scripts were
cut at an arbitrary character — e.g. 無|駄 ("waste") across two rows,
worse in portrait.

`split_text_proportionally` now scores each split point for unspaced
scripts: free after punctuation (、。!?…), cheap after a Japanese particle
(を は が に…) or before a kanji/katakana word start following hiragana, a
full "part" of cost for a mid-word cut. It also never leaves a piece
empty when there are enough characters.

Before → after (real prod line): `ねえ、おかしいでしょ、無` / `駄な時間を省いてあげる` →
`ねえ、おかしいでしょ、` / `無駄な時間を省いてあげる`.

Thai can still split inside a word (no word-boundary data without a
dictionary).

## Testing
- New unit tests for the prod case and a particle-boundary case; all 201
lyrics output tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved how translated lyrics are split across rows for unspaced
text. Splits now favor punctuation and natural language boundaries,
helping avoid cutting through words where possible.
  * Thai text may still be split within words.
* **Chores**
  * Updated the project version to 0.260.1.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude Opus 5.5 authored and GitHub committed 2026-10-01 14:39:13 -04:00
1 parent f8d0febcf5
commit bc1fc774eb
5 files changed
+71 -9

No files matched your search

+1
View File
@@ -57,6 +57,7 @@
## Recent Changes
- **Translated lyrics: Japanese/Chinese translations no longer split mid-word** (2026-10-01, v0.260.1): when a long lyric line is split across rows, its translation is shared between them. Unspaced scripts were cut at an arbitrary character, so prod job ca2e881a showed 無|駄 ("waste") across two rows. `split_text_proportionally` now prefers punctuation, then a break after a Japanese particle or before a kanji/katakana word start, and only cuts mid-word as a last resort; it no longer leaves a piece empty when there are enough characters. Thai can still split inside a word (no word-boundary data).
- **Translated lyrics for language learners** (2026-10-01, v0.260.0): an "Add translated lyrics" option at job creation (any job, free for now) with a language picker (33 UI locales) and a live preview of the real video frame (`GET /api/themes/translation-preview`). After review, the render worker translates the final lyrics line by line with Gemini (`backend/services/lyrics_translation.py`, GCS-cached so re-renders are free) into `lyrics/translations.json`; the landscape and portrait videos draw each translation in smaller unsung-colour text beneath its line, 3 lines per screen (`ass/config.py:translation_layout`, `LyricsLine._create_translation_event`). Translations are their own ASS events with auto paragraph direction, so mixed scripts work (a Hebrew song keeps its right-to-left highlight with an English translation beneath, and vice versa); long translations wrap/scale to fit; a line split by the resizer shares its translation across the pieces. Not editable; a translation failure renders without it (`state_data.lyrics_translation`). See [archive/2026-10-01-translated-lyrics-plan.md](archive/2026-10-01-translated-lyrics-plan.md).
- **Re-render works for tenants that archive to Dropbox** (2026-09-30, v0.254.1): randy-vild jobs get a brand code and a Dropbox archive folder (`RVILD-0001 - …`), so v0.253.0's "never distributed" guard rejected the very jobs the button was for. A re-render now reuses the job's brand code and overwrites the same folder instead of allocating a new code. YouTube/GDrive-published jobs are still rejected.
- **Tenant "Re-render" with the current theme + edited tenant tracks no longer lose their downloads** (2026-09-30, v0.253.0): Finished portal tracks have a **Re-render** button (`POST /api/jobs/{id}/rerender`). It re-applies the tenant's current theme and rebuilds the screens, lyrics video, encodes and CDG/TXT from the already-reviewed lyrics, with no review step (`backend/services/theme_rerender_service.py`, same mechanism as the private→public visibility change). The encoding worker caches finished jobs by id in memory, so a re-run submitted while the VM was still up could get the previous render back. Render and encode now submit as `{job_id}_g{worker_generation}` (`encoding_worker_job_id`). Separately, `outputs_deleted_at` (set by Edit) was only cleared when the re-render uploaded to YouTube/Dropbox/GDrive, so edited tenant jobs re-rendered but hid every download (job 2579a1ea). Every successful render now clears it.
@@ -15,6 +15,9 @@ import { URLS } from '../helpers/constants';
const API_URL = URLS.production.api;
// API-only: the prod config's always-on trace hangs on the large image responses
test.use({ trace: 'off', video: 'off', screenshot: 'off' });
test.describe('Translated lyrics preview', () => {
for (const language of ['es', 'he', 'ja']) {
test(`renders a preview frame for ${language}`, async ({ request }) => {
@@ -72,10 +72,45 @@ def grapheme_clusters(text: str) -> List[str]:
return clusters
_BREAK_AFTER = set("、。,,..!!??;;::・…))」』】〕〉》 ")
# Japanese particles: a break right after one usually falls between phrases
_JA_PARTICLES = set("をはがにでともへの")
def _char_script(ch: str) -> str:
if "\u3040" <= ch <= "\u309f":
return "hiragana"
if "\u30a0" <= ch <= "\u30ff" or "\u31f0" <= ch <= "\u31ff":
return "katakana"
if "\u4e00" <= ch <= "\u9fff" or "\u3400" <= ch <= "\u4dbf":
return "han"
return "other"
def _unspaced_break_penalty(tokens: List[str], end: int, unit: float) -> float:
"""Cost (in characters) of splitting unspaced text before ``tokens[end]``.
Free after punctuation; cheap after a Japanese particle or where hiragana gives way
to kanji/katakana (usually a new word); otherwise a mid-word cut, avoided unless
nothing else fits.
"""
if end <= 0 or end >= len(tokens):
return 0.0
prev, nxt = tokens[end - 1][-1], tokens[end][0]
if prev in _BREAK_AFTER:
return 0.0
if _char_script(prev) == "hiragana" and _char_script(nxt) in ("han", "katakana"):
return 0.25 * unit
if prev in _JA_PARTICLES and _char_script(nxt) != "other":
return 0.25 * unit
return unit
def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
"""Split ``text`` into ``len(weights)`` consecutive parts sized by ``weights``.
Splits at spaces; unspaced scripts (CJK, Thai) split between characters.
Splits at spaces. Unspaced scripts (CJK, Thai) split between grapheme clusters,
preferring punctuation and likely word starts over cutting a word in half.
Parts can be empty when there are fewer tokens than weights.
"""
n = len(weights)
@@ -83,11 +118,14 @@ def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
return [text]
tokens = text.split()
joiner = " "
unspaced = False
if len(tokens) < n and is_unspaced_script(text):
tokens, joiner = grapheme_clusters(text), ""
tokens, joiner, unspaced = grapheme_clusters(text), "", True
total_weight = sum(max(0.0, w) for w in weights) or float(n)
lengths = [len(t) for t in tokens]
total_len = sum(lengths) or 1
# A mid-word cut costs as much as moving the split by a whole average part
unit = total_len / n
parts: List[str] = []
start = 0
cumulative_weight = 0.0
@@ -97,14 +135,23 @@ def split_text_proportionally(text: str, weights: List[float]) -> List[str]:
break
cumulative_weight += max(0.0, w) if sum(weights) > 0 else 1.0
target = total_len * cumulative_weight / total_weight
# Smallest end index whose cumulative length is closest to the target
best_end, best_diff, running = start, None, sum(lengths[:start])
for end in range(start, len(tokens) + 1):
# End index whose cumulative length is closest to the target (plus break cost)
best_end, best_cost, running = start, None, sum(lengths[:start])
# Leave at least one token for this part and each remaining one, when there are enough
remaining_parts = n - 1 - i
lo, hi = start, len(tokens)
if len(tokens) - start >= remaining_parts + 1:
lo, hi = start + 1, len(tokens) - remaining_parts
for end in range(start, hi + 1):
if end > start:
running += lengths[end - 1]
diff = abs(running - target)
if best_diff is None or diff < best_diff:
best_end, best_diff = end, diff
if end < lo:
continue
cost = abs(running - target)
if unspaced:
cost += _unspaced_break_penalty(tokens, end, unit)
if best_cost is None or cost < best_cost:
best_end, best_cost = end, cost
parts.append(joiner.join(tokens[start:best_end]))
start = best_end
return parts
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.260.0"
version = "0.260.1"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
@@ -300,3 +300,14 @@ def test_instrumental_card_position_unchanged_by_translations(tmp_path):
plain = [seg("first line", 1), seg("after the break", 40)]
translated = [seg("first line", 1, translation="primera"), seg("after the break", 40, translation="después")]
assert instrumental_y(plain) == instrumental_y(translated)
def test_japanese_split_prefers_punctuation_over_mid_word():
# Seen in prod (job ca2e881a): 無駄 ("waste") was cut in half across two rows
parts = split_text_proportionally("ねえ、おかしいでしょ、無駄な時間を省いてあげる", [22, 32])
assert parts == ["ねえ、おかしいでしょ、", "無駄な時間を省いてあげる"]
def test_japanese_split_falls_back_to_a_phrase_boundary():
parts = split_text_proportionally("自分の生活をしようとしてるのに", [1, 1])
assert parts == ["自分の生活を", "しようとしてるのに"] # after the particle を, not inside しよう