Compare commits

...
Author SHA1 Message Date
Andrew Beveridge ☄️ 80f140c53d Merge branch 'main' into feature/agentic-playwright-tests-docs 2025-12-28 13:41:48 -05:00
Andrew Beveridge ☄️ 76fab7d73e Merge pull request #103 from nomadkaraoke/fix/runner-work-permissions
fix: correct _work directory ownership in runner startup script
2025-12-28 13:38:27 -05:00
Andrew BeveridgeandClaude Opus 4.5 3916652871 fix: correct _work directory ownership in runner startup script
The _work directory was owned by root but runner service runs as 'runner' user,
causing "Access denied" errors when jobs tried to create _temp directory.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 13:30:15 -05:00
Andrew Beveridge ☄️ 1bc6dbe1a2 Merge pull request #100 from nomadkaraoke/fix/upload-endpoint-theme-support
Fix: Add theme support to /api/jobs/upload endpoint
2025-12-28 13:27:23 -05:00
Andrew Beveridge ☄️ 71b33a6b93 Merge pull request #101 from nomadkaraoke/feature/multiple-runners
feat: scale to 5 self-hosted GitHub runners for parallel CI jobs
2025-12-28 13:26:04 -05:00
Andrew BeveridgeandClaude Opus 4.5 c3e1edfcb4 feat: scale to 5 self-hosted GitHub runners for parallel CI jobs
- Replace single github-runner VM with 5 instances (github-runner-1 through 5)
- Enables parallel job execution across multiple PRs
- Each runner: e2-standard-4, 200GB SSD, same tooling
- Update README with new runner count and troubleshooting commands

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 13:11:49 -05:00
Andrew BeveridgeandClaude Opus 4.5 44c591a629 Fix: Add theme support to /api/jobs/upload endpoint
The /api/jobs/upload endpoint was missing theme_id and color_overrides
parameters, causing themes selected in the frontend to not be passed
through when creating jobs via file upload. This resulted in preview
videos having black backgrounds instead of themed backgrounds.

Changes:
- Add theme_id and color_overrides Form parameters to upload endpoint
- Change enable_cdg/enable_txt to Optional[bool] for smart defaults
- Add _resolve_cdg_txt_defaults() to enable CDG/TXT when theme is set
- Add _prepare_theme_for_job() call when theme_id is set (no custom style files)
- Pass theme_id and color_overrides to JobCreate
- Use theme's YouTube description template if no custom one provided

Root cause: The frontend was sending theme_id in the form data, but the
backend endpoint was not accepting this parameter, so it was silently
ignored. PR #72 fixed this for audio_search.py but this endpoint was missed.

Fixes issue where job c9516773 (file upload) had black background while
job 47475700 (song search) correctly showed themed background.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:32:18 -05:00
Andrew BeveridgeandClaude Opus 4.5 fd63d0d948 docs: Add Playwright assertion best practices to lessons learned
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:25:48 -05:00
Andrew BeveridgeandClaude Opus 4.5 84f4e3b769 refactor: Replace waitForTimeout with proper assertions in Playwright tests
- Remove all waitForTimeout() calls in favor of expect().toBeVisible() assertions
- Use Playwright's built-in auto-retry mechanism for reliable tests
- Remove diagnostic code (console.log, screenshots)
- Address CodeRabbit nitpicks about arbitrary timeouts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:21:51 -05:00
Andrew BeveridgeandClaude Opus 4.5 597233b432 Merge main into feature/agentic-playwright-tests-docs
- Resolve version conflict: bump to 0.76.26
- Keep Header.tsx with Review Mode changes
- Take main's yarn.lock

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:20:05 -05:00
Andrew Beveridge ☄️ 2203595e51 Merge pull request #99 from nomadkaraoke/feature/coderabbit-review-command
feat: add /coderabbit-review slash command
2025-12-28 12:17:40 -05:00
Andrew BeveridgeandClaude Opus 4.5 635c4e7ee6 feat: add /coderabbit-review slash command
Adds a Claude Code slash command that reviews and addresses CodeRabbit
comments on the current PR:
- Fetches both review threads and nitpick comments
- Analyzes each comment and decides if action is needed
- Makes appropriate code fixes
- Commits/pushes changes and resolves threads

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:16:28 -05:00
Andrew BeveridgeandClaude Opus 4.5 3a65a8ee13 feat: Add Review Mode for agentic corrections with inline actions
- Add Review Mode toggle in Header (shows when agentic mode is active)
- Add batch action buttons: Accept All, Accept High Confidence, Revert All
- Wire up CorrectedWordWithActions component for agentic corrections
- Show original word preview (strikethrough) on all agentic corrections
- Add showActions prop to conditionally show inline buttons
- Mobile: tap opens detail modal instead of inline buttons
- Fix CodeRabbit issues: format bare URLs as markdown links
- Fix fixture data: align original_word values with segment words
- Add new Playwright tests for Review Mode functionality

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:11:28 -05:00
Andrew Beveridge ☄️ 1c5711cfa5 Merge pull request #98 from nomadkaraoke/docs/runner-setup
docs: add self-hosted runner documentation
2025-12-28 12:06:31 -05:00
Andrew Beveridge ☄️ 08d7db9943 Merge pull request #97 from nomadkaraoke/fix/runner-startup-script
fix: make runner startup script reproducible
2025-12-28 12:06:24 -05:00
Andrew BeveridgeandClaude Opus 4.5 d2c8c24eee fix: make runner startup script reproducible
Fixes issues discovered during initial runner setup:

- Use `gpg --batch` for all GPG commands to avoid tty errors
- Install Python 3.13 via pyenv (not broken PPA/source build)
- Set up Python 3.13 in tool cache for setup-python action
- Use Temurin directly for Java 21 (skip broken Debian repos)

These changes ensure the runner VM can be recreated from scratch
and work correctly with GitHub Actions setup-python.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:03:10 -05:00
Andrew Beveridge e709e0ddaf Merge remote-tracking branch 'origin/main' into feature/agentic-playwright-tests-docs 2025-12-26 13:32:40 -05:00
Andrew BeveridgeandClaude Opus 4.5 b7324e5aa9 Add agentic correction Playwright E2E tests and update docs
- Add Playwright E2E tests for agentic correction workflow
  - Test UI component loading (read-only banner, load file button)
  - Test file upload flow with mock correction data
  - Test fixture with agentic correction categories
  - ESM module fix for __dirname in test files
- Create comprehensive AGENTIC_CORRECTION_GUIDE.md user documentation
- Update AGENTIC_IMPLEMENTATION_STATUS.md to reflect current state
- Remove outdated AGENTIC_NEXT_STEPS.md (steps completed)
- Add test-results/ to frontend .gitignore
- Add test scripts to frontend package.json

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-26 01:39:16 -05:00
21 changed files with 1759 additions and 773 deletions

No files matched your search

+109
View File
@@ -0,0 +1,109 @@
---
description: Review and address CodeRabbit comments on current PR
allowed-tools: Read, Glob, Grep, Edit, Write, Bash, mcp__github__pull_request_read, mcp__github__get_me, mcp__github__list_pull_requests, mcp__github__add_issue_comment
---
# CodeRabbit Comment Review
You are reviewing the current PR to address all CodeRabbit automated review comments.
## Your Task
### 1. Identify the Current PR
First, determine the current branch and find the associated PR:
```bash
git branch --show-current
```
Then use GitHub tools to find the PR for this branch in the `nomadkaraoke/karaoke-gen` repository.
### 2. Fetch CodeRabbit Comments
Get all review comments and regular comments on the PR:
**Review Comments (Potential Issues)**
- Use `pull_request_read` with `method: "get_review_comments"` to get all review comment threads
- Look for comments from the `coderabbitai` bot user
- These are typically marked as "Potential issue" and MUST be resolved before merge
**PR Comments (Including Nitpicks)**
- Use `pull_request_read` with `method: "get_comments"` to get regular PR comments
- CodeRabbit posts a summary comment that includes "Nitpick comments" section
- These should also be reviewed and considered for action
### 3. Analyze Each Comment
For each CodeRabbit comment:
1. Read the comment carefully to understand the concern
2. Look at the referenced code location
3. Decide if a code change is warranted:
- **Make changes** for: valid bugs, security issues, clear improvements, style violations
- **Skip changes** for: subjective preferences, false positives, already addressed, not applicable
4. If skipping, note the reason
### 4. Make Code Changes
For each comment requiring action:
1. Navigate to the file and line mentioned
2. Understand the surrounding context
3. Make the minimal change needed to address the concern
4. Verify the change doesn't break anything obvious
### 5. Commit and Push
After making all changes:
```bash
# Stage all changes
git add -A
# Commit with descriptive message
git commit -m "fix: address CodeRabbit review comments
- [list each fix briefly]
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
# Push to the PR branch
git push
```
### 6. Resolve Threads
After pushing:
- Reply to each CodeRabbit review thread indicating what was done
- If a comment was addressed, reply with what was changed
- If a comment was intentionally skipped, explain why
- This helps mark threads as resolved so the PR can be merged
## Output Format
Provide a summary report:
```
## CodeRabbit Review Summary
### Comments Addressed
- [ ] File:line - Brief description of issue → What was changed
### Comments Skipped (with reason)
- [ ] File:line - Issue → Reason for skipping
### Actions Taken
- Committed: [commit hash]
- Pushed to: [branch name]
- Threads replied to: [count]
### Status
Ready for merge: Yes/No (if No, explain why)
```
## Guidelines
- Focus on code quality improvements, not just making CodeRabbit happy
- Don't blindly accept every suggestion - use judgment
- Group related fixes into logical commits if needed
- Keep changes minimal and focused on the specific issues raised
- If a suggestion would require significant refactoring, note it for future work instead
+74 -14
View File
@@ -341,9 +341,12 @@ async def upload_and_create_job(
style_cdg_instrumental_background: Optional[UploadFile] = File(None, description="CDG instrumental background"),
style_cdg_title_background: Optional[UploadFile] = File(None, description="CDG title screen background"),
style_cdg_outro_background: Optional[UploadFile] = File(None, description="CDG outro screen background"),
# Processing options (CDG/TXT require style config, disabled by default)
enable_cdg: bool = Form(False, description="Generate CDG+MP3 package (requires style config)"),
enable_txt: bool = Form(False, description="Generate TXT+MP3 package (requires style config)"),
# Theme configuration (use pre-made theme from GCS)
theme_id: Optional[str] = Form(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default."),
color_overrides: Optional[str] = Form(None, description="JSON-encoded color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)"),
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
enable_cdg: Optional[bool] = Form(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise"),
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
# Finalisation options
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
@@ -477,7 +480,35 @@ async def upload_and_create_job(
# Extract request metadata for tracking and filtering
request_metadata = extract_request_metadata(request, created_from="upload")
# Parse color_overrides from JSON if provided
parsed_color_overrides: Dict[str, str] = {}
if color_overrides:
try:
parsed_color_overrides = json.loads(color_overrides)
except json.JSONDecodeError as e:
raise HTTPException(
status_code=400,
detail=f"Invalid color_overrides JSON: {e}"
)
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
theme_id, enable_cdg, enable_txt
)
# Check if any custom style files are being uploaded (overrides theme)
has_custom_style_files = any([
style_params,
style_intro_background,
style_karaoke_background,
style_end_background,
style_font,
style_cdg_instrumental_background,
style_cdg_title_background,
style_cdg_outro_background,
])
# Parse comma-separated model lists into arrays
parsed_backing_vocals_models = None
if backing_vocals_models:
@@ -492,8 +523,10 @@ async def upload_and_create_job(
artist=artist,
title=title,
filename=file.filename,
enable_cdg=enable_cdg,
enable_txt=enable_txt,
theme_id=theme_id,
color_overrides=parsed_color_overrides,
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=brand_prefix,
enable_youtube_upload=enable_youtube_upload,
youtube_description=youtube_description,
@@ -521,9 +554,27 @@ async def upload_and_create_job(
# Record job creation metric
metrics.record_job_created(job_id, source="upload")
logger.info(f"Created job {job_id} for {artist} - {title}")
# If theme is set and no custom style files are being uploaded, prepare theme style now
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
# can access the style configuration for preview videos
theme_style_params_path = None
theme_style_assets = {}
theme_youtube_desc = None
if theme_id and not has_custom_style_files:
try:
theme_style_params_path, theme_style_assets, theme_youtube_desc = _prepare_theme_for_job(
job_id, theme_id, parsed_color_overrides or None
)
logger.info(f"Applied theme '{theme_id}' to job {job_id}")
except HTTPException:
raise # Re-raise validation errors (e.g., theme not found)
except Exception as e:
logger.warning(f"Failed to prepare theme '{theme_id}' for job {job_id}: {e}")
# Continue without theme - job can still be processed with defaults
# Upload main audio file to GCS
audio_gcs_path = f"uploads/{job_id}/audio/{file.filename}"
logger.info(f"Uploading audio to GCS: {audio_gcs_path}")
@@ -592,21 +643,30 @@ async def upload_and_create_job(
update_data = {
'input_media_gcs_path': audio_gcs_path,
'filename': file.filename,
'enable_cdg': enable_cdg,
'enable_txt': enable_txt,
'enable_cdg': resolved_cdg,
'enable_txt': resolved_txt,
}
# Handle style assets - either from custom uploads or from theme
if style_assets:
# Custom style files uploaded
update_data['style_assets'] = style_assets
if 'style_params' in style_assets:
update_data['style_params_gcs_path'] = style_assets['style_params']
elif theme_style_assets:
# Theme style assets (no custom uploads)
update_data['style_assets'] = theme_style_assets
if theme_style_params_path:
update_data['style_params_gcs_path'] = theme_style_params_path
if brand_prefix:
update_data['brand_prefix'] = brand_prefix
if effective_discord_webhook_url:
update_data['discord_webhook_url'] = effective_discord_webhook_url
if youtube_description:
update_data['youtube_description_template'] = youtube_description
# Use theme YouTube description if no custom one provided
effective_youtube_description = youtube_description or theme_youtube_desc
if effective_youtube_description:
update_data['youtube_description_template'] = effective_youtube_description
# Native API distribution (use effective values which include defaults)
if effective_dropbox_path:
+128
View File
@@ -1606,5 +1606,133 @@ class TestIsUrlFunction:
assert is_url("") is False
class TestUploadEndpointThemeSupport:
"""Test that /jobs/upload endpoint supports theme configuration.
CRITICAL: These tests verify that the /api/jobs/upload endpoint correctly handles
theme_id and color_overrides parameters, ensuring preview videos have themed
backgrounds instead of black backgrounds.
This addresses a bug where:
- The frontend sends theme_id and color_overrides when uploading files
- But the backend /api/jobs/upload endpoint was ignoring these parameters
- Result: Jobs created via file upload had black backgrounds instead of themed ones
"""
def test_upload_endpoint_accepts_theme_id_parameter(self):
"""Verify the upload endpoint has theme_id as a form parameter.
CRITICAL: The frontend sends theme_id when uploading files with a theme.
If this parameter is missing from the endpoint, the theme is silently ignored
and preview videos will have black backgrounds instead of themed ones.
"""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
has_theme_id_param = 'theme_id: Optional[str] = Form(' in source_code
assert has_theme_id_param, (
"file_upload.py /jobs/upload endpoint does not have theme_id as a Form parameter. "
"The frontend sends theme_id when uploading files with a theme, but the backend "
"ignores it. Add: theme_id: Optional[str] = Form(None, description='Theme ID...')"
)
def test_upload_endpoint_accepts_color_overrides_parameter(self):
"""Verify the upload endpoint has color_overrides as a form parameter."""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
has_color_overrides_param = 'color_overrides: Optional[str] = Form(' in source_code
assert has_color_overrides_param, (
"file_upload.py /jobs/upload endpoint does not have color_overrides as a Form parameter. "
"The frontend sends color_overrides when customizing theme colors."
)
def test_upload_endpoint_calls_prepare_theme_for_job(self):
"""Verify the upload endpoint calls _prepare_theme_for_job when theme_id is set.
CRITICAL: When a job is created via the upload endpoint with a theme_id
(and no custom style files), the code must call _prepare_theme_for_job() to set:
1. style_params_gcs_path (pointing to the copied style_params.json)
2. style_assets (populated with asset mappings)
Without this, LyricsTranscriber won't have access to the theme's styles
and preview videos will have black backgrounds instead of themed ones.
"""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
# The function should be called in upload_and_create_job
has_theme_prep_call = '_prepare_theme_for_job(' in source_code
assert has_theme_prep_call, (
"file_upload.py does not call _prepare_theme_for_job(). "
"When theme_id is provided to /jobs/upload without custom style files, "
"the endpoint MUST call _prepare_theme_for_job() to copy the theme's "
"style_params.json to the job folder."
)
def test_upload_endpoint_uses_resolve_cdg_txt_defaults(self):
"""Verify the upload endpoint uses _resolve_cdg_txt_defaults for theme-based defaults."""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
has_resolve_call = '_resolve_cdg_txt_defaults(' in source_code
assert has_resolve_call, (
"file_upload.py does not call _resolve_cdg_txt_defaults(). "
"When theme_id is set, enable_cdg and enable_txt should default to True."
)
def test_upload_endpoint_has_optional_cdg_txt_params(self):
"""Verify enable_cdg and enable_txt are Optional[bool] to support theme defaults.
CRITICAL: If enable_cdg/enable_txt are bool instead of Optional[bool],
they will default to False and override the theme-based defaults.
"""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
has_optional_cdg = 'enable_cdg: Optional[bool] = Form(' in source_code
has_optional_txt = 'enable_txt: Optional[bool] = Form(' in source_code
assert has_optional_cdg, (
"file_upload.py has enable_cdg as bool instead of Optional[bool]. "
"This prevents theme-based defaults from working."
)
assert has_optional_txt, (
"file_upload.py has enable_txt as bool instead of Optional[bool]. "
"This prevents theme-based defaults from working."
)
def test_job_create_includes_theme_id_and_color_overrides(self):
"""Verify JobCreate is called with theme_id and color_overrides."""
from backend.api.routes import file_upload as file_upload_module
with open(file_upload_module.__file__, 'r') as f:
source_code = f.read()
has_theme_id_in_job_create = 'theme_id=theme_id,' in source_code
has_color_overrides_in_job_create = 'color_overrides=parsed_color_overrides' in source_code
assert has_theme_id_in_job_create, (
"file_upload.py does not pass theme_id to JobCreate."
)
assert has_color_overrides_in_job_create, (
"file_upload.py does not pass color_overrides to JobCreate."
)
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+19
View File
@@ -106,6 +106,25 @@ Unit tests with mocks didn't catch the `input_media_gcs_path` bug. Emulator inte
Emulator tests need their own `conftest.py` that does NOT mock `google.cloud.*`. Mocks from unit tests can leak into integration tests.
### Playwright: Use Assertions, Not Timeouts
**Problem**: `waitForTimeout()` is flaky and slow:
```typescript
// BAD - arbitrary wait, still flaky
await page.waitForTimeout(2000);
const hasContent = await page.locator('.content').count();
```
**Solution**: Use Playwright's auto-retry assertions:
```typescript
// GOOD - waits only until condition met, retries automatically
await expect(page.getByText('Hello,')).toBeVisible({ timeout: 5000 });
```
**Why it matters**: Playwright assertions have built-in retry logic and fail fast when conditions aren't met. Arbitrary timeouts either waste time (waiting when condition is already met) or cause flaky tests (not waiting long enough on slow CI).
## Deployment Notes
### Cloud Run Timeouts
+24 -13
View File
@@ -254,15 +254,16 @@ The function uses environment variables:
Known gaps are configured in `main.py` and should match your local `validate_and_sync.py`.
## Self-Hosted GitHub Actions Runner
## Self-Hosted GitHub Actions Runners
A dedicated GCP VM runs self-hosted GitHub Actions runners to solve disk space issues
on GitHub-hosted runners (14GB vs 200GB).
Multiple GCP VMs run self-hosted GitHub Actions runners to:
- Solve disk space issues on GitHub-hosted runners (14GB vs 200GB)
- Enable parallel job execution across multiple runners
### Specs
- **Instance**: e2-standard-4 (4 vCPU, 16GB RAM)
- **Disk**: 200GB SSD
- **Instances**: 5x e2-standard-4 (4 vCPU, 16GB RAM each)
- **Disk**: 200GB SSD per runner
- **Pre-installed**: Docker, Python 3.13, Node.js 20, Java 21, Poetry, FFmpeg, gcloud CLI
- **Labels**: `self-hosted`, `linux`, `x64`, `gcp`, `large-disk`
@@ -280,15 +281,15 @@ on GitHub-hosted runners (14GB vs 200GB).
gcloud secrets versions add github-runner-pat --data-file=-
```
3. Run `pulumi up` to create the runner VM
3. Run `pulumi up` to create the runner VMs
4. Verify the runner registered:
4. Verify the runners registered:
- Go to https://github.com/nomadkaraoke/karaoke-gen/settings/actions/runners
- You should see `gcp-runner-github-runner` with status "Idle"
- You should see `gcp-runner-github-runner-1` through `gcp-runner-github-runner-5`
### Usage in Workflows
Jobs that need more disk space use the self-hosted runner:
Jobs that need more disk space use the self-hosted runners:
```yaml
jobs:
@@ -299,27 +300,37 @@ jobs:
# ... rest of job
```
GitHub automatically distributes jobs across available runners.
### Troubleshooting
**Runner not appearing in GitHub:**
```bash
# Check startup logs
gcloud compute ssh github-runner --zone=us-central1-a -- \
# Check startup logs (replace N with runner number 1-3)
gcloud compute ssh github-runner-N --zone=us-central1-a -- \
"sudo cat /var/log/github-runner-startup.log"
```
**Re-register runner:**
```bash
# Restart the VM to re-run startup script
gcloud compute instances reset github-runner --zone=us-central1-a
gcloud compute instances reset github-runner-N --zone=us-central1-a
```
**Check runner service status:**
```bash
gcloud compute ssh github-runner --zone=us-central1-a -- \
gcloud compute ssh github-runner-N --zone=us-central1-a -- \
"sudo systemctl status actions.runner.*"
```
**List all runners:**
```bash
for i in 1 2 3 4 5; do
echo "=== github-runner-$i ==="
gcloud compute instances describe github-runner-$i --zone=us-central1-a --format='get(status)'
done
```
## Next Steps
1. Run `pulumi up` to create infrastructure
+89 -54
View File
@@ -1255,7 +1255,7 @@ apt-get install -y \\
# ==================== Docker ====================
echo "Installing Docker..."
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --batch --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
@@ -1264,23 +1264,38 @@ apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin do
systemctl enable docker
systemctl start docker
# ==================== Python 3.13 ====================
echo "Installing Python 3.13..."
add-apt-repository -y ppa:deadsnakes/ppa || true
apt-get update
apt-get install -y python3.13 python3.13-venv python3.13-dev python3-pip || {
# Fallback: build from source if PPA not available
echo "PPA not available, installing Python 3.13 from source..."
apt-get install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libreadline-dev libffi-dev libsqlite3-dev wget libbz2-dev
cd /tmp
wget https://www.python.org/ftp/python/3.13.0/Python-3.13.0.tgz
tar -xf Python-3.13.0.tgz
cd Python-3.13.0
./configure --enable-optimizations
make -j $(nproc)
make altinstall
ln -sf /usr/local/bin/python3.13 /usr/bin/python3.13
}
# ==================== Python 3.13 via pyenv ====================
# setup-python action on self-hosted runners needs Python in the tool cache
echo "Installing Python 3.13 via pyenv..."
# Install pyenv dependencies
apt-get install -y make build-essential libssl-dev zlib1g-dev \\
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \\
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
# Install pyenv system-wide
export PYENV_ROOT=/opt/pyenv
curl https://pyenv.run | bash
# Add to system profile for future sessions
cat > /etc/profile.d/pyenv.sh << 'PYENV_PROFILE'
export PYENV_ROOT=/opt/pyenv
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
PYENV_PROFILE
# Source pyenv for current session
export PATH="/opt/pyenv/bin:$PATH"
eval "$(/opt/pyenv/bin/pyenv init -)"
# Install Python 3.13
/opt/pyenv/bin/pyenv install 3.13.0
/opt/pyenv/bin/pyenv global 3.13.0
# Create symlinks for system-wide access
ln -sf /opt/pyenv/shims/python3.13 /usr/local/bin/python3.13
ln -sf /opt/pyenv/shims/python3 /usr/local/bin/python3
ln -sf /opt/pyenv/shims/pip3 /usr/local/bin/pip3
# ==================== Node.js 20 ====================
echo "Installing Node.js 20..."
@@ -1288,14 +1303,12 @@ curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs
# ==================== Java 21 (for Firestore emulator) ====================
echo "Installing Java 21..."
apt-get install -y openjdk-21-jdk || {
# Fallback: use Temurin
curl -fsSL https://packages.adoptium.net/artifactory/api/gpg/key/public | gpg --dearmor -o /usr/share/keyrings/adoptium.gpg
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb $(lsb_release -cs) main" > /etc/apt/sources.list.d/adoptium.list
apt-get update
apt-get install -y temurin-21-jdk
}
echo "Installing Java 21 via Temurin..."
# Use Temurin (Adoptium) - more reliable than Debian repos
curl -fsSL https://packages.adoptium.net/artifactory/api/gpg/key/public | gpg --batch --dearmor -o /usr/share/keyrings/adoptium.gpg
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb $(lsb_release -cs) main" > /etc/apt/sources.list.d/adoptium.list
apt-get update
apt-get install -y temurin-21-jdk
# ==================== FFmpeg ====================
echo "Installing FFmpeg..."
@@ -1308,7 +1321,7 @@ ln -sf /root/.local/bin/poetry /usr/local/bin/poetry
# ==================== Google Cloud SDK ====================
echo "Installing Google Cloud SDK..."
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --batch --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" > /etc/apt/sources.list.d/google-cloud-sdk.list
apt-get update
apt-get install -y google-cloud-cli google-cloud-cli-firestore-emulator
@@ -1375,6 +1388,23 @@ echo "Installing runner service..."
echo "GitHub Actions runner setup complete at $(date)"
echo "Runner registered with labels: self-hosted,linux,x64,gcp,large-disk"
# ==================== Setup Python in tool cache ====================
# setup-python action looks for Python in RUNNER_TOOL_CACHE/_tool/Python
echo "Setting up Python 3.13 in tool cache for setup-python action..."
TOOL_CACHE="/home/runner/actions-runner/_work/_tool"
mkdir -p $TOOL_CACHE/Python/3.13.0/x64
# Copy pyenv Python to tool cache
cp -r /opt/pyenv/versions/3.13.0/* $TOOL_CACHE/Python/3.13.0/x64/
# Create marker file that setup-python looks for
touch $TOOL_CACHE/Python/3.13.0/x64.complete
# Fix permissions - must own entire _work directory, not just _tool
chown -R runner:runner /home/runner/actions-runner/_work
echo "Python 3.13 added to tool cache"
# ==================== Docker cleanup cron ====================
# Prevent disk from filling up with old Docker images
echo "Setting up Docker cleanup cron..."
@@ -1388,35 +1418,40 @@ chmod +x /etc/cron.daily/docker-cleanup
echo "Setup complete!"
"""
# GitHub runner VM instance
github_runner_vm = gcp.compute.Instance(
"github-runner",
name="github-runner",
machine_type="e2-standard-4", # 4 vCPU, 16GB RAM - good for Docker builds
zone="us-central1-a",
boot_disk=gcp.compute.InstanceBootDiskArgs(
initialize_params=gcp.compute.InstanceBootDiskInitializeParamsArgs(
image="debian-cloud/debian-12",
size=200, # GB - plenty for Docker builds and caches
type="pd-ssd", # SSD for faster I/O
# GitHub runner VM instances (multiple runners for parallel job execution)
NUM_RUNNERS = 5
github_runner_vms = []
for i in range(1, NUM_RUNNERS + 1):
vm = gcp.compute.Instance(
f"github-runner-{i}",
name=f"github-runner-{i}",
machine_type="e2-standard-4", # 4 vCPU, 16GB RAM - good for Docker builds
zone="us-central1-a",
boot_disk=gcp.compute.InstanceBootDiskArgs(
initialize_params=gcp.compute.InstanceBootDiskInitializeParamsArgs(
image="debian-cloud/debian-12",
size=200, # GB - plenty for Docker builds and caches
type="pd-ssd", # SSD for faster I/O
),
),
),
network_interfaces=[gcp.compute.InstanceNetworkInterfaceArgs(
network="default",
access_configs=[gcp.compute.InstanceNetworkInterfaceAccessConfigArgs()], # Ephemeral IP is fine
)],
service_account=gcp.compute.InstanceServiceAccountArgs(
email=github_runner_sa.email,
scopes=["cloud-platform"],
),
metadata_startup_script=GITHUB_RUNNER_STARTUP_SCRIPT,
tags=["github-runner"],
allow_stopping_for_update=True,
# Ensure the secret exists before creating the VM
opts=pulumi.ResourceOptions(depends_on=[github_runner_pat_secret]),
)
network_interfaces=[gcp.compute.InstanceNetworkInterfaceArgs(
network="default",
access_configs=[gcp.compute.InstanceNetworkInterfaceAccessConfigArgs()], # Ephemeral IP is fine
)],
service_account=gcp.compute.InstanceServiceAccountArgs(
email=github_runner_sa.email,
scopes=["cloud-platform"],
),
metadata_startup_script=GITHUB_RUNNER_STARTUP_SCRIPT,
tags=["github-runner"],
allow_stopping_for_update=True,
# Ensure the secret exists before creating the VM
opts=pulumi.ResourceOptions(depends_on=[github_runner_pat_secret]),
)
github_runner_vms.append(vm)
# Export GitHub runner resources
pulumi.export("github_runner_vm_name", github_runner_vm.name)
pulumi.export("github_runner_vm_names", [vm.name for vm in github_runner_vms])
pulumi.export("github_runner_service_account", github_runner_sa.email)
@@ -0,0 +1,287 @@
# Agentic AI Lyrics Correction - User Guide
**Last Updated:** 2025-12-26
This guide explains how to use the agentic AI correction system for lyrics transcription.
## Overview
The agentic correction system uses LLMs to classify and correct transcription errors in lyrics. Unlike the rule-based correction handlers, the agentic system:
1. **Classifies gaps first** - Determines the type of error before attempting correction
2. **Routes to specialized handlers** - Each error type has its own handler
3. **Provides detailed reasoning** - Every correction includes confidence and explanation
4. **Supports human review** - Uncertain corrections are flagged for human verification
## Enabling Agentic Correction
### Environment Variables
```bash
# Required: Enable agentic mode
export USE_AGENTIC_AI=1
# Required: Specify model (provider/model format)
export AGENTIC_AI_MODEL="openai/gpt-4" # or "anthropic/claude-3-sonnet-20240229" or "ollama/llama3.2:latest"
# Optional: LLM Provider API keys (only needed for cloud providers)
export OPENAI_API_KEY="your-key" # For OpenAI models
export ANTHROPIC_API_KEY="your-key" # For Anthropic models
# Optional: Observability with Langfuse
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"
# Optional: Tuning parameters
export AGENTIC_TIMEOUT_SECONDS=30 # Request timeout
export AGENTIC_MAX_RETRIES=2 # Max retry attempts
export AGENTIC_CIRCUIT_THRESHOLD=3 # Failures before circuit opens
export AGENTIC_CIRCUIT_OPEN_SECONDS=60 # Circuit open duration
export DISABLE_LLM_CACHE=1 # Disable response caching
```
### Supported LLM Providers
| Provider | Model Format | API Key Required |
|----------|-------------|------------------|
| **OpenAI** | `openai/gpt-4`, `openai/gpt-4-turbo` | `OPENAI_API_KEY` |
| **Anthropic** | `anthropic/claude-3-sonnet-20240229` | `ANTHROPIC_API_KEY` |
| **Ollama** (local) | `ollama/llama3.2:latest`, `ollama/mistral:latest` | None |
### Using with Ollama (Local, Free)
1. Install Ollama: [https://ollama.ai](https://ollama.ai)
2. Pull a model: `ollama pull llama3.2:latest`
3. Set environment:
```bash
export USE_AGENTIC_AI=1
export AGENTIC_AI_MODEL="ollama/llama3.2:latest"
```
## Gap Classification Categories
The agentic system classifies gaps into 8 categories:
| Category | Description | Handler Action |
|----------|-------------|----------------|
| `PUNCTUATION_ONLY` | Style differences (quotes, punctuation) | No action |
| `SOUND_ALIKE` | Homophones ("out" → "now") | Replace word |
| `BACKGROUND_VOCALS` | Parenthesized backing vocals | Delete |
| `EXTRA_WORDS` | Filler words ("And", "But", "Well") | Delete |
| `REPEATED_SECTION` | Chorus/verse repetitions | Flag for human review |
| `COMPLEX_MULTI_ERROR` | Multiple error types | Flag for human review |
| `AMBIGUOUS` | Unclear without audio | Flag for human review |
| `NO_ERROR` | Matches reference lyrics | No action |
## Running with Agentic Correction
### CLI Usage
```bash
# Basic usage with agentic correction
USE_AGENTIC_AI=1 AGENTIC_AI_MODEL="openai/gpt-4" \
lyrics-transcriber /path/to/song.mp3 --artist "Artist" --title "Song"
# With Ollama (free, local)
USE_AGENTIC_AI=1 AGENTIC_AI_MODEL="ollama/llama3.2:latest" \
lyrics-transcriber /path/to/song.mp3 --artist "Artist" --title "Song"
# With debugging output
USE_AGENTIC_AI=1 AGENTIC_AI_MODEL="openai/gpt-4" \
lyrics-transcriber /path/to/song.mp3 --artist "Artist" --title "Song" --log_level debug
```
### As a Library
```python
from lyrics_transcriber import LyricsTranscriber
from lyrics_transcriber.core.controller import TranscriberConfig, LyricsConfig, OutputConfig
import os
# Enable agentic mode
os.environ["USE_AGENTIC_AI"] = "1"
os.environ["AGENTIC_AI_MODEL"] = "openai/gpt-4"
os.environ["OPENAI_API_KEY"] = "your-key"
transcriber = LyricsTranscriber(
audio_filepath="/path/to/song.mp3",
artist="Artist",
title="Title",
transcriber_config=TranscriberConfig(
audioshake_api_token="...",
),
lyrics_config=LyricsConfig(
genius_api_token="...",
),
output_config=OutputConfig(
output_dir="./out",
),
)
result = transcriber.process()
# Check agentic corrections
for correction in result.corrections:
if correction.handler == "AgenticCorrector":
print(f"Agentic correction: {correction.original_word} → {correction.corrected_word}")
print(f" Category: {correction.reason}") # Contains [CATEGORY] tag
print(f" Confidence: {correction.confidence}")
```
## Frontend Review UI
When agentic corrections are made, the review UI shows:
1. **Agentic Correction Metrics Panel** - Summary of corrections by category
2. **Category Icons** - Visual indicators for correction types
3. **Confidence Filters** - Filter by low (<60%) or high (≥80%) confidence
4. **Correction Detail Cards** - Click any corrected word to see:
- Original vs corrected text
- Category classification
- Confidence score
- AI reasoning
- Action buttons (Revert, Edit, Accept)
### Duration Timeline View
Toggle to duration view to see:
- Word timing bars proportional to duration
- Red warning indicators for abnormally long words (>2s)
- Color coding: Green (corrected), Orange (uncorrected gaps), Blue (anchors)
## Observability with Langfuse
When Langfuse is configured, you get:
- **Trace grouping** by session ID
- **Full prompts and responses**
- **Token counts and latency**
- **Cost estimates** for paid APIs
- **Model performance metrics**
Set up at [https://langfuse.com](https://langfuse.com)
## Human Feedback Collection
The system supports collecting feedback on corrections:
### API Endpoints
```bash
# Submit annotation
POST /api/v1/annotations
{
"audio_hash": "abc123",
"annotation_type": "SOUND_ALIKE",
"action_taken": "REPLACE",
"original_text": "out",
"corrected_text": "now",
"confidence": 5,
"reasoning": "Reference lyrics clearly show 'now'"
}
# Get annotations for song
GET /api/v1/annotations/{audio_hash}
# Get statistics
GET /api/v1/annotations/stats
```
### Annotation Storage
Annotations are stored in `{cache_dir}/correction_annotations.jsonl` in JSONL format for easy processing and training data export.
## Troubleshooting
### "Failed to import agentic modules"
Ensure required packages are installed:
```bash
pip install langchain langchain-openai langchain-anthropic langchain-ollama langfuse
```
### "Langfuse keys are set but initialization failed"
Check your Langfuse credentials:
```bash
echo $LANGFUSE_PUBLIC_KEY
echo $LANGFUSE_SECRET_KEY
```
### "Circuit breaker is open"
The system has detected too many failures. Wait 60 seconds or check:
- LLM API is accessible
- API keys are valid
- Model name is correct
### "Timeout waiting for LLM response"
Increase timeout:
```bash
export AGENTIC_TIMEOUT_SECONDS=60
```
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Agentic Correction Flow │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Gap Found │
│ │ │
│ ▼ │
│ 2. Classification Prompt → LLM │
│ │ │
│ ▼ │
│ 3. GapClassification (category, confidence, reasoning) │
│ │ │
│ ▼ │
│ 4. Handler Registry → Specialized Handler │
│ │ │
│ ├── SOUND_ALIKE → SoundAlikeHandler → Replace │
│ ├── BACKGROUND_VOCALS → BackgroundVocalsHandler → Delete│
│ ├── EXTRA_WORDS → ExtraWordsHandler → Delete │
│ ├── PUNCTUATION_ONLY → PunctuationHandler → No Action │
│ ├── NO_ERROR → NoErrorHandler → No Action │
│ └── AMBIGUOUS/COMPLEX/REPEATED → Flag for Human Review │
│ │ │
│ ▼ │
│ 5. CorrectionProposal(s) returned │
│ │ │
│ ▼ │
│ 6. Adapter → WordCorrection(s) for frontend │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Files Reference
| File | Purpose |
|------|---------|
| `correction/agentic/agent.py` | Main AgenticCorrector class |
| `correction/agentic/adapter.py` | Converts proposals to WordCorrection |
| `correction/agentic/router.py` | Model selection logic |
| `correction/agentic/handlers/` | Category-specific handlers |
| `correction/agentic/models/schemas.py` | Pydantic models |
| `correction/agentic/prompts/classifier.py` | Classification prompt |
| `correction/agentic/providers/` | LLM provider implementations |
| `correction/feedback/` | Human feedback collection |
## Contributing
When adding new gap categories or handlers:
1. Add category to `GapCategory` enum in `models/schemas.py`
2. Create handler in `handlers/` extending `BaseHandler`
3. Register handler in `handlers/registry.py`
4. Add few-shot examples to `prompts/classifier.py`
5. Add tests in `tests/unit/correction/agentic/`
## See Also
- [AGENTIC_IMPLEMENTATION_STATUS.md](./AGENTIC_IMPLEMENTATION_STATUS.md) - Implementation progress
- [AGENTIC_UI_IMPROVEMENTS_COMPLETE.md](./AGENTIC_UI_IMPROVEMENTS_COMPLETE.md) - Frontend features
- [LANGCHAIN_MIGRATION.md](./LANGCHAIN_MIGRATION.md) - LangChain architecture details
@@ -1,450 +1,187 @@
# Agentic Correction System - Implementation Status
**Last Updated:** 2025-10-27
**Status:** Phase 1 and Phase 2 (Backend) Complete
**Last Updated:** 2025-12-26
**Status:** Core Implementation Complete, Ready for User Testing
## Overview
This document tracks the implementation of the classification-first agentic correction system with human feedback loop as specified in the plan.
The agentic AI correction system for lyrics transcription is now feature-complete for the core workflow. This document tracks implementation status and what's available for testing.
## ✅ Completed
## ✅ Completed Features
### Phase 1: Classification-First Correction Workflow
#### 1.1 Gap Classification Schema ✅
**File:** `lyrics_transcriber/correction/agentic/models/schemas.py`
#### Gap Classification System
- **8 classification categories**: PUNCTUATION_ONLY, SOUND_ALIKE, BACKGROUND_VOCALS, EXTRA_WORDS, REPEATED_SECTION, COMPLEX_MULTI_ERROR, AMBIGUOUS, NO_ERROR
- **Category-specific handlers** for each type with appropriate actions
- **Classification prompt** with few-shot examples
- **Handler registry pattern** for extensibility
- Added `GapCategory` enum with 8 categories:
- `PUNCTUATION_ONLY`: Style differences only
- `SOUND_ALIKE`: Homophones and similar-sounding errors
- `BACKGROUND_VOCALS`: Transcribed backing vocals in parentheses
- `EXTRA_WORDS`: Filler words like "And", "But"
- `REPEATED_SECTION`: Chorus repetitions
- `COMPLEX_MULTI_ERROR`: Large gaps with multiple error types
- `AMBIGUOUS`: Unclear without audio
- `NO_ERROR`: Matches at least one reference source
#### LLM Provider Support
- **OpenAI** (GPT-4, GPT-4-turbo)
- **Anthropic** (Claude 3 Sonnet)
- **Ollama** (local, free - llama3.2, mistral)
- **LangChain integration** for unified API
- Added `GapClassification` model with fields:
- `gap_id`: Unique identifier
- `category`: Gap category
- `confidence`: 0-1 score
- `reasoning`: Explanation
- `suggested_handler`: Handler recommendation
#### Reliability Features
- **Circuit breaker** pattern for failure protection
- **Configurable retries** and timeouts
- **Graceful degradation** to FLAG for human review
- Updated `CorrectionProposal` with:
- `gap_category`: Classification category
- `requires_human_review`: Flag for manual review
- `artist`, `title`: Song metadata
- Added "NoAction" and "Flag" to action types
### Phase 2: Human Feedback Collection
#### 1.2 Classification Prompt ✅
**File:** `lyrics_transcriber/correction/agentic/prompts/classifier.py`
#### Backend API
- `POST /api/v1/annotations` - Save correction annotations
- `GET /api/v1/annotations/{audio_hash}` - Get annotations for song
- `GET /api/v1/annotations/stats` - Aggregated statistics
- Created `build_classification_prompt()` function that:
- Includes gap text, context, and reference lyrics from all sources
- Includes artist/title for proper noun context
- Provides few-shot examples from `gaps_review.yaml`
- Requests structured JSON output matching `GapClassification` schema
#### Storage
- JSONL format for easy parsing and export
- Automatic datetime serialization
- Training data export capability
- Implemented `load_few_shot_examples()` with:
- Dynamic loading from `examples.yaml` (if exists)
- Hardcoded fallback examples covering all categories
- Examples extracted directly from your manual gap annotations
### Phase 3: Frontend UI Enhancements
#### 1.3 Category-Specific Handlers ✅
**Files:** `lyrics_transcriber/correction/agentic/handlers/`
#### Agentic Correction Metrics Panel
- Shows correction counts by category
- Visual icons for each gap category
- Confidence distribution filtering
Implemented 8 handler classes:
#### Correction Detail Cards
- Click any corrected word to see details
- Shows original vs corrected text
- Displays AI reasoning and confidence
- Action buttons: Revert, Edit, Accept
1. **`PunctuationHandler`**: Returns NO_ACTION for style differences
2. **`NoErrorHandler`**: Returns NO_ACTION when reference matches
3. **`BackgroundVocalsHandler`**: Proposes DELETE for parenthesized content
4. **`ExtraWordsHandler`**: Detects and removes filler words
5. **`SoundAlikeHandler`**: Extracts replacement from reference context
6. **`RepeatedSectionHandler`**: Flags for human review
7. **`ComplexMultiErrorHandler`**: Flags complex gaps
8. **`AmbiguousHandler`**: Flags unclear cases
#### Duration Timeline View
- Toggle to see word timing proportions
- Warning indicators for abnormally long words (>2s)
- Color coding: Green (corrected), Orange (gaps), Blue (anchors)
- Created `HandlerRegistry` for mapping categories to handlers
- All handlers extend `BaseHandler` abstract class
- Handlers return `CorrectionProposal` objects with metadata
### Phase 4: Observability
#### 1.4 AgenticCorrector Workflow ✅
**File:** `lyrics_transcriber/correction/agentic/agent.py`
#### Langfuse Integration
- Automatic trace grouping by session
- Full prompt/response logging
- Token counts and latency metrics
- Cost estimation for paid APIs
- Added `classify_gap()` method:
- Builds classification prompt
- Calls AI provider for classification
- Returns `GapClassification` or None
### Phase 5: Testing Infrastructure
- Added `propose_for_gap()` method implementing two-step workflow:
1. Classify the gap using LLM
2. Route to appropriate handler based on category
3. Handler generates correction proposals
4. Add metadata (artist, title, category) to proposals
5. Handle errors gracefully with fallback to FLAG
#### Unit Tests
- 15 tests covering agentic handlers and feedback store
- All tests passing
- Kept legacy `propose()` method marked as deprecated
#### E2E Tests (Playwright)
- Test fixtures with mock agentic correction data
- UI component tests
- File upload flow tests
- 7 tests passing
#### 1.5 LyricsCorrector Integration ✅
**File:** `lyrics_transcriber/correction/corrector.py`
## 🚧 Future Enhancements (Not Blocking)
- Updated agentic correction section to:
- Prepare structured gap data (words with IDs, times)
- Extract context (10 preceding/following words)
- Build reference contexts from all sources
- Pass artist and title from metadata
- Call new `propose_for_gap()` method instead of old prompt-based approach
### Frontend Annotation Modal
- `CorrectionAnnotationModal.tsx` - UI for collecting human feedback on corrections
- Would trigger after user edits for training data collection
- Low priority - can use raw API for now
### Phase 2: Human Feedback Collection System (Backend)
### Analysis Scripts
- `scripts/analyze_annotations.py` - Generate reports from collected annotations
- `scripts/generate_few_shot_examples.py` - Auto-generate prompt examples from feedback
#### 2.1 Correction Annotation Schema ✅
**File:** `lyrics_transcriber/correction/feedback/schemas.py`
### A/B Testing Framework
- Compare different prompt versions
- Track model performance over time
- Created `CorrectionAnnotationType` enum (9 types including MANUAL_EDIT)
- Created `CorrectionAction` enum (7 actions: NO_ACTION, REPLACE, DELETE, INSERT, MERGE, SPLIT, FLAG)
## Quick Start
- Created `CorrectionAnnotation` model with:
- Unique `annotation_id` (UUID)
- Song identification (`audio_hash`, `artist`, `title`)
- Classification (`annotation_type`, `action_taken`)
- Content (`original_text`, `corrected_text`)
- Human metadata (`confidence` 1-5, `reasoning` min 10 chars)
- Agentic comparison (`agentic_proposal`, `agentic_category`, `agentic_agreed`)
- Reference tracking (`reference_sources_consulted`)
- Session tracking (`session_id`, `timestamp`)
### Enable Agentic Correction
- Created `AnnotationStatistics` for aggregated metrics
#### 2.2 Feedback Storage Backend ✅
**File:** `lyrics_transcriber/correction/feedback/store.py`
- Implemented `FeedbackStore` class with JSONL storage:
- File: `{cache_dir}/correction_annotations.jsonl`
- One annotation per line for easy appending
- Automatic datetime serialization/deserialization
- Methods implemented:
- `save_annotation()`: Save single annotation
- `save_annotations()`: Batch save
- `get_all_annotations()`: Load all with error recovery
- `get_annotations_by_song()`: Filter by audio hash
- `get_annotations_by_category()`: Filter by type
- `get_statistics()`: Aggregate metrics (counts, averages, patterns)
- `export_to_training_data()`: Export high-confidence annotations for fine-tuning
#### 2.3 Backend API Endpoints ✅
**File:** `lyrics_transcriber/review/server.py`
- Initialized `NewFeedbackStore` in `ReviewServer.__init__()`
- Added 3 new API endpoints:
- `POST /api/v1/annotations`: Save annotation with validation
- `GET /api/v1/annotations/{audio_hash}`: Get annotations for song
- `GET /api/v1/annotations/stats`: Get aggregated statistics
- All endpoints include proper error handling and HTTP status codes
## 🚧 In Progress / Not Started
### Phase 2: Human Feedback Collection (Frontend)
#### 2.4 UI Annotation Modal Component ⏳
**File:** `lyrics_transcriber/frontend/src/components/CorrectionAnnotationModal.tsx` (to create)
**Required:** React modal component with:
- Annotation type dropdown (9 categories)
- Confidence slider (1-5 scale)
- Reasoning textarea (required, min 10 chars)
- Display of agentic AI suggestion (if applicable)
- Display of reference lyrics context
- "Save & Continue" and "Skip" buttons
- Local state management until final submission
#### 2.5 Edit Workflow Integration ⏳
**Files:**
- `lyrics_transcriber/frontend/src/components/EditModal.tsx`
- `lyrics_transcriber/frontend/src/components/EditWordList.tsx`
**Required:** Wrap edit actions to trigger annotation modal:
- Show modal after user confirms word edit/delete/merge/split
- Collect annotation data in React state
- Submit all annotations on "Finish Review"
- Add settings toggle for "Enable correction annotations"
#### 2.6 Frontend Types and API Client ⏳
**Files:**
- `lyrics_transcriber/frontend/src/types.ts`
- `lyrics_transcriber/frontend/src/api.ts`
**Required:**
- Add `CorrectionAnnotation` TypeScript interface
- Add `submitAnnotations()` method to API client
- Add `getAnnotationStats()` method
### Phase 3: Continuous Improvement Infrastructure
#### 3.1 Analysis Scripts ⏳
**File:** `scripts/analyze_annotations.py` (to create)
**Required:** Python script that:
- Loads all annotations from JSONL
- Generates Markdown report with:
- Most common error categories
- Agentic AI accuracy by category
- Frequently mis-heard words/phrases
- Cases where reference lyrics were wrong
- Outputs to `CORRECTION_ANALYSIS.md`
#### 3.2 Few-Shot Example Generator ⏳
**File:** `scripts/generate_few_shot_examples.py` (to create)
**Required:** Python script that:
- Selects high-confidence annotations (confidence >= 4)
- Formats as YAML prompt examples
- Outputs to `lyrics_transcriber/correction/agentic/prompts/examples.yaml`
- Can be run periodically to update classifier
#### 3.3 Classifier Dynamic Examples ⏳
**File:** `lyrics_transcriber/correction/agentic/prompts/classifier.py` (update)
**Required:**
- Already has `load_few_shot_examples()` infrastructure
- Will automatically load from `examples.yaml` if file exists
- No changes needed - just need to generate the YAML file
#### 3.4 Feedback Loop Documentation ⏳
**File:** `HUMAN_FEEDBACK_LOOP.md` (to create)
**Required:** Document:
- How to use annotation collection in UI
- How to run analysis scripts
- How to regenerate few-shot examples
- How to evaluate improvement over time
- Path to fine-tuning custom model with RLHF
### Phase 4: Testing and Validation
#### 4.1 Unit Tests ⏳
**File:** `tests/unit/correction/test_classifier.py` (to create)
**Required:** Test cases for:
- Gap classifier with examples from `gaps_review.yaml`
- Each category handler
- Edge cases (ambiguous, no reference match)
#### 4.2 Integration Tests ⏳
**File:** `tests/integration/test_agentic_workflow.py` (update)
**Required:** End-to-end tests:
- Classification → correction flow
- Use Time Bomb song as fixture
- Verify correct handlers are invoked
- Verify FLAG actions for ambiguous cases
#### 4.3 Feedback System Tests ⏳
**File:** `tests/unit/correction/test_feedback_store.py` (to create)
**Required:** Test cases for:
- Save and retrieve annotations
- JSONL format correctness
- Statistics generation
- Training data export
## Testing the Implementation
### Backend Testing (Can be done now)
1. **Test Classification Workflow:**
```bash
USE_AGENTIC_AI=1 python -m lyrics_transcriber.cli.cli_main Time-Bomb.flac \
--artist "Rancid" --title "Time Bomb"
# Required
export USE_AGENTIC_AI=1
export AGENTIC_AI_MODEL="openai/gpt-4" # or "anthropic/claude-3-sonnet-20240229" or "ollama/llama3.2:latest"
# For cloud providers
export OPENAI_API_KEY="your-key" # For OpenAI
export ANTHROPIC_API_KEY="your-key" # For Anthropic
# Optional: Observability
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
```
Expected: Gaps will be classified into categories, handlers will propose corrections
### Run with Agentic Correction
2. **Test Annotation Storage:**
```python
from lyrics_transcriber.correction.feedback.store import FeedbackStore
from lyrics_transcriber.correction.feedback.schemas import CorrectionAnnotation, CorrectionAnnotationType, CorrectionAction
store = FeedbackStore("cache")
annotation = CorrectionAnnotation(
audio_hash="test123",
annotation_type=CorrectionAnnotationType.SOUND_ALIKE,
action_taken=CorrectionAction.REPLACE,
original_text="out",
corrected_text="now",
confidence=5.0,
reasoning="Reference lyrics confirm it should be 'now'",
artist="Rancid",
title="Time Bomb",
session_id="test_session"
)
store.save_annotation(annotation)
stats = store.get_statistics()
print(stats)
```
3. **Test API Endpoints:**
```bash
# Start review server and test endpoints
curl -X POST http://localhost:8000/api/v1/annotations \
-H "Content-Type: application/json" \
-d '{"audio_hash": "test", "annotation_type": "sound_alike", ...}'
# With OpenAI
USE_AGENTIC_AI=1 AGENTIC_AI_MODEL="openai/gpt-4" \
lyrics-transcriber /path/to/song.mp3 --artist "Artist" --title "Song"
curl http://localhost:8000/api/v1/annotations/stats
# With local Ollama (free)
USE_AGENTIC_AI=1 AGENTIC_AI_MODEL="ollama/llama3.2:latest" \
lyrics-transcriber /path/to/song.mp3 --artist "Artist" --title "Song"
```
### Frontend Testing (Once UI is implemented)
### Run Frontend Tests
1. Open review UI
2. Make a correction (edit/delete/merge word)
3. Annotation modal should appear
4. Fill in annotation details
5. Click "Save & Continue"
6. Make more corrections
7. Click "Finish Review"
8. Check `cache/correction_annotations.jsonl` for saved data
```bash
cd lyrics_transcriber_temp/lyrics_transcriber/frontend
yarn install
yarn playwright install --with-deps chromium
yarn test
```
## Architecture Decisions
## File Reference
### Classification-First Approach
- **Why:** Breaks complex problem into simpler steps
- **Benefit:** Each handler can focus on one error type
- **Trade-off:** Two LLM calls per gap (classification + handler logic), but handlers are deterministic
| Component | Files |
|-----------|-------|
| Core Agent | `correction/agentic/agent.py` |
| Handlers | `correction/agentic/handlers/*.py` |
| Classification Prompt | `correction/agentic/prompts/classifier.py` |
| LLM Providers | `correction/agentic/providers/*.py` |
| Feedback Storage | `correction/feedback/store.py` |
| API Endpoints | `review/server.py` |
| Frontend Metrics | `frontend/src/components/AgenticCorrectionMetrics.tsx` |
| Frontend Detail Card | `frontend/src/components/CorrectionDetailCard.tsx` |
| E2E Tests | `frontend/e2e/agentic-corrections.spec.ts` |
### JSONL Storage
- **Why:** Simple, append-only, no database required
- **Benefit:** Easy to parse, version control friendly, portable
- **Trade-off:** Not suitable for millions of annotations (but we expect hundreds/thousands)
## Architecture
### Fail-Fast on Errors
- **Why:** Better to flag for human review than make wrong correction
- **Benefit:** Maintains transcription quality
- **Trade-off:** More human review needed initially (improves over time with feedback)
```
┌─────────────────────────────────────────────────────────────┐
│ Agentic Correction Flow │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Gap Found in Transcription │
│ │ │
│ ▼ │
│ 2. Classification Prompt → LLM │
│ │ │
│ ▼ │
│ 3. GapClassification (category, confidence, reasoning) │
│ │ │
│ ▼ │
│ 4. Handler Registry → Specialized Handler │
│ │ │
│ ├── SOUND_ALIKE → SoundAlikeHandler → Replace │
│ ├── BACKGROUND_VOCALS → BackgroundVocalsHandler → Delete│
│ ├── EXTRA_WORDS → ExtraWordsHandler → Delete │
│ ├── PUNCTUATION_ONLY → PunctuationHandler → No Action │
│ ├── NO_ERROR → NoErrorHandler → No Action │
│ └── AMBIGUOUS/COMPLEX/REPEATED → Flag for Human Review │
│ │ │
│ ▼ │
│ 5. CorrectionProposal(s) returned │
│ │ │
│ ▼ │
│ 6. Adapter → WordCorrection(s) for frontend │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Handler Registry Pattern
- **Why:** Easy to add new category handlers
- **Benefit:** Extensible, testable, follows Open/Closed Principle
- **Trade-off:** Slightly more complex than if/else routing
## Next Steps
**Priority 1 (Required for feedback loop):**
1. Implement `CorrectionAnnotationModal.tsx`
2. Integrate modal into edit workflow
3. Update frontend types and API client
4. Test end-to-end annotation collection
**Priority 2 (Analysis and improvement):**
5. Create `analyze_annotations.py` script
6. Create `generate_few_shot_examples.py` script
7. Generate initial `examples.yaml` from collected data
8. Document feedback loop in `HUMAN_FEEDBACK_LOOP.md`
**Priority 3 (Validation):**
9. Write unit tests for classifiers and handlers
10. Write integration tests for full workflow
11. Write tests for feedback store
## Files Created
### Python Backend
- `lyrics_transcriber/correction/agentic/models/schemas.py` (updated)
- `lyrics_transcriber/correction/agentic/prompts/__init__.py` (new)
- `lyrics_transcriber/correction/agentic/prompts/classifier.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/__init__.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/base.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/punctuation.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/no_error.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/background_vocals.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/extra_words.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/sound_alike.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/repeated_section.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/complex_multi_error.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/ambiguous.py` (new)
- `lyrics_transcriber/correction/agentic/handlers/registry.py` (new)
- `lyrics_transcriber/correction/agentic/agent.py` (updated)
- `lyrics_transcriber/correction/corrector.py` (updated)
- `lyrics_transcriber/correction/feedback/__init__.py` (new)
- `lyrics_transcriber/correction/feedback/schemas.py` (new)
- `lyrics_transcriber/correction/feedback/store.py` (new)
- `lyrics_transcriber/review/server.py` (updated)
### Documentation
- `AGENTIC_IMPLEMENTATION_STATUS.md` (this file)
## Known Issues / Limitations
1. **Classification accuracy depends on LLM quality**
- Solution: Use better models (GPT-4, Claude Sonnet) for classification
- Solution: Collect human feedback to improve prompts
2. **SoundAlikeHandler may fail to extract replacement**
- Fallback: Flags for human review
- Future: Use fuzzy matching and phonetic algorithms
3. **No A/B testing framework yet**
- Can't compare different prompt versions easily
- Future enhancement: Track model performance over time
4. **Frontend not implemented**
- Can't collect human feedback yet
- Priority for next implementation phase
## Fixes Applied
### Enum Value Case Mismatch (2025-10-27)
**Issue:** LLM was returning category values in uppercase (e.g., `"SOUND_ALIKE"`) but Pydantic enum expected lowercase with underscores (e.g., `"sound_alike"`), causing validation errors.
**Fix:** Updated all enum values in `GapCategory` and `CorrectionAnnotationType` to use uppercase format that LLMs naturally return. Also updated the prompt to explicitly show the expected format.
**Files Changed:**
- `lyrics_transcriber/correction/agentic/models/schemas.py`
- `lyrics_transcriber/correction/feedback/schemas.py`
- `lyrics_transcriber/correction/agentic/prompts/classifier.py`
### JSON Parsing with Invalid Escapes (2025-10-27)
**Issue:** LLM responses contained invalid JSON escape sequences like `\'` (e.g., in `"out, I\'m"`), which caused JSON parsing to fail. Python's json.loads() only allows specific escape sequences (`\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, `\t`).
**Fix:** Enhanced `ResponseParser` to automatically fix common JSON issues before parsing:
- Replace invalid `\'` with `'` (single quotes don't need escaping in JSON)
- Remove trailing commas before `}` or `]`
- Retry parsing after fixes before falling back to raw response
**Files Changed:**
- `lyrics_transcriber/correction/agentic/providers/response_parser.py`
**Result:** System now handles LLM responses with imperfect JSON formatting gracefully.
## Performance Considerations
- **Two LLM calls per gap:** Classification + handler logic (if handler needs LLM)
- Most handlers are deterministic (no LLM call)
- Only `SoundAlikeHandler` might need LLM for complex cases
- **JSONL file grows linearly:**
- ~1KB per annotation
- 1000 annotations = ~1MB
- Should be fine for years of use
- **Frontend modal after each edit:**
- Could be annoying for many edits
- Consider: Batch annotation at end instead
- Add "Skip" button and settings toggle
## Success Metrics
Once implemented, track:
1. **Annotation Collection Rate:** % of corrections that get annotated
2. **Agentic Agreement Rate:** % where human agrees with AI proposal
3. **Classification Accuracy:** % correctly classified by category (human-verified)
4. **Correction Quality Over Time:** Track accuracy improvements as more feedback collected
5. **Human Review Rate:** % of gaps flagged vs auto-corrected (should decrease over time)
## References
- Original plan: `.cursor/plans/agentic-correction-system-*.plan.md`
- Manual gap annotations: `gaps_review.yaml`
- Test song: `Time-Bomb.flac` by Rancid
## See Also
- [AGENTIC_CORRECTION_GUIDE.md](./AGENTIC_CORRECTION_GUIDE.md) - User guide for the agentic system
- [AGENTIC_UI_IMPROVEMENTS_COMPLETE.md](./AGENTIC_UI_IMPROVEMENTS_COMPLETE.md) - Frontend feature details
- [LANGCHAIN_MIGRATION.md](./LANGCHAIN_MIGRATION.md) - LangChain architecture details
@@ -1,165 +0,0 @@
# Agentic AI Corrector - Next Steps
## Current Status ✅
- ✅ Sessions working - All traces grouped in Langfuse
- ✅ LangChain + LangGraph integrated
- ✅ Circuit breaker and retry logic
- ✅ Observability with Langfuse
- ✅ Gap extraction code added to `corrector.py`
## Problems Identified 🔍
### 1. Adapter Always Returns 0 Corrections
**Root Cause**: LLM proposals don't include `word_id`, so the adapter can't map them to actual words.
**Why**: The prompt doesn't:
- Provide structured word data with IDs
- Request word IDs in the response
- Give enough context for unambiguous identification
### 2. LLM Proposals Are Too Aggressive
**Issue**: LLM suggests corrections for stylistic differences (quotes, spacing, etc.) that don't need fixing.
**Why**: No classification step - the LLM jumps straight to proposing actions without first determining if the gap needs correction.
### 3. Proposals Lack Actionable Detail
**Issue**: Responses like `"replacement_text": "I'm gonna"` don't specify:
- Which word to replace
- Where to insert it
- What to do with surrounding words
**Why**: The prompt is too vague and doesn't enforce structured, unambiguous actions.
## Next Steps 📋
### Step 1: Extract Gap Data (YOU)
Run this command:
```bash
DUMP_GAPS=1 USE_AGENTIC_AI=0 python -m lyrics_transcriber.cli.cli_main Time-Bomb.flac \
--artist "Rancid" --title "Time Bomb"
```
This creates `gaps_review.yaml` with all gap data.
### Step 2: Manual Annotation (YOU)
Open `gaps_review.yaml` and for each gap (recommend doing 10-20), fill in:
```yaml
annotations:
your_decision: "Replace 'out,' with 'now'"
action_type: "REPLACE"
target_word_ids: ["w7"]
replacement_text: "now"
notes: "Transcription error - reference says 'now'"
```
See `HOW_TO_EXTRACT_GAPS.md` for detailed examples.
### Step 3: Workflow Redesign (ME)
Based on your annotations, I'll:
#### A. Fix the Prompt
```python
prompt = f"""You are correcting transcription errors in song lyrics.
TRANSCRIBED WORDS (may contain errors):
{json.dumps([{"id": w.id, "text": w.text, "time": w.start_time} for w in gap_words], indent=2)}
REFERENCE LYRICS CONTEXT:
{ref_text}
TASK:
1. First, classify if this gap needs correction:
- Compare transcribed words to reference
- Ignore stylistic differences (quotes, punctuation, spacing)
- Only flag actual transcription errors
2. If correction is needed, propose actions with specific word IDs:
- Use the "id" field to identify which word(s) to operate on
- Be precise and unambiguous
OUTPUT FORMAT:
{{
"needs_correction": true/false,
"reason": "Brief explanation",
"proposals": [
{{
"target_word_ids": ["w7"], // Which words to operate on
"action": "REPLACE",
"replacement_text": "now",
"confidence": 0.95,
"reason": "Transcription error"
}}
]
}}
"""
```
#### B. Update the Schema
```python
class CorrectionDecision(BaseModel):
needs_correction: bool
reason: str
proposals: List[CorrectionProposal]
class CorrectionProposal(BaseModel):
target_word_ids: List[str] # Now required!
action: Literal["REPLACE", "DELETE", "INSERT", "MERGE", "SPLIT", "NO_ACTION"]
replacement_text: Optional[str]
insert_position: Optional[Literal["before", "after"]] # For INSERT
confidence: float
reason: str
```
#### C. Fix the Adapter
```python
def adapt_proposals_to_word_corrections(
decision: CorrectionDecision,
word_map: Dict[str, Word],
linear_position_map: Dict[str, int],
) -> List[WordCorrection]:
"""Convert proposals with proper word ID mapping."""
if not decision.needs_correction:
return [] # Early exit
results = []
for proposal in decision.proposals:
# Now we have target_word_ids!
for word_id in proposal.target_word_ids:
if word_id not in word_map:
logger.warning(f"Word ID {word_id} not found in word_map")
continue
word = word_map[word_id]
# ... create WordCorrection with proper mapping
```
### Step 4: Testing (ME)
- Update integration tests
- Verify proposals → corrections flow works
- Check Langfuse traces show proper structure
### Step 5: Iteration (BOTH)
- Run on Time-Bomb with new workflow
- Compare results to your annotations
- Refine prompt/schema based on accuracy
## Expected Outcome 🎯
After this iteration:
- ✅ LLM proposals include word IDs
- ✅ Adapter successfully creates corrections
- ✅ Classification step reduces false positives
- ✅ Actions are precise and unambiguous
- ✅ You see actual corrections being applied
## Timeline
- **Now**: Gap extraction code is ready
- **You**: Extract and annotate gaps (30-60 min?)
- **Me**: Redesign based on annotations (1-2 hours)
- **Together**: Test and iterate (30 min)
Ready to extract those gaps? 🚀
@@ -21,3 +21,4 @@ dist-ssr
*.njsproj
*.sln
*.sw?
test-results/
@@ -0,0 +1,207 @@
import { test, expect } from '@playwright/test';
import * as path from 'path';
import { fileURLToPath } from 'url';
// Get __dirname equivalent in ESM
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
/**
* E2E tests for the agentic correction workflow in the lyrics transcriber frontend.
*
* These tests verify:
* 1. The UI loads correctly with agentic correction data
* 2. The AgenticCorrectionMetrics panel displays correctly
* 3. Corrected words are highlighted and clickable
* 4. The CorrectionDetailCard shows proper information
*/
// Helper function to load fixture data
async function loadFixtureData(page: import('@playwright/test').Page) {
const fixturePath = path.join(__dirname, 'fixtures', 'agentic-correction-data.json');
// Create a file chooser promise before clicking
const fileChooserPromise = page.waitForEvent('filechooser');
// Click the Load File button
await page.getByRole('button', { name: /load file/i }).click();
// Handle the file chooser
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(fixturePath);
// Wait for data to load by asserting expected content appears
await expect(page.getByText('Hello,')).toBeVisible({ timeout: 5000 });
}
test.describe('Agentic Correction Workflow', () => {
test.beforeEach(async ({ page }) => {
// Navigate to the app
await page.goto('/');
// Wait for the initial load
await expect(page.getByText('Lyrics Correction Review')).toBeVisible();
});
test('should load the app in read-only mode', async ({ page }) => {
// Verify read-only mode alert is shown
await expect(page.getByText('Running in read-only mode')).toBeVisible();
// Verify Load File button is present
await expect(page.getByRole('button', { name: /load file/i })).toBeVisible();
});
test('should load correction data from JSON file', async ({ page }) => {
await loadFixtureData(page);
// Verify we're no longer in loading state
await expect(page.getByText('Loading Lyrics Correction Review...')).not.toBeVisible();
// Verify expected content from fixture is visible
await expect(page.getByText('Hello,')).toBeVisible();
});
test('should render transcription view after loading data', async ({ page }) => {
await loadFixtureData(page);
// Wait for transcription content to render
await expect(page.getByText('Hello,')).toBeVisible();
// Verify the Corrected Transcription header is visible
await expect(page.getByText('Corrected Transcription')).toBeVisible();
});
});
test.describe('UI Components', () => {
test('should show Load File button on initial load', async ({ page }) => {
await page.goto('/');
// The Load File button should be visible
const loadButton = page.getByRole('button', { name: /load file/i });
await expect(loadButton).toBeVisible();
// Should have upload icon
const uploadIcon = page.locator('svg[data-testid="UploadFileIcon"]');
await expect(uploadIcon).toBeVisible();
});
test('should show read-only mode banner', async ({ page }) => {
await page.goto('/');
// The read-only alert should be visible
await expect(page.getByRole('alert')).toBeVisible();
await expect(page.getByText(/read-only mode/i)).toBeVisible();
});
test('should have correction metrics component', async ({ page }) => {
await page.goto('/');
// The page structure should be there with Paper components
const metricsSection = page.locator('.MuiPaper-root');
await expect(metricsSection.first()).toBeVisible();
});
});
test.describe('File Upload Flow', () => {
test('should open file dialog when clicking Load File', async ({ page }) => {
await page.goto('/');
// Set up listener for file chooser
let fileChooserOpened = false;
page.on('filechooser', () => {
fileChooserOpened = true;
});
// Click the button
const loadButton = page.getByRole('button', { name: /load file/i });
await loadButton.click();
// Wait for the file chooser event to be processed
await page.waitForEvent('filechooser', { timeout: 5000 }).catch(() => {
// Event already fired
});
// Verify file chooser was triggered
expect(fileChooserOpened).toBe(true);
});
});
test.describe('Review Mode', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await expect(page.getByText('Lyrics Correction Review')).toBeVisible();
});
test('should show Review Mode toggle when agentic data is loaded', async ({ page }) => {
await loadFixtureData(page);
// Wait for content to load
await expect(page.getByText('Hello,')).toBeVisible();
// The Review Mode toggle should be visible when agentic corrections are present
// It will appear as "Review Off" chip initially (only in non-read-only mode with agentic data)
// Note: In read-only mode, the toggle won't appear
const reviewChip = page.getByText(/Review Off|Review Mode/i);
const chipCount = await reviewChip.count();
// Log for debugging purposes (will show in test output)
if (chipCount === 0) {
// Review toggle only shows in edit mode, not read-only mode
// This is expected behavior when loading files in read-only mode
}
});
test('should show batch actions panel when Review Mode is enabled', async ({ page }) => {
await loadFixtureData(page);
// Wait for content to load
await expect(page.getByText('Hello,')).toBeVisible();
// Find the Review Mode toggle (only visible in non-read-only mode)
const reviewToggle = page.getByText(/Review Off/i);
if (await reviewToggle.isVisible({ timeout: 2000 }).catch(() => false)) {
await reviewToggle.click();
// Wait for the batch actions panel to appear
await expect(page.getByRole('button', { name: /Accept High Confidence/i })).toBeVisible({ timeout: 5000 });
await expect(page.getByRole('button', { name: /Accept All/i })).toBeVisible();
await expect(page.getByRole('button', { name: /Revert All/i })).toBeVisible();
}
});
test('should render corrected words with original text preview', async ({ page }) => {
await loadFixtureData(page);
// Wait for transcription content to load
await expect(page.getByText('Hello,')).toBeVisible();
// Verify correction-related content is rendered
// The corrected word "now" should be visible (from fixture: "you're" -> "now")
await expect(page.getByText('now')).toBeVisible();
});
test('should toggle Review Mode on and off', async ({ page }) => {
await loadFixtureData(page);
// Wait for content to load
await expect(page.getByText('Hello,')).toBeVisible();
// Find the Review toggle (only visible in non-read-only mode)
const reviewOff = page.getByText(/Review Off/i);
if (await reviewOff.isVisible({ timeout: 2000 }).catch(() => false)) {
// Click to enable Review Mode
await reviewOff.click();
// Should now show "Review Mode" label (in filled state)
await expect(page.getByText(/Review Mode/i).first()).toBeVisible();
// Click to disable Review Mode
await page.getByText(/Review Mode/i).first().click();
// Should show "Review Off" again
await expect(page.getByText(/Review Off/i)).toBeVisible();
}
});
});
@@ -0,0 +1,226 @@
{
"original_segments": [
{
"id": "seg-1",
"text": "Hello, is it me you're looking for?",
"words": [
{"id": "w1", "text": "Hello,", "start_time": 0.0, "end_time": 0.5},
{"id": "w2", "text": "is", "start_time": 0.5, "end_time": 0.7},
{"id": "w3", "text": "it", "start_time": 0.7, "end_time": 0.9},
{"id": "w4", "text": "me", "start_time": 0.9, "end_time": 1.1},
{"id": "w5", "text": "you're", "start_time": 1.1, "end_time": 1.3},
{"id": "w6", "text": "looking", "start_time": 1.3, "end_time": 1.6},
{"id": "w7", "text": "for?", "start_time": 1.6, "end_time": 1.9}
],
"start_time": 0.0,
"end_time": 1.9
},
{
"id": "seg-2",
"text": "I can see it in your eyes",
"words": [
{"id": "w8", "text": "I", "start_time": 2.0, "end_time": 2.2},
{"id": "w9", "text": "can", "start_time": 2.2, "end_time": 2.4},
{"id": "w10", "text": "see", "start_time": 2.4, "end_time": 2.6},
{"id": "w11", "text": "it", "start_time": 2.6, "end_time": 2.8},
{"id": "w12", "text": "in", "start_time": 2.8, "end_time": 3.0},
{"id": "w13", "text": "your", "start_time": 3.0, "end_time": 3.2},
{"id": "w14", "text": "eyes", "start_time": 3.2, "end_time": 3.5}
],
"start_time": 2.0,
"end_time": 3.5
},
{
"id": "seg-3",
"text": "I can see it in your smile",
"words": [
{"id": "w15", "text": "I", "start_time": 4.0, "end_time": 4.2},
{"id": "w16", "text": "can", "start_time": 4.2, "end_time": 4.4},
{"id": "w17", "text": "see", "start_time": 4.4, "end_time": 4.6},
{"id": "w18", "text": "it", "start_time": 4.6, "end_time": 4.8},
{"id": "w19", "text": "in", "start_time": 4.8, "end_time": 5.0},
{"id": "w20", "text": "your", "start_time": 5.0, "end_time": 5.2},
{"id": "w21", "text": "smile", "start_time": 5.2, "end_time": 5.5}
],
"start_time": 4.0,
"end_time": 5.5
}
],
"reference_lyrics": {
"genius": {
"segments": [
{
"id": "ref-seg-1",
"text": "Hello, is it me you're looking for?",
"words": [
{"id": "rw1", "text": "Hello,", "start_time": null, "end_time": null},
{"id": "rw2", "text": "is", "start_time": null, "end_time": null},
{"id": "rw3", "text": "it", "start_time": null, "end_time": null},
{"id": "rw4", "text": "me", "start_time": null, "end_time": null},
{"id": "rw5", "text": "you're", "start_time": null, "end_time": null},
{"id": "rw6", "text": "looking", "start_time": null, "end_time": null},
{"id": "rw7", "text": "for?", "start_time": null, "end_time": null}
],
"start_time": null,
"end_time": null
}
],
"metadata": {
"source": "genius",
"track_name": "Hello",
"artist_names": "Lionel Richie",
"album_name": "Can't Slow Down",
"duration_ms": 280000,
"explicit": false,
"language": "en",
"is_synced": false,
"lyrics_provider": "Genius",
"lyrics_provider_id": "12345",
"provider_metadata": {}
},
"source": "genius"
}
},
"anchor_sequences": [
{
"id": "anchor-1",
"transcribed_word_ids": ["w1", "w2", "w3", "w4"],
"transcription_position": 0,
"reference_positions": {"genius": 0},
"reference_word_ids": {"genius": ["rw1", "rw2", "rw3", "rw4"]},
"confidence": 0.95
}
],
"gap_sequences": [
{
"id": "gap-1",
"transcribed_word_ids": ["w5", "w6", "w7"],
"transcription_position": 4,
"preceding_anchor_id": "anchor-1",
"following_anchor_id": null,
"reference_word_ids": {"genius": ["rw5", "rw6", "rw7"]}
}
],
"resized_segments": [],
"corrections_made": 3,
"confidence": 0.85,
"corrections": [
{
"id": "corr-1",
"handler": "AgenticCorrector",
"original_word": "you're",
"corrected_word": "now",
"segment_id": "seg-1",
"word_id": "w5",
"corrected_word_id": "cw5",
"source": "genius",
"confidence": 0.92,
"reason": "Transcription error - misheard word, reference lyrics confirm it should be 'now' [SOUND_ALIKE] (confidence: 92%)",
"alternatives": {"your": 0.65},
"is_deletion": false,
"split_index": null,
"split_total": null,
"reference_positions": {"genius": 4},
"length": 1
},
{
"id": "corr-2",
"handler": "AgenticCorrector",
"original_word": "it",
"corrected_word": "",
"segment_id": "seg-2",
"word_id": "w11",
"corrected_word_id": null,
"source": "genius",
"confidence": 0.88,
"reason": "Extra filler word detected, not in reference lyrics [EXTRA_WORDS] (confidence: 88%)",
"alternatives": {},
"is_deletion": true,
"split_index": null,
"split_total": null,
"reference_positions": {},
"length": 1
},
{
"id": "corr-3",
"handler": "AgenticCorrector",
"original_word": "I",
"corrected_word": "",
"segment_id": "seg-3",
"word_id": "w15",
"corrected_word_id": null,
"source": "genius",
"confidence": 0.75,
"reason": "Extra word at start of line, not in reference [EXTRA_WORDS] (confidence: 75%)",
"alternatives": {},
"is_deletion": true,
"split_index": null,
"split_total": null,
"reference_positions": {},
"length": 1
}
],
"corrected_segments": [
{
"id": "seg-1",
"text": "Hello, is it me you're looking for?",
"words": [
{"id": "w1", "text": "Hello,", "start_time": 0.0, "end_time": 0.5},
{"id": "w2", "text": "is", "start_time": 0.5, "end_time": 0.7},
{"id": "w3", "text": "it", "start_time": 0.7, "end_time": 0.9},
{"id": "w4", "text": "me", "start_time": 0.9, "end_time": 1.1},
{"id": "cw5", "text": "now", "start_time": 1.1, "end_time": 1.3, "created_during_correction": true},
{"id": "w6", "text": "looking", "start_time": 1.3, "end_time": 1.6},
{"id": "w7", "text": "for?", "start_time": 1.6, "end_time": 1.9}
],
"start_time": 0.0,
"end_time": 1.9
},
{
"id": "seg-2",
"text": "I can see in your eyes",
"words": [
{"id": "w8", "text": "I", "start_time": 2.0, "end_time": 2.2},
{"id": "w9", "text": "can", "start_time": 2.2, "end_time": 2.4},
{"id": "w10", "text": "see", "start_time": 2.4, "end_time": 2.6},
{"id": "w12", "text": "in", "start_time": 2.8, "end_time": 3.0},
{"id": "w13", "text": "your", "start_time": 3.0, "end_time": 3.2},
{"id": "w14", "text": "eyes", "start_time": 3.2, "end_time": 3.5}
],
"start_time": 2.0,
"end_time": 3.5
},
{
"id": "seg-3",
"text": "can see it in your smile",
"words": [
{"id": "w16", "text": "can", "start_time": 4.2, "end_time": 4.4},
{"id": "w17", "text": "see", "start_time": 4.4, "end_time": 4.6},
{"id": "w18", "text": "it", "start_time": 4.6, "end_time": 4.8},
{"id": "w19", "text": "in", "start_time": 4.8, "end_time": 5.0},
{"id": "w20", "text": "your", "start_time": 5.0, "end_time": 5.2},
{"id": "w21", "text": "smile", "start_time": 5.2, "end_time": 5.5}
],
"start_time": 4.0,
"end_time": 5.5
}
],
"metadata": {
"anchor_sequences_count": 1,
"gap_sequences_count": 1,
"total_words": 21,
"correction_ratio": 0.143,
"audio_filepath": "/path/to/test-audio.mp3",
"audio_hash": "test-audio-hash-123",
"available_handlers": [
{"id": "agentic", "name": "AgenticCorrector", "description": "AI-powered agentic correction", "enabled": true},
{"id": "sound_alike", "name": "SoundAlikeHandler", "description": "Fixes sound-alike errors", "enabled": true}
],
"enabled_handlers": ["AgenticCorrector", "SoundAlikeHandler"]
},
"correction_steps": [],
"word_id_map": {
"w5": "cw5"
},
"segment_id_map": {}
}
@@ -11,7 +11,10 @@
"lint": "eslint .",
"preview": "vite preview",
"predeploy": "npm run build-prod",
"deploy": "gh-pages -d dist"
"deploy": "gh-pages -d dist",
"test": "playwright test",
"test:ui": "playwright test --ui",
"test:headed": "playwright test --headed"
},
"dependencies": {
"@emotion/react": "^11.14.0",
@@ -61,7 +61,7 @@ export default defineConfig({
// Run local dev server before starting tests
webServer: {
command: 'yarn dev',
command: 'npm run dev',
url: 'http://localhost:5173',
reuseExistingServer: !process.env.CI,
timeout: 120000,
@@ -23,6 +23,7 @@ interface CorrectedWordWithActionsProps {
onClick?: () => void
backgroundColor?: string
shouldFlash?: boolean
showActions?: boolean // Controls whether inline action buttons are visible
}
const WordContainer = styled(Box, {
@@ -98,7 +99,8 @@ export default function CorrectedWordWithActions({
onAccept,
onClick,
backgroundColor,
shouldFlash
shouldFlash,
showActions = true
}: CorrectedWordWithActionsProps) {
const theme = useTheme()
const isMobile = useMediaQuery(theme.breakpoints.down('sm'))
@@ -115,7 +117,7 @@ export default function CorrectedWordWithActions({
onClick={onClick}
>
<OriginalWordLabel>{originalWord}</OriginalWordLabel>
<Box
component="span"
sx={{
@@ -127,40 +129,42 @@ export default function CorrectedWordWithActions({
{word}
</Box>
<ActionsContainer>
<Tooltip title="Revert to original" placement="top" arrow>
<ActionButton
size="small"
onClick={(e) => handleAction(e, onRevert)}
aria-label="revert correction"
>
<UndoIcon />
</ActionButton>
</Tooltip>
<Tooltip title="Edit correction" placement="top" arrow>
<ActionButton
size="small"
onClick={(e) => handleAction(e, onEdit)}
aria-label="edit correction"
>
<EditIcon />
</ActionButton>
</Tooltip>
{!isMobile && (
<Tooltip title="Accept correction" placement="top" arrow>
{showActions && (
<ActionsContainer>
<Tooltip title="Revert to original" placement="top" arrow>
<ActionButton
size="small"
onClick={(e) => handleAction(e, onAccept)}
aria-label="accept correction"
sx={{ color: 'success.main' }}
onClick={(e) => handleAction(e, onRevert)}
aria-label="revert correction"
>
<CheckCircleOutlineIcon />
<UndoIcon />
</ActionButton>
</Tooltip>
)}
</ActionsContainer>
<Tooltip title="Edit correction" placement="top" arrow>
<ActionButton
size="small"
onClick={(e) => handleAction(e, onEdit)}
aria-label="edit correction"
>
<EditIcon />
</ActionButton>
</Tooltip>
{!isMobile && (
<Tooltip title="Accept correction" placement="top" arrow>
<ActionButton
size="small"
onClick={(e) => handleAction(e, onAccept)}
aria-label="accept correction"
sx={{ color: 'success.main' }}
>
<CheckCircleOutlineIcon />
</ActionButton>
</Tooltip>
)}
</ActionsContainer>
)}
</WordContainer>
)
}
@@ -1,4 +1,4 @@
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton } from '@mui/material'
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton, Chip } from '@mui/material'
import LockIcon from '@mui/icons-material/Lock'
import UploadFileIcon from '@mui/icons-material/UploadFile'
import FindReplaceIcon from '@mui/icons-material/FindReplace'
@@ -8,6 +8,9 @@ import RedoIcon from '@mui/icons-material/Redo'
import TimerIcon from '@mui/icons-material/Timer'
import RestoreIcon from '@mui/icons-material/Restore'
import RateReviewIcon from '@mui/icons-material/RateReview'
import VisibilityIcon from '@mui/icons-material/Visibility'
import CheckCircleIcon from '@mui/icons-material/CheckCircle'
import HighlightOffIcon from '@mui/icons-material/HighlightOff'
import { CorrectionData, InteractionMode } from '../types'
import CorrectionMetrics from './CorrectionMetrics'
import AgenticCorrectionMetrics from './AgenticCorrectionMetrics'
@@ -44,6 +47,13 @@ interface HeaderProps {
canRedo: boolean
annotationsEnabled?: boolean
onAnnotationsToggle?: (enabled: boolean) => void
// Review mode props
reviewMode?: boolean
onReviewModeToggle?: (enabled: boolean) => void
// Batch action props
onAcceptAllCorrections?: () => void
onAcceptHighConfidenceCorrections?: () => void
onRevertAllCorrections?: () => void
}
export default function Header({
@@ -70,6 +80,11 @@ export default function Header({
canRedo,
annotationsEnabled = true,
onAnnotationsToggle,
reviewMode = false,
onReviewModeToggle,
onAcceptAllCorrections,
onAcceptHighConfidenceCorrections,
onRevertAllCorrections,
}: HeaderProps) {
const theme = useTheme()
const isMobile = useMediaQuery(theme.breakpoints.down('md'))
@@ -129,33 +144,81 @@ export default function Header({
</Box>
)}
{isReadOnly && (
<Box sx={{ display: 'flex', justifyContent: 'flex-end', mb: 1 }}>
<Button
variant="outlined"
size="small"
startIcon={<UploadFileIcon />}
onClick={onFileLoad}
fullWidth={isMobile}
>
Load File
</Button>
<Box sx={{
display: 'flex',
flexDirection: isMobile ? 'column' : 'row',
gap: 1,
justifyContent: 'space-between',
alignItems: isMobile ? 'stretch' : 'center',
mb: 1
}}>
<Typography variant="h4" sx={{ fontSize: isMobile ? '1.3rem' : '1.5rem' }}>
Nomad Karaoke: Lyrics Transcription Review
</Typography>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
{!isReadOnly && isAgenticMode && onReviewModeToggle && (
<Tooltip title={reviewMode
? "Hide inline correction actions"
: "Show inline actions on all corrections for quick review"
}>
<Chip
icon={<VisibilityIcon />}
label={reviewMode ? "Review Mode" : "Review Off"}
onClick={() => onReviewModeToggle(!reviewMode)}
color={reviewMode ? "secondary" : "default"}
variant={reviewMode ? "filled" : "outlined"}
size="small"
sx={{
cursor: 'pointer',
'& .MuiChip-icon': { fontSize: '1rem' }
}}
/>
</Tooltip>
)}
{!isReadOnly && onAnnotationsToggle && (
<Tooltip title={annotationsEnabled
? "Click to disable annotation prompts when editing"
: "Click to enable annotation prompts when editing"
}>
<Chip
icon={<RateReviewIcon />}
label={annotationsEnabled ? "Feedback On" : "Feedback Off"}
onClick={() => onAnnotationsToggle(!annotationsEnabled)}
color={annotationsEnabled ? "primary" : "default"}
variant={annotationsEnabled ? "filled" : "outlined"}
size="small"
sx={{
cursor: 'pointer',
'& .MuiChip-icon': { fontSize: '1rem' }
}}
/>
</Tooltip>
)}
{isReadOnly && (
<Button
variant="outlined"
size="small"
startIcon={<UploadFileIcon />}
onClick={onFileLoad}
fullWidth={isMobile}
>
Load File
</Button>
)}
</Box>
)}
</Box>
<Box sx={{
display: 'flex',
gap: 1,
mb: 1,
flexDirection: isMobile ? 'column' : 'row',
height: isMobile ? 'auto' : '140px',
minHeight: isMobile ? 'auto' : '140px'
height: '140px'
}}>
<Box sx={{
width: isMobile ? '100%' : '280px',
minWidth: isMobile ? '100%' : '280px',
width: '280px',
position: 'relative',
height: isMobile ? 'auto' : '100%'
height: '100%'
}}>
{isAgenticMode ? (
<AgenticCorrectionMetrics
@@ -252,6 +315,65 @@ export default function Header({
</Box>
</Box>
{/* Batch Actions Panel - shown when review mode is enabled */}
{!isReadOnly && reviewMode && isAgenticMode && data.corrections?.length > 0 && (
<Paper sx={{ p: 0.8, mb: 1, backgroundColor: 'action.hover' }}>
<Box sx={{
display: 'flex',
flexDirection: isMobile ? 'column' : 'row',
gap: 1,
alignItems: isMobile ? 'stretch' : 'center',
justifyContent: 'space-between'
}}>
<Typography variant="subtitle2" sx={{ fontSize: '0.75rem', color: 'text.secondary' }}>
Batch Actions ({data.corrections.length} corrections)
</Typography>
<Box sx={{
display: 'flex',
gap: 1,
flexWrap: 'wrap'
}}>
{onAcceptHighConfidenceCorrections && (
<Button
variant="contained"
size="small"
color="success"
startIcon={<CheckCircleIcon />}
onClick={onAcceptHighConfidenceCorrections}
sx={{ fontSize: '0.75rem', py: 0.5 }}
>
Accept High Confidence ({data.corrections.filter(c => c.confidence >= 0.8).length})
</Button>
)}
{onAcceptAllCorrections && (
<Button
variant="outlined"
size="small"
color="success"
startIcon={<CheckCircleIcon />}
onClick={onAcceptAllCorrections}
sx={{ fontSize: '0.75rem', py: 0.5 }}
>
Accept All
</Button>
)}
{onRevertAllCorrections && (
<Button
variant="outlined"
size="small"
color="error"
startIcon={<HighlightOffIcon />}
onClick={onRevertAllCorrections}
sx={{ fontSize: '0.75rem', py: 0.5 }}
>
Revert All
</Button>
)}
</Box>
</Box>
</Paper>
)}
<Paper sx={{ p: 0.8, mb: 1 }}>
<Box sx={{
display: 'flex',
@@ -263,18 +385,17 @@ export default function Header({
}}>
<Box sx={{
display: 'flex',
gap: 0.5,
flexDirection: 'row',
flexWrap: 'wrap',
alignItems: 'center',
minHeight: '44px'
gap: 1,
flexDirection: isMobile ? 'column' : 'row',
alignItems: isMobile ? 'flex-start' : 'center',
height: '32px'
}}>
<ModeSelector
effectiveMode={effectiveMode}
onChange={onModeChange}
/>
{!isReadOnly && (
<Box sx={{ display: 'flex', height: '44px', alignItems: 'center' }}>
<Box sx={{ display: 'flex', height: '32px' }}>
<Tooltip title="Undo">
<span>
<IconButton
@@ -285,8 +406,8 @@ export default function Header({
border: `1px solid ${theme.palette.divider}`,
borderRadius: '4px',
mx: 0.25,
height: '44px',
width: '44px'
height: '32px',
width: '32px'
}}
>
<UndoIcon fontSize="small" />
@@ -303,8 +424,8 @@ export default function Header({
border: `1px solid ${theme.palette.divider}`,
borderRadius: '4px',
mx: 0.25,
height: '44px',
width: '44px'
height: '32px',
width: '32px'
}}
>
<RedoIcon fontSize="small" />
@@ -359,10 +480,10 @@ export default function Header({
Timing Offset
</Button>
{timingOffsetMs !== 0 && (
<Typography
variant="body2"
sx={{
ml: 1,
<Typography
variant="body2"
sx={{
ml: 1,
fontWeight: 'bold',
color: theme.palette.secondary.main
}}
@@ -372,23 +493,6 @@ export default function Header({
)}
</Box>
)}
{!isReadOnly && onAnnotationsToggle && (
<Tooltip title={annotationsEnabled
? "Click to disable annotation prompts when editing"
: "Click to enable annotation prompts when editing"
}>
<Button
variant="outlined"
size="small"
onClick={() => onAnnotationsToggle(!annotationsEnabled)}
startIcon={<RateReviewIcon />}
color={annotationsEnabled ? "primary" : "inherit"}
sx={{ minWidth: 'fit-content', height: '32px' }}
>
{annotationsEnabled ? "Feedback On" : "Feedback Off"}
</Button>
</Tooltip>
)}
<AudioPlayer
apiClient={apiClient}
onTimeUpdate={onTimeUpdate}
@@ -88,6 +88,12 @@ interface MemoizedTranscriptionViewProps {
anchors: AnchorSequence[]
disableHighlighting: boolean
onDataChange?: (updatedData: CorrectionData) => void
// Review mode props
reviewMode: boolean
onRevertCorrection: (wordId: string) => void
onEditCorrection: (wordId: string) => void
onAcceptCorrection: (wordId: string) => void
onShowCorrectionDetail: (wordId: string) => void
}
// Create a memoized TranscriptionView component
@@ -103,7 +109,12 @@ const MemoizedTranscriptionView = memo(function MemoizedTranscriptionView({
currentTime,
anchors,
disableHighlighting,
onDataChange
onDataChange,
reviewMode,
onRevertCorrection,
onEditCorrection,
onAcceptCorrection,
onShowCorrectionDetail
}: MemoizedTranscriptionViewProps) {
return (
<TranscriptionView
@@ -118,6 +129,11 @@ const MemoizedTranscriptionView = memo(function MemoizedTranscriptionView({
currentTime={disableHighlighting ? undefined : currentTime}
anchors={anchors}
onDataChange={onDataChange}
reviewMode={reviewMode}
onRevertCorrection={onRevertCorrection}
onEditCorrection={onEditCorrection}
onAcceptCorrection={onAcceptCorrection}
onShowCorrectionDetail={onShowCorrectionDetail}
/>
);
});
@@ -202,6 +218,13 @@ interface MemoizedHeaderProps {
onUnCorrectAll: () => void
annotationsEnabled: boolean
onAnnotationsToggle: (enabled: boolean) => void
// Review mode props
reviewMode: boolean
onReviewModeToggle: (enabled: boolean) => void
// Batch action props
onAcceptAllCorrections: () => void
onAcceptHighConfidenceCorrections: () => void
onRevertAllCorrections: () => void
}
// Create a memoized Header component
@@ -228,7 +251,12 @@ const MemoizedHeader = memo(function MemoizedHeader({
canRedo,
onUnCorrectAll,
annotationsEnabled,
onAnnotationsToggle
onAnnotationsToggle,
reviewMode,
onReviewModeToggle,
onAcceptAllCorrections,
onAcceptHighConfidenceCorrections,
onRevertAllCorrections
}: MemoizedHeaderProps) {
return (
<Header
@@ -255,6 +283,11 @@ const MemoizedHeader = memo(function MemoizedHeader({
onUnCorrectAll={onUnCorrectAll}
annotationsEnabled={annotationsEnabled}
onAnnotationsToggle={onAnnotationsToggle}
reviewMode={reviewMode}
onReviewModeToggle={onReviewModeToggle}
onAcceptAllCorrections={onAcceptAllCorrections}
onAcceptHighConfidenceCorrections={onAcceptHighConfidenceCorrections}
onRevertAllCorrections={onRevertAllCorrections}
/>
);
});
@@ -291,6 +324,9 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
const [isFindReplaceModalOpen, setIsFindReplaceModalOpen] = useState(false)
const [isTimingOffsetModalOpen, setIsTimingOffsetModalOpen] = useState(false)
const [timingOffsetMs, setTimingOffsetMs] = useState(0)
// Review mode state for agentic corrections
const [reviewMode, setReviewMode] = useState(false)
// Annotation collection state
const [annotations, setAnnotations] = useState<Omit<CorrectionAnnotation, 'annotation_id' | 'timestamp'>[]>([])
@@ -793,11 +829,82 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
// For now, just log acceptance
// In the future, this could be tracked in the annotation system
console.log('Accepted correction for word:', wordId)
// TODO: Track acceptance in annotation system
// This could be used to build confidence in the AI's corrections over time
}, [])
// Batch action handlers for review mode
const handleAcceptAllCorrections = useCallback(() => {
// Accept all corrections - for now just log, could track in annotation system
data.corrections?.forEach(c => {
console.log('Batch accepted correction:', c.corrected_word_id || c.word_id)
})
console.log(`Accepted all ${data.corrections?.length || 0} corrections`)
}, [data.corrections])
const handleAcceptHighConfidenceCorrections = useCallback(() => {
// Accept corrections with confidence >= 80%
const highConfidence = data.corrections?.filter(c => c.confidence >= 0.8) || []
highConfidence.forEach(c => {
console.log('Batch accepted high-confidence correction:', c.corrected_word_id || c.word_id)
})
console.log(`Accepted ${highConfidence.length} high-confidence corrections`)
}, [data.corrections])
const handleRevertAllCorrections = useCallback(() => {
if (!window.confirm(`Are you sure you want to revert all ${data.corrections?.length || 0} corrections? This cannot be undone.`)) {
return
}
// Revert all corrections by reverting each one
// Process in reverse order to avoid ID conflicts
const corrections = [...(data.corrections || [])].reverse()
let newData = data
for (const correction of corrections) {
const wordId = correction.corrected_word_id || correction.word_id
// Find the segment containing the corrected word
const segmentIndex = newData.corrected_segments.findIndex(segment =>
segment.words.some(w => w.id === wordId)
)
if (segmentIndex === -1) continue
const segment = newData.corrected_segments[segmentIndex]
// Replace the corrected word with the original
const newWords = segment.words.map(word => {
if (word.id === wordId) {
return {
...word,
text: correction.original_word,
id: correction.word_id
}
}
return word
})
const newText = newWords.map(w => w.text).join(' ')
const newSegment = { ...segment, words: newWords, text: newText }
const newSegments = newData.corrected_segments.map((seg, idx) =>
idx === segmentIndex ? newSegment : seg
)
newData = {
...newData,
corrected_segments: newSegments,
corrections: newData.corrections?.filter(c =>
c.corrected_word_id !== wordId && c.word_id !== wordId
) || []
}
}
updateDataWithHistory(newData, 'revert all corrections')
console.log(`Reverted all ${corrections.length} corrections`)
}, [data, updateDataWithHistory])
const handleShowCorrectionDetail = useCallback((wordId: string) => {
// Find the correction for this word
const correction = data.corrections?.find(c =>
@@ -1177,6 +1284,11 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
onUnCorrectAll={handleUnCorrectAll}
annotationsEnabled={annotationsEnabled}
onAnnotationsToggle={handleAnnotationsToggle}
reviewMode={reviewMode}
onReviewModeToggle={setReviewMode}
onAcceptAllCorrections={handleAcceptAllCorrections}
onAcceptHighConfidenceCorrections={handleAcceptHighConfidenceCorrections}
onRevertAllCorrections={handleRevertAllCorrections}
/>
<Grid container direction={isMobile ? 'column' : 'row'}>
@@ -1198,6 +1310,11 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
// needs to update history
updateDataWithHistory(updatedData, 'direct data change');
}}
reviewMode={reviewMode}
onRevertCorrection={handleRevertCorrection}
onEditCorrection={handleEditCorrection}
onAcceptCorrection={handleAcceptCorrection}
onShowCorrectionDetail={handleShowCorrectionDetail}
/>
{!isReadOnly && apiClient && (
<Box sx={{
@@ -54,7 +54,12 @@ export default function TranscriptionView({
onPlaySegment,
currentTime = 0,
anchors = [],
onDataChange
onDataChange,
reviewMode = false,
onRevertCorrection,
onEditCorrection,
onAcceptCorrection,
onShowCorrectionDetail
}: TranscriptionViewProps) {
const [selectedSegmentIndex, setSelectedSegmentIndex] = useState<number | null>(null)
const [viewMode, setViewMode] = useState<'text' | 'duration'>('text')
@@ -237,6 +242,11 @@ export default function TranscriptionView({
currentTime={currentTime}
gaps={data.gap_sequences}
corrections={data.corrections}
reviewMode={reviewMode}
onRevertCorrection={onRevertCorrection}
onEditCorrection={onEditCorrection}
onAcceptCorrection={onAcceptCorrection}
onShowCorrectionDetail={onShowCorrectionDetail}
/>
</TextContainer>
</Box>
@@ -1,4 +1,4 @@
import { Typography, Box } from '@mui/material'
import { Typography, Box, useMediaQuery, useTheme } from '@mui/material'
import { WordComponent } from './Word'
import { useWordClick } from '../hooks/useWordClick'
import {
@@ -16,6 +16,7 @@ import React from 'react'
import ContentCopyIcon from '@mui/icons-material/ContentCopy'
import IconButton from '@mui/material/IconButton'
import { getWordsFromIds } from '../utils/wordUtils'
import CorrectedWordWithActions from '../../CorrectedWordWithActions'
export interface HighlightedTextProps {
text?: string
@@ -36,6 +37,12 @@ export interface HighlightedTextProps {
gaps?: GapSequence[]
flashingHandler?: string | null
corrections?: WordCorrection[]
// Review mode props for agentic corrections
reviewMode?: boolean
onRevertCorrection?: (wordId: string) => void
onEditCorrection?: (wordId: string) => void
onAcceptCorrection?: (wordId: string) => void
onShowCorrectionDetail?: (wordId: string) => void
}
export function HighlightedText({
@@ -57,7 +64,15 @@ export function HighlightedText({
gaps = [],
flashingHandler,
corrections = [],
reviewMode = false,
onRevertCorrection,
onEditCorrection,
onAcceptCorrection,
onShowCorrectionDetail,
}: HighlightedTextProps) {
const theme = useTheme()
const isMobile = useMediaQuery(theme.breakpoints.down('sm'))
const { handleWordClick } = useWordClick({
mode,
onElementClick,
@@ -157,43 +172,83 @@ export function HighlightedText({
const renderContent = () => {
if (wordPositions && !segments) {
return wordPositions.map((wordPos, index) => (
<React.Fragment key={wordPos.word.id}>
<WordComponent
key={`${wordPos.word.id}-${index}`}
word={wordPos.word.text}
shouldFlash={shouldWordFlash(wordPos)}
isAnchor={wordPos.type === 'anchor'}
isCorrectedGap={wordPos.isCorrected}
isUncorrectedGap={wordPos.type === 'gap' && !wordPos.isCorrected}
isCurrentlyPlaying={shouldHighlightWord(wordPos)}
onClick={() => handleWordClick(
wordPos.word.text,
wordPos.word.id,
wordPos.type === 'anchor' ? wordPos.sequence as AnchorSequence : undefined,
wordPos.type === 'gap' ? wordPos.sequence as GapSequence : undefined
)}
correction={(() => {
const correction = corrections?.find(c =>
c.corrected_word_id === wordPos.word.id ||
c.word_id === wordPos.word.id
);
return correction ? {
return wordPositions.map((wordPos, index) => {
// Find correction for this word
const correction = corrections?.find(c =>
c.corrected_word_id === wordPos.word.id ||
c.word_id === wordPos.word.id
);
// Use CorrectedWordWithActions for agentic corrections
if (correction && correction.handler === 'AgenticCorrector') {
return (
<React.Fragment key={wordPos.word.id}>
<CorrectedWordWithActions
word={wordPos.word.text}
originalWord={correction.original_word}
correction={{
originalWord: correction.original_word,
handler: correction.handler,
confidence: correction.confidence,
source: correction.source,
reason: correction.reason
}}
shouldFlash={shouldWordFlash(wordPos)}
showActions={reviewMode && !isMobile}
onRevert={() => onRevertCorrection?.(wordPos.word.id)}
onEdit={() => onEditCorrection?.(wordPos.word.id)}
onAccept={() => onAcceptCorrection?.(wordPos.word.id)}
onClick={() => {
if (isMobile) {
onShowCorrectionDetail?.(wordPos.word.id)
} else {
handleWordClick(
wordPos.word.text,
wordPos.word.id,
wordPos.type === 'anchor' ? wordPos.sequence as AnchorSequence : undefined,
wordPos.type === 'gap' ? wordPos.sequence as GapSequence : undefined
)
}
}}
/>
{index < wordPositions.length - 1 && ' '}
</React.Fragment>
);
}
// Default rendering with WordComponent
return (
<React.Fragment key={wordPos.word.id}>
<WordComponent
key={`${wordPos.word.id}-${index}`}
word={wordPos.word.text}
shouldFlash={shouldWordFlash(wordPos)}
isAnchor={wordPos.type === 'anchor'}
isCorrectedGap={wordPos.isCorrected}
isUncorrectedGap={wordPos.type === 'gap' && !wordPos.isCorrected}
isCurrentlyPlaying={shouldHighlightWord(wordPos)}
onClick={() => handleWordClick(
wordPos.word.text,
wordPos.word.id,
wordPos.type === 'anchor' ? wordPos.sequence as AnchorSequence : undefined,
wordPos.type === 'gap' ? wordPos.sequence as GapSequence : undefined
)}
correction={correction ? {
originalWord: correction.original_word,
handler: correction.handler,
confidence: correction.confidence,
source: correction.source,
reason: correction.reason
} : null;
})()}
/>
{index < wordPositions.length - 1 && ' '}
</React.Fragment>
))
} : null}
/>
{index < wordPositions.length - 1 && ' '}
</React.Fragment>
);
})
} else if (segments) {
return segments.map((segment) => (
<Box key={segment.id} sx={{
display: 'flex',
<Box key={segment.id} sx={{
display: 'flex',
alignItems: 'flex-start',
mb: 0
}}>
@@ -212,12 +267,44 @@ export function HighlightedText({
const sequence = wordPos?.type === 'gap' ? wordPos.sequence as GapSequence : undefined;
// Find correction information for the tooltip
const correction = corrections?.find(c =>
c.corrected_word_id === word.id ||
// Find correction information
const correction = corrections?.find(c =>
c.corrected_word_id === word.id ||
c.word_id === word.id
);
// Use CorrectedWordWithActions for agentic corrections
if (correction && correction.handler === 'AgenticCorrector') {
return (
<React.Fragment key={word.id}>
<CorrectedWordWithActions
word={word.text}
originalWord={correction.original_word}
correction={{
originalWord: correction.original_word,
handler: correction.handler,
confidence: correction.confidence,
source: correction.source,
reason: correction.reason
}}
shouldFlash={shouldWordFlash(wordPos || { word: word.text, id: word.id })}
showActions={reviewMode && !isMobile}
onRevert={() => onRevertCorrection?.(word.id)}
onEdit={() => onEditCorrection?.(word.id)}
onAccept={() => onAcceptCorrection?.(word.id)}
onClick={() => {
if (isMobile) {
onShowCorrectionDetail?.(word.id)
} else {
handleWordClick(word.text, word.id, anchor, sequence)
}
}}
/>
{wordIndex < segment.words.length - 1 && ' '}
</React.Fragment>
);
}
const correctionInfo = correction ? {
originalWord: correction.original_word,
handler: correction.handler,
@@ -85,6 +85,12 @@ export interface TranscriptionViewProps {
anchors?: AnchorSequence[]
flashingHandler?: string | null
onDataChange?: (updatedData: CorrectionData) => void
// Review mode props for agentic corrections
reviewMode?: boolean
onRevertCorrection?: (wordId: string) => void
onEditCorrection?: (wordId: string) => void
onAcceptCorrection?: (wordId: string) => void
onShowCorrectionDetail?: (wordId: string) => void
}
// Add LinePosition type here since it's used in multiple places
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.76.25"
version = "0.76.26"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"