@coderabbitai ignore ## Summary - Add `is_private` flag to jobs that changes distribution behavior for non-published tracks - Private jobs upload to Dropbox `Tracks-NonPublished` folder with `NOMADNP` brand prefix - Private jobs skip YouTube and Google Drive uploads entirely - Completion emails omit YouTube section for private jobs - Available to all users at job creation; admin can toggle on existing jobs - When admin toggles completed job to private, existing outputs are auto-deleted ## Changes ### Backend - **Job model**: Added `is_private: bool = False` to `Job` and `JobCreate` - **Config**: Added `DEFAULT_PRIVATE_DROPBOX_PATH` and `DEFAULT_PRIVATE_BRAND_PREFIX` settings - **Distribution helper**: New `get_effective_distribution_for_job()` in `job_defaults_service.py` applies private overrides at runtime - **Orchestrator + native paths**: Both distribution code paths use the helper (per "Fix Both Sides" lesson) - **Email template**: YouTube section conditionally removed when `is_private=True` - **Credential pre-validation**: Skip YouTube/GDrive credential checks for private jobs - **API endpoints**: All 4 job creation endpoints accept `is_private`; admin can PATCH it - **Admin auto-delete**: Toggling completed job to private triggers output deletion ### Frontend - **Job creation form**: "Private (Non-Published)" checkbox on all 3 tabs (upload, URL, search) - **Admin dashboard**: Private toggle in job configuration section - **API interfaces**: Updated TypeScript interfaces and API methods ### Tests - 23 unit tests covering model, distribution overrides, email template, notification service, request models, admin toggle, and orchestrator config - Fixed MagicMock truthiness issue in existing tests ## Test plan - [x] All 23 private tracks tests pass - [x] All 1794 existing backend tests pass (1 flaky unrelated test) - [x] CodeRabbit review: credential pre-validation issue found and fixed - [ ] Manual: Create private job end-to-end, verify Dropbox folder - [ ] Manual: Verify completion email has Dropbox link but no YouTube section - [ ] Manual: Admin toggle on completed job deletes outputs 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
15 KiB
Plan: Private (Non-Published) Tracks
Created: 2026-02-24 Branch: feat/sess-20260224-1149-private-tracks Status: Implemented
Overview
Add a is_private flag to jobs that changes distribution behavior:
- Dropbox: Uses
Tracks-NonPublishedfolder (notTracks-Organized) with brand codeNOMADNP(notNOMAD) - YouTube: Skipped entirely
- Google Drive: Skipped entirely
- Email: Completion email includes Dropbox link but no YouTube section
- Discord: Still notified (for internal tracking)
Available to all users at job creation and toggleable by admin on existing jobs.
Requirements
- All users can mark a job as "Private (Non-Published)" when creating it
- Admin can toggle
is_privateon any existing job via admin dashboard - Private jobs upload to Dropbox at
Tracks-NonPublishedwithNOMADNP-XXXXbrand codes - Private jobs skip YouTube and Google Drive uploads entirely
- Completion emails omit YouTube section when no YouTube URL (private jobs + any other edge case)
- When admin toggles a completed job to private, existing outputs are auto-deleted
- Discord notifications still fire for private jobs
- Existing (non-private) jobs are completely unaffected (backward compatible)
Technical Approach
Core Design: Single Boolean Flag with Runtime Overrides
Store is_private: bool = False on the job model. At distribution time, a helper function returns effective distribution parameters based on this flag. This means:
- Toggling is_private automatically changes all distribution behavior - no need to manually update dropbox_path, brand_prefix, enable_youtube_upload, gdrive_folder_id
- Job fields still store the "normal" defaults - the override happens at the config-to-orchestrator boundary
- Admin can flip one field and everything adapts
Key Helper Function
# In job_defaults_service.py
def get_effective_distribution_for_job(job) -> EffectiveDistributionSettings:
"""Return distribution settings, applying private overrides if is_private=True."""
if getattr(job, 'is_private', False):
settings = get_settings()
return EffectiveDistributionSettings(
dropbox_path=settings.default_private_dropbox_path,
brand_prefix=settings.default_private_brand_prefix,
enable_youtube_upload=False,
gdrive_folder_id=None,
# Keep other settings from job (discord, youtube_description, etc.)
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
youtube_description=None,
)
# Non-private: return job's own settings
return EffectiveDistributionSettings(
dropbox_path=getattr(job, 'dropbox_path', None),
brand_prefix=getattr(job, 'brand_prefix', None),
enable_youtube_upload=getattr(job, 'enable_youtube_upload', False),
gdrive_folder_id=getattr(job, 'gdrive_folder_id', None),
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
youtube_description=getattr(job, 'youtube_description_template', None),
)
This helper is used in both distribution paths (orchestrator and native fallback).
Distribution Paths That Need Changes
Two code paths handle distribution (see LESSONS-LEARNED: "Fix Both Sides of Dual Code Paths"):
- Orchestrator path (
video_worker_orchestrator.py): Primary path.create_orchestrator_config_from_job()uses the helper to build config. - Native distribution path (
video_worker.py→_handle_native_distribution()): Fallback path. Also uses the helper instead of reading job fields directly.
Email Template Change
Add conditional YouTube section removal in render_template(), matching the existing pattern for feedback_url:
# Remove YouTube section if no URL provided
if not variables.get("youtube_url"):
result = re.sub(
r"\n*Here's the link for the karaoke video published to YouTube:\n\{youtube_url\}\n*",
'\n',
result,
flags=re.DOTALL
)
variables["youtube_url"] = ""
And change render_job_completion() to pass None instead of "[YouTube URL not available]" when youtube_url is empty.
Admin Toggle Auto-Delete
When admin PATCH updates is_private from False to True on a job that has existing outputs, reuse the existing delete-outputs logic to remove YouTube + GDrive + Dropbox outputs. Admin can then "Reprocess" to re-distribute to the private Dropbox path.
Implementation Steps
Phase 1: Backend Model & Config (foundation)
-
Add
is_privateto Job model (backend/models/job.py)- Add
is_private: bool = Falsefield aftermade_for_youfield - Add to
JobCreatemodel as well
- Add
-
Add private distribution config (
backend/config.py)- Add
default_private_dropbox_path: Optional[str] = os.getenv("DEFAULT_PRIVATE_DROPBOX_PATH") - Add
default_private_brand_prefix: Optional[str] = os.getenv("DEFAULT_PRIVATE_BRAND_PREFIX", "NOMADNP")
- Add
-
Add helper function (
backend/services/job_defaults_service.py)- Add
get_effective_distribution_for_job(job)function that returns distribution settings with private overrides applied
- Add
Phase 2: Backend API (job creation + admin)
-
Add
is_privateto request models (backend/models/requests.py)- Add
is_private: Optional[bool] = NonetoURLSubmissionRequest - Add
is_private: Optional[bool] = NonetoAudioSearchRequest
- Add
-
Update URL creation endpoint (
backend/api/routes/jobs.py)- Accept and pass through
is_privatefield toJobCreate
- Accept and pass through
-
Update file upload endpoint (
backend/api/routes/file_upload.py)- Accept
is_privateForm parameter - Pass through to
JobCreate
- Accept
-
Update audio search endpoint (
backend/api/routes/audio_search.py)- Accept
is_privateinAudioSearchRequest - Pass through to
JobCreate
- Accept
-
Update admin endpoints (
backend/api/routes/admin.py)- Add
is_private: Optional[bool] = NonetoJobUpdateRequest - Add
"is_private"toEDITABLE_JOB_FIELDS - In the PATCH handler: detect when
is_privatechanges from False→True on a completed job, auto-delete existing outputs (call existing delete-outputs service logic)
- Add
Phase 3: Distribution Pipeline
-
Update orchestrator config creation (
backend/workers/video_worker_orchestrator.py)- In
create_orchestrator_config_from_job(): Useget_effective_distribution_for_job()helper to determine dropbox_path, brand_prefix, enable_youtube_upload, gdrive_folder_id instead of reading directly from job
- In
-
Update native distribution fallback (
backend/workers/video_worker.py)- In
_handle_native_distribution(): Useget_effective_distribution_for_job()helper instead of reading directly from job fields
- In
Phase 4: Email Template
- Add YouTube section conditional removal (
backend/services/template_service.py)- In
render_template(): Add regex to remove YouTube paragraph whenyoutube_urlis empty (matching existingfeedback_urlpattern) - In
render_job_completion(): PassNonefor youtube_url when empty (instead of"[YouTube URL not available]")
- In
Phase 5: Frontend
-
Add private checkbox to job creation form (
frontend/components/job/JobSubmission.tsx)- Add "Private (Non-Published)" checkbox below the existing "Skip lyrics review" checkbox
- Show for all users (not just admin)
- Include brief description text: "Private tracks go to Dropbox only, not YouTube or Google Drive"
-
Update API interfaces (
frontend/lib/api.ts)- Add
is_private?: booleanto all job creation option interfaces - Add
is_privatetoJobUpdateRequestinterface - Pass through in
uploadJob(),createJobFromUrl(),searchAudio()methods
- Add
-
Add private toggle to admin job detail (
frontend/app/admin/jobs/page.tsx)- Add
is_privateas an inline-editable field in the job overview section (use Switch component) - Show current state with visual indicator (e.g., lock icon for private jobs)
- When toggling to private on a completed job, show confirmation dialog about auto-deleting outputs
- Add
Phase 6: Environment Configuration
- Update CI/deployment config (
.github/workflows/ci.ymlor Cloud Run env)- Add
DEFAULT_PRIVATE_DROPBOX_PATHenvironment variable (e.g.,/Karaoke/Tracks-NonPublished) - Add
DEFAULT_PRIVATE_BRAND_PREFIXenvironment variable (e.g.,NOMADNP) - These should be configured in Cloud Run service and CI test environment
- Add
Phase 7: Testing
-
Unit tests for private distribution logic (
backend/tests/test_private_tracks.py)- Test
get_effective_distribution_for_job()with private and non-private jobs - Test
create_orchestrator_config_from_job()applies private overrides correctly - Test
_run_distribution()skips YouTube and GDrive when private config applied - Test
_upload_to_dropbox()uses private path and brand prefix - Test brand code generation with NOMADNP prefix in Tracks-NonPublished
- Test
_handle_native_distribution()respects private overrides - Test non-private jobs are completely unaffected (regression tests)
- Test
-
Unit tests for email changes (add to
backend/tests/test_email_service.pyortest_template_service.py)- Test YouTube section removed from email when youtube_url is None
- Test YouTube section present when youtube_url is provided
- Test Dropbox section still present for private jobs
- Test full email rendering for private job scenario
-
Unit tests for admin toggle (add to
backend/tests/test_admin.pyor new file)- Test admin can update is_private field
- Test toggling is_private to True on completed job triggers output deletion
- Test toggling is_private to True on non-completed job does NOT delete outputs
- Test toggling is_private to False does nothing special
-
Integration test (
backend/tests/emulator/ortest_distribution_services.py)- Test full distribution pipeline with is_private=True: verify Dropbox path, brand prefix, YouTube skip, GDrive skip
-
Frontend E2E regression test (
frontend/e2e/regression/)- Test private checkbox appears in job creation form
- Test private toggle in admin dashboard
Files to Create/Modify
| File | Action | Description |
|---|---|---|
backend/models/job.py |
Modify | Add is_private: bool = False to Job and JobCreate |
backend/config.py |
Modify | Add default_private_dropbox_path and default_private_brand_prefix |
backend/services/job_defaults_service.py |
Modify | Add get_effective_distribution_for_job() helper |
backend/models/requests.py |
Modify | Add is_private to URL and AudioSearch request models |
backend/api/routes/jobs.py |
Modify | Accept and pass is_private |
backend/api/routes/file_upload.py |
Modify | Accept and pass is_private |
backend/api/routes/audio_search.py |
Modify | Accept and pass is_private |
backend/api/routes/admin.py |
Modify | Add is_private to editable fields + auto-delete logic |
backend/workers/video_worker_orchestrator.py |
Modify | Use helper in create_orchestrator_config_from_job() |
backend/workers/video_worker.py |
Modify | Use helper in _handle_native_distribution() |
backend/services/template_service.py |
Modify | Conditional YouTube section removal |
frontend/components/job/JobSubmission.tsx |
Modify | Add private checkbox |
frontend/lib/api.ts |
Modify | Add is_private to interfaces and API calls |
frontend/app/admin/jobs/page.tsx |
Modify | Add private toggle to admin detail |
backend/tests/test_private_tracks.py |
Create | Comprehensive unit tests for private track logic |
Testing Strategy
Unit Tests (~25-30 tests in test_private_tracks.py)
get_effective_distribution_for_job()with private=True and Falsecreate_orchestrator_config_from_job()private overrides (YouTube=False, GDrive=None, private Dropbox path, NOMADNP brand)- Distribution stage skips: YouTube not called, GDrive not called, Dropbox IS called with correct path
- Brand code generation uses NOMADNP prefix and Tracks-NonPublished path
- Native distribution fallback applies same overrides
- Regression tests: Non-private jobs have unchanged behavior (same paths, same brand code, YouTube+GDrive+Dropbox all fire)
Email Template Tests (~5 tests)
- YouTube section stripped when youtube_url is None
- YouTube section preserved when youtube_url has value
- Dropbox section always present
- Full private job email rendering (Dropbox + no YouTube)
- Full non-private job email rendering (both present) - regression
Admin Toggle Tests (~5 tests)
- is_private editable via PATCH
- Toggle to private on completed job deletes outputs
- Toggle to private on pending/in-progress job does NOT delete
- Toggle to non-private does nothing special
Frontend Tests
- E2E: Private checkbox visible and functional in job creation
- E2E: Admin toggle visible and includes confirmation dialog
Manual Verification
- Create a private job end-to-end, verify Dropbox folder in Tracks-NonPublished with NOMADNP code
- Verify completion email has Dropbox link but no YouTube section
- Toggle existing completed job to private, verify outputs deleted
Regression Risk Analysis
| Risk | Mitigation |
|---|---|
| Non-private jobs affected | is_private defaults to False; helper returns unchanged values for non-private. Explicit regression tests. |
| Dual code paths (orchestrator vs native) | Both paths use same helper function. Tests cover both paths. (Lesson: "Fix Both Sides of Dual Code Paths") |
| Email template regex breaks other content | Regex targets specific YouTube paragraph text. Tests verify non-YouTube content untouched. |
| Brand code namespace collision | NOMADNP codes are in a completely separate Dropbox folder (Tracks-NonPublished), no overlap with NOMAD codes in Tracks-Organized |
| Admin auto-delete on wrong job | Only triggers when is_private changes from False→True AND job has terminal status (complete). Confirmation dialog in frontend. |
| Made-for-you orders | MFY webhook doesn't set is_private (defaults False). Admin can toggle after. No change to MFY flow. |
| Existing Firestore documents | Missing is_private field defaults to False via model default. No migration needed. |
Open Questions
- Who sees private toggle? → Everyone (all users at creation, admin on existing)
- Auto-delete on toggle? → Yes, auto-delete all outputs when toggling to private on completed job
- Email format? → Remove YouTube section when no YouTube URL (improves all edge cases)
- Discord? → Still notify (useful for internal tracking)
Rollback Plan
- Feature flag: The
is_privatefield defaults toFalse, so settingDEFAULT_PRIVATE_DROPBOX_PATHto empty/None effectively disables private distribution - No schema migration needed: Missing
is_privatefield on existing Firestore documents defaults toFalse - Email change is backward-compatible: The YouTube section removal only happens when youtube_url is None, which is already an edge case showing "[YouTube URL not available]"
- Frontend: Private checkbox is additive UI, removing it just means users can't create new private jobs