Files
Andrew Beveridge ☄️andClaude Opus 5.5 8a5f754efd ci: remove post-deploy canary + never email test accounts on stale review (v0.245.2) (#1068)
## Summary
- Removes the `deploy-verify` job ("Deploy - Post-Deploy Canary (Happy
Path)") from `ci.yml`, plus the `Detect backend version change` step /
`version_changed` output that only gated it.
- `stale_review_processor`: jobs owned by automated-test accounts
(`is_test_email` — `e2e-test-runner@nomadkaraoke.com`,
`@inbox.testmail.app`) no longer get the 24h reminder, review-expired,
or duration-confirm-expired emails. They are still auto-expired at 48h
so stranded test jobs don't linger.
- Updates comments + `docs/TROUBLESHOOTING.md` / `docs/TESTING.md` that
referenced the canary. Version → 0.245.2.

## Why
The canary created a real production job (piri – dog, as
`e2e-test-runner@nomadkaraoke.com`) on every backend version bump.
`ci.yml` uses `cancel-in-progress: true`, so back-to-back merges
cancelled it mid-run — the process is killed before Playwright's
`afterEach` cleanup runs, stranding the job in `awaiting_review`. The
stale-review processor then emailed the test account (which forwards to
Andrew's inbox) at 24h/48h (e.g. job `adb8f393` from cancelled run
36288893657; earlier `a5f84cc3`, `10b532e2`). The canary was overkill;
the daily E2E and the traffic-gated candidate-revision smoke remain, and
the email guard backstops the daily E2E if it ever strands a job.

## Test plan
- [x] New unit tests: test-account reminder is silent (+ still flagged),
review expiry + duration-confirm expiry cancel without email
- [x] `pytest backend/tests --ignore=backend/tests/emulator` — 4883
passed
- [x] `actionlint` — no new findings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* CI no longer runs a post-deployment production canary or sends its
test results and failure notifications. Candidate-revision smoke testing
remains in place.
* Production real-job verification is handled manually or by the daily
E2E run.
* **Documentation**
* Testing and rollback guidance now reflects the updated verification
and recovery steps. Daily E2E failures are identified as a reason to
roll back.
* **Bug Fixes**
* Automated test accounts no longer receive stale-review reminder or
expiry emails; existing expiry and cancellation handling remains in
place.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 23:37:45 -04:00

64 lines
2.1 KiB
Python

"""
Utilities for identifying test data and internal accounts.
Test data is generated by automated E2E tests and should be filterable
in admin dashboards to show only real user data by default.
Internal accounts belong to team members and should be excluded from
abuse investigation views.
"""
# Email domains used by automated testing frameworks
TEST_EMAIL_DOMAINS = [
"inbox.testmail.app", # Used by E2E happy path tests (ephemeral throwaway users)
]
# Specific internal accounts that are only ever used by automated tests.
# These live on the internal nomadkaraoke.com domain (so they'd otherwise pass
# is_internal_email, not is_test_email), but every job they create is E2E test
# data. Listed here so admin dashboards' exclude_test filter hides their jobs.
# e2e-test-runner is the persistent account the daily E2E workflow
# impersonates; without this entry its piri/dog jobs leak into the admin
# "My Jobs" view.
TEST_EMAIL_ADDRESSES = [
"e2e-test-runner@nomadkaraoke.com",
]
# Email domains belonging to internal team members
INTERNAL_EMAIL_DOMAINS = [
"nomadkaraoke.com", # Team accounts
"beveridge.uk", # Founder's personal domain
]
def is_test_email(email: str) -> bool:
"""
Check if an email address belongs to automated test data.
Args:
email: Email address to check
Returns:
True if the email matches a test email domain pattern or is a known
automated-test account
"""
if not email:
return False
email_lower = email.lower()
if email_lower in TEST_EMAIL_ADDRESSES:
return True
return any(email_lower.endswith(f"@{domain}") for domain in TEST_EMAIL_DOMAINS)
def is_internal_email(email: str) -> bool:
"""Check if an email address belongs to an internal team member."""
if not email:
return False
email_lower = email.lower()
return any(email_lower.endswith(f"@{domain}") for domain in INTERNAL_EMAIL_DOMAINS)
def is_internal_or_test_email(email: str) -> bool:
"""Check if an email is internal or automated test data."""
return is_test_email(email) or is_internal_email(email)