fix: tenant portal edge config stampede spawned cold API instances (v0.268.6) (#1141)

## Summary
This follows up #1140. While investigating the stall-alert flapping, I
found that 18 of the 28 requests that were the first on a new autoscaled
instance (each ≥10s) over 4 days came from the tenant portal's Pages
Function. Each one was `/api/tenant/config?tenant=…` with no user agent.

At 03:55 UTC, something loaded 17 `singa.nomadkaraoke.com` pages at
once. All 17 missed the 5-minute edge cache in the same second and
called the API. Cloud Run spawned 5 cold instances, and those HTML
responses waited 15–17s.

## Changes
New `frontend/lib/edge/tenantConfigCache.ts`, used by
`functions/[[path]].ts`:
- **Stale-while-revalidate:**
  - Under 5 minutes old, the cached config is served as is.
- Older than that, the stale copy is still served and refreshed in the
background via `waitUntil`. Copies are stored for 7 days.
- Entries cached before this deploy have no timestamp, so they are
treated as stale.
- **Bounded cold miss:** the backend fetch has a 3s
`AbortSignal.timeout`. On timeout or error the HTML is served without
injection, and `TenantProvider` falls back to fetching the config
client-side, as it already did.
- **Single flight:** concurrent misses in one isolate share one fetch.
- **Payload shape check:** a malformed 200 response isn't cached for a
week. This came from a CodeRabbit finding.

Other changes:
- `jest.setup.js` guards the `window.matchMedia` mock so
`@jest-environment node` tests can run. It's a no-op for jsdom tests.
- `docs/ARCHITECTURE.md` has a new section on the edge config cache.

**Trade-off:** a branding change now reaches each colo on the first page
view after 5 minutes, and that view still shows the old branding.
Before, it showed up on the first view after expiry.

## Testing
- 10 new Jest tests (node environment):
  - cold miss;
  - fresh hit;
  - stale hit with background refresh;
  - a 17-request burst makes one fetch;
  - 503 and timeout return null;
  - a failed refresh keeps the stale copy;
  - an entry without a timestamp is treated as stale;
  - a malformed payload isn't cached.
- `wrangler pages functions build` compiles the bundle.
- `make test`: all passed.
- Local CodeRabbit review: 1 finding, fixed.

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DTfrTaaZQfXUUduAhZpLeq

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude Opus 5.5 authored and GitHub committed 2026-10-13 18:29:45 -04:00
1 parent 3566fd136e
commit ccf6da020f
6 files changed
+331 -95

No files matched your search

+8
View File
@@ -818,6 +818,14 @@ window.__TENANT_CONFIG__ = { id: "vocalstar", branding: { ... }, features: { ...
The frontend's `TenantProvider` reads `window.__TENANT_CONFIG__` on startup (no API round-trip needed for initial render). This approach works for both custom domains (`vocalstar.nomadkaraoke.com`) and the shared tenant portal domain.
**Edge config cache** (`frontend/lib/edge/tenantConfigCache.ts`, v0.268.6). The edge cache works as stale-while-revalidate:
- A config fetched under 5 minutes ago is served as is.
- An older copy is still served, and a background refresh runs via `waitUntil`. Copies are kept up to 7 days.
- A colo that has never cached the tenant waits on `/api/tenant/config` for at most 3s. On timeout or error the HTML goes out without injection, and `TenantProvider` fetches the config itself.
- Concurrent misses in one isolate share a single backend fetch.
Before this, a crawler loading 17 portal pages at once caused 17 simultaneous cache misses. Cloud Run spawned 5 cold instances, and the HTML waited 15-17s (2026-10-13). Branding changes still reach the edge within about 5 minutes, plus one page view to trigger the refresh.
**Magic link emails**: When a tenant user requests a magic link, the email uses the tenant-specific URL (e.g., `https://vocalstar.nomadkaraoke.com/auth/verify?token=...`) rather than the main app URL. The `job_notification_service.py` resolves the correct base URL from the tenant config's `subdomain` field.
**Admin preview**: Admins can preview any tenant's branding in the main app by appending `?preview_tenant=<tenant_id>` to any URL. This bypasses subdomain detection and loads the specified tenant config, useful for verifying branding before DNS cutover.
@@ -0,0 +1,155 @@
/**
* @jest-environment node
*/
import {
getTenantConfig,
_resetInFlightForTests,
FRESH_SECONDS,
type TenantConfigResponse,
} from "@/lib/edge/tenantConfigCache"
const CONFIG: TenantConfigResponse = {
tenant: {
id: "singa",
name: "Singa",
subdomain: "singa",
is_active: true,
branding: {
primary_color: "#ff0000",
secondary_color: "#00ff00",
accent_color: null,
background_color: null,
favicon_url: null,
site_title: "Singa",
},
},
is_default: false,
}
class MemCache {
store = new Map<string, Response>()
async match(key: string) {
const r = this.store.get(key)
return r ? r.clone() : undefined
}
async put(key: string, r: Response) {
this.store.set(key, r)
}
}
function setup(fetchImpl?: jest.Mock) {
const cache = new MemCache()
const fetchFn =
fetchImpl ??
jest.fn(async () => new Response(JSON.stringify(CONFIG), { status: 200 }))
const background: Promise<unknown>[] = []
let now = 1_000_000
const deps = {
cache,
fetchFn: fetchFn as unknown as typeof fetch,
waitUntil: (p: Promise<unknown>) => {
background.push(p)
},
now: () => now,
}
return { cache, fetchFn, background, deps, advance: (ms: number) => (now += ms) }
}
const KEY = "https://singa.nomadkaraoke.com/__tenant_config__/singa"
beforeEach(() => _resetInFlightForTests())
describe("getTenantConfig (edge stale-while-revalidate)", () => {
it("fetches on a cold miss and caches the result", async () => {
const { fetchFn, cache, deps } = setup()
expect(await getTenantConfig("singa", KEY, deps)).toEqual(CONFIG)
expect(fetchFn).toHaveBeenCalledTimes(1)
expect(cache.store.has(KEY)).toBe(true)
})
it("serves a fresh cache hit without touching the backend", async () => {
const { fetchFn, deps, background, advance } = setup()
await getTenantConfig("singa", KEY, deps)
advance(60_000)
expect(await getTenantConfig("singa", KEY, deps)).toEqual(CONFIG)
expect(fetchFn).toHaveBeenCalledTimes(1)
expect(background).toHaveLength(0)
})
it("serves a stale hit immediately and refreshes in the background", async () => {
const { fetchFn, deps, background, advance } = setup()
await getTenantConfig("singa", KEY, deps)
advance((FRESH_SECONDS + 1) * 1000)
expect(await getTenantConfig("singa", KEY, deps)).toEqual(CONFIG)
expect(background).toHaveLength(1)
await Promise.all(background)
expect(fetchFn).toHaveBeenCalledTimes(2)
})
it("collapses a burst of concurrent cold misses into one backend call", async () => {
let release!: () => void
const gate = new Promise<void>((r) => (release = r))
const fetchFn = jest.fn(async () => {
await gate
return new Response(JSON.stringify(CONFIG), { status: 200 })
})
const { deps } = setup(fetchFn)
const burst = Array.from({ length: 17 }, () => getTenantConfig("singa", KEY, deps))
release()
const results = await Promise.all(burst)
expect(results.every((r) => r?.tenant?.id === "singa")).toBe(true)
expect(fetchFn).toHaveBeenCalledTimes(1)
})
it("returns null on a backend error so the HTML is served uninjected", async () => {
const fetchFn = jest.fn(async () => new Response("nope", { status: 503 }))
const { deps, cache } = setup(fetchFn)
expect(await getTenantConfig("singa", KEY, deps)).toBeNull()
expect(cache.store.has(KEY)).toBe(false)
})
it("does not cache a malformed 200 payload", async () => {
const fetchFn = jest.fn(
async () => new Response(JSON.stringify({ tenant: { id: "singa" }, is_default: false }), { status: 200 })
)
const { deps, cache } = setup(fetchFn)
expect(await getTenantConfig("singa", KEY, deps)).toBeNull()
expect(cache.store.has(KEY)).toBe(false)
})
it("passes a timeout signal so a cold backend can't hold the page", async () => {
const { fetchFn, deps } = setup()
await getTenantConfig("singa", KEY, deps)
const init = (fetchFn.mock.calls[0] as unknown[])[1] as RequestInit
expect(init.signal).toBeInstanceOf(AbortSignal)
})
it("returns null when the fetch aborts (timeout)", async () => {
const fetchFn = jest.fn(async () => {
throw new DOMException("The operation timed out.", "TimeoutError")
})
const { deps } = setup(fetchFn)
expect(await getTenantConfig("singa", KEY, deps)).toBeNull()
})
it("keeps serving the stale copy when the background refresh fails", async () => {
const responses = [
new Response(JSON.stringify(CONFIG), { status: 200 }),
new Response("down", { status: 500 }),
]
const fetchFn = jest.fn(async () => responses.shift()!)
const { deps, background, advance } = setup(fetchFn)
await getTenantConfig("singa", KEY, deps)
advance((FRESH_SECONDS + 1) * 1000)
await getTenantConfig("singa", KEY, deps)
await Promise.all(background)
expect(await getTenantConfig("singa", KEY, deps)).toEqual(CONFIG)
})
it("treats entries without a fetched-at stamp (pre-deploy cache) as stale", async () => {
const { cache, deps, background } = setup()
await cache.put(KEY, new Response(JSON.stringify(CONFIG)))
expect(await getTenantConfig("singa", KEY, deps)).toEqual(CONFIG)
expect(background).toHaveLength(1)
})
})
+7 -80
View File
@@ -10,30 +10,7 @@
* Non-HTML requests (JS, CSS, images) pass through unmodified.
*/
interface TenantBranding {
primary_color: string
secondary_color: string
accent_color: string | null
background_color: string | null
favicon_url: string | null
site_title: string
}
interface TenantPublicConfig {
id: string
name: string
subdomain: string
is_active: boolean
branding: TenantBranding
}
interface TenantConfigResponse {
tenant: TenantPublicConfig | null
is_default: boolean
}
const API_BASE_URL = "https://api.nomadkaraoke.com"
const CACHE_TTL_SECONDS = 300 // 5 minutes
import { getTenantConfig } from "../lib/edge/tenantConfigCache"
/**
* Compute a contrasting foreground color (black or white) for a given hex color.
@@ -71,60 +48,6 @@ function extractTenantFromHost(hostname: string): string | null {
return null
}
/**
* Fetch tenant config from backend with Cloudflare Cache API caching.
*/
async function fetchTenantConfigCached(
tenantId: string,
cacheKey: string
): Promise<TenantConfigResponse | null> {
// Try cache first
const cache = caches.default
const cachedResponse = await cache.match(cacheKey)
if (cachedResponse) {
try {
return await cachedResponse.json()
} catch {
// Cache corrupted, proceed to fetch
}
}
// Fetch from backend
try {
const response = await fetch(
`${API_BASE_URL}/api/tenant/config?tenant=${encodeURIComponent(tenantId)}`,
{
headers: {
"Content-Type": "application/json",
"X-Tenant-ID": tenantId,
},
}
)
if (!response.ok) {
console.error(`Tenant config fetch failed: ${response.status}`)
return null
}
const data: TenantConfigResponse = await response.json()
// Cache the response
const cacheResponse = new Response(JSON.stringify(data), {
headers: {
"Content-Type": "application/json",
"Cache-Control": `s-maxage=${CACHE_TTL_SECONDS}`,
},
})
// Don't await — fire-and-forget cache put
cache.put(cacheKey, cacheResponse)
return data
} catch (err) {
console.error("Failed to fetch tenant config:", err)
return null
}
}
/**
* Escape a string for safe injection into an HTML <script> tag.
* Prevents XSS via </script> or <!-- injection.
@@ -153,9 +76,13 @@ export const onRequest: PagesFunction = async (context) => {
return response
}
// Fetch tenant config (cached at edge)
// Tenant config: edge-cached, stale-while-revalidate, bounded wait on a miss.
const cacheKey = `${url.origin}/__tenant_config__/${tenantId}`
const config = await fetchTenantConfigCached(tenantId, cacheKey)
const config = await getTenantConfig(tenantId, cacheKey, {
cache: caches.default,
fetchFn: fetch,
waitUntil: (p) => context.waitUntil(p),
})
if (!config || !config.tenant) {
return response
}
+16 -14
View File
@@ -64,20 +64,22 @@ jest.mock('@/i18n/routing', () => {
}
})
// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: jest.fn().mockImplementation(query => ({
matches: false,
media: query,
onchange: null,
addListener: jest.fn(),
removeListener: jest.fn(),
addEventListener: jest.fn(),
removeEventListener: jest.fn(),
dispatchEvent: jest.fn(),
})),
})
// Mock window.matchMedia (skipped in `@jest-environment node` tests, e.g. edge code)
if (typeof window !== 'undefined') {
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: jest.fn().mockImplementation(query => ({
matches: false,
media: query,
onchange: null,
addListener: jest.fn(),
removeListener: jest.fn(),
addEventListener: jest.fn(),
removeEventListener: jest.fn(),
dispatchEvent: jest.fn(),
})),
})
}
// Mock IntersectionObserver
global.IntersectionObserver = class IntersectionObserver {
+144
View File
@@ -0,0 +1,144 @@
/**
* Edge (Cloudflare Pages Function) cache for the tenant portal's config.
*
* Stale-while-revalidate: a cached config is served even when older than
* FRESH_SECONDS, and refreshed in the background. A request only waits on the
* backend when its colo has never cached the tenant, and then for at most
* FETCH_TIMEOUT_MS (the client fetches the config itself if it isn't injected).
*
* Why (2026-10-13): with a plain 5-min TTL, a crawler loading 17 portal pages at
* once missed the cache 17 times in the same second. That burst made Cloud Run
* spawn 5 cold API instances, and the HTML responses waited 15-17s on them.
*/
export interface TenantConfigResponse {
tenant: {
id: string
name: string
subdomain: string
is_active: boolean
branding: {
primary_color: string
secondary_color: string
accent_color: string | null
background_color: string | null
favicon_url: string | null
site_title: string
}
} | null
is_default: boolean
}
export const FRESH_SECONDS = 300 // refresh in the background after 5 minutes
export const STORE_SECONDS = 7 * 24 * 3600 // keep a stale copy for a week
export const FETCH_TIMEOUT_MS = 3000
const FETCHED_AT_HEADER = "X-Fetched-At"
interface EdgeCache {
match(key: string): Promise<Response | undefined>
put(key: string, response: Response): Promise<void>
}
export interface TenantConfigDeps {
cache: EdgeCache
fetchFn: typeof fetch
waitUntil: (p: Promise<unknown>) => void
now?: () => number
apiBaseUrl?: string
}
/** Minimal shape check — a malformed 200 must not be cached for a week. */
function isTenantConfig(data: unknown): data is TenantConfigResponse {
if (!data || typeof data !== "object") return false
const d = data as { tenant?: unknown; is_default?: unknown }
if (typeof d.is_default !== "boolean") return false
if (d.tenant === null) return true
const t = d.tenant as { id?: unknown; branding?: { primary_color?: unknown; secondary_color?: unknown } } | undefined
return (
!!t &&
typeof t.id === "string" &&
typeof t.branding?.primary_color === "string" &&
typeof t.branding?.secondary_color === "string"
)
}
// Concurrent misses inside one isolate share a single backend fetch.
const inFlight = new Map<string, Promise<TenantConfigResponse | null>>()
async function fetchAndStore(
tenantId: string,
cacheKey: string,
deps: TenantConfigDeps
): Promise<TenantConfigResponse | null> {
const existing = inFlight.get(cacheKey)
if (existing) return existing
const now = deps.now ?? Date.now
const base = deps.apiBaseUrl ?? "https://api.nomadkaraoke.com"
const p = (async () => {
try {
const response = await deps.fetchFn(
`${base}/api/tenant/config?tenant=${encodeURIComponent(tenantId)}`,
{
headers: { "Content-Type": "application/json", "X-Tenant-ID": tenantId },
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
}
)
if (!response.ok) {
console.error(`Tenant config fetch failed: ${response.status}`)
return null
}
const data: unknown = await response.json()
if (!isTenantConfig(data)) {
console.error("Tenant config fetch returned an unexpected payload")
return null
}
await deps.cache.put(
cacheKey,
new Response(JSON.stringify(data), {
headers: {
"Content-Type": "application/json",
"Cache-Control": `s-maxage=${STORE_SECONDS}`,
[FETCHED_AT_HEADER]: String(now()),
},
})
)
return data
} catch (err) {
console.error("Failed to fetch tenant config:", err)
return null
} finally {
inFlight.delete(cacheKey)
}
})()
inFlight.set(cacheKey, p)
return p
}
export async function getTenantConfig(
tenantId: string,
cacheKey: string,
deps: TenantConfigDeps
): Promise<TenantConfigResponse | null> {
const now = deps.now ?? Date.now
const cached = await deps.cache.match(cacheKey)
if (cached) {
try {
const fetchedAt = Number(cached.headers.get(FETCHED_AT_HEADER) || 0)
const data: TenantConfigResponse = await cached.json()
if (!fetchedAt || now() - fetchedAt > FRESH_SECONDS * 1000) {
// Stale (or written before this header existed): serve it, refresh behind.
deps.waitUntil(fetchAndStore(tenantId, cacheKey, deps))
}
return data
} catch {
// Corrupted cache entry — fall through to a fetch.
}
}
return fetchAndStore(tenantId, cacheKey, deps)
}
/** Test helper: forget in-flight fetches between tests. */
export function _resetInFlightForTests(): void {
inFlight.clear()
}
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.268.5"
version = "0.268.6"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"