Cloudflare Worker Operations¶

The sapari-proxy-staging and sapari-proxy-production Workers front staging.sapari.io and app.sapari.io. Both Workers run the same code (worker/src/) — the difference is env-specific bindings (R2 buckets, backend URL, secret values).

The Worker handles two path prefixes:

• /media/v1/<jwt> — verifies a media JWT, streams bytes from R2 via the native binding, caches at the edge. (Stage 3, live on staging since 2026-04-19.)
• /api/* — proxies to the Hetzner backend (api-staging.sapari.io / api.sapari.io) as a same-origin API path. Preserves SSE semantics.

Everything else is Pages — the SPA (sapari-staging / sapari-production Pages projects) serves directly without going through the Worker.

Local dev does not run the Worker. Instead, the backend exposes a GET /media/v1/{token} route (in interfaces/main.py, gated to ENVIRONMENT in {DEVELOPMENT, LOCAL}) that verifies the same JWT and 302-redirects to a presigned MinIO URL. Vite proxies /media/* to the backend. The handler returns 404 outside dev so a stray request can never reach production fallback logic. See docs/architecture/storage.md §Local Development for the wiring.

This doc is the operational runbook. For architecture rationale, see architecture/decisions.md. For annual rotation of the JWT signing secret, see media-token-rotation.md. For the overall R2 migration plan, see R2_MEDIA_PROXY_PLAN.md.

The moving parts¶

Five systems contribute to a working Worker deploy. When playback or API proxying breaks, the fault is almost always in one of them — and the symptoms for each are distinct. Read the troubleshooting matrix below before poking anything.

The split between wrangler.toml and dashboard isn't a preference — both were tried. Custom domain and Pages binding produce specific errors when declared in wrangler.toml. Route patterns can technically go in [[routes]] but conflict with the dashboard-managed custom domain in practice. The dashboard is authoritative for routing; wrangler is authoritative for code.

Route patterns, not catch-all¶

The Worker is attached to its custom domain via route patterns like staging.sapari.io/api/*, not as a catch-all custom domain. This means:

• staging.sapari.io/api/* — captured by the Worker (route pattern added in dashboard).
• staging.sapari.io/media/v1/* — captured by the Worker (route pattern added in dashboard).
• Everything else (including /, /assets/*, /favicon.svg) — goes straight to the Pages project.

The env.ASSETS.fetch(request) fallback in worker/src/index.ts only runs for requests that reach the Worker. Requests that don't match any route pattern never reach it. This is why adding a new handler in src/index.ts is not sufficient on its own — the route pattern in the dashboard also has to be added, or the request will bypass the Worker entirely.

Adding a new path prefix is a two-step change:

1. Add the handler branch in worker/src/index.ts and deploy.
2. In the dashboard: Workers & Pages → sapari-proxy-<env> → Settings → Domains & Routes → Add → Route → <domain>/<prefix>/* on zone sapari.io.

If step 2 is skipped, the request returns the SPA's index.html (Pages' SPA fallback) with 200 status. The Worker will show no corresponding entry in wrangler tail. This is the highest-probability failure mode for "I deployed but it's not working."

Cache-Control policy (per-key-type)¶

buildResponseHeaders in worker/src/media.ts selects Cache-Control by R2 key suffix. The suffix list lives in worker/src/constants.ts:IMMUTABLE_KEY_SUFFIXES — matching any entry opts the response into public, max-age=31536000, immutable (1 year). Everything else gets the default public, max-age=3600 (1 hour).

When adding a new content-addressable key type (where the bytes for a given key are immutable for the lifetime of that key), append the suffix to IMMUTABLE_KEY_SUFFIXES rather than introducing a separate handler. The list is deliberately explicit — no .jpg catchall — so a new key shape can't accidentally inherit immutable caching before the operator has thought about it.

Deploy from scratch (full env, e.g. production cutover)¶

Follow these steps in order. Each step is independently verifiable — if something breaks, the symptom points to the step that failed.

1. Backend env vars on the server¶

SSH to the Hetzner server for the env being provisioned (production: whatever IP; staging: deploy@100.110.63.6 over Tailscale). Edit /home/deploy/sapari/.env:

MEDIA_TOKEN_SECRET=<openssl rand -base64 32>
MEDIA_TOKEN_KID=v1
MEDIA_TOKEN_TTL_SECONDS=300
MEDIA_PROXY_BASE_URL=https://<env-domain>   # staging.sapari.io or app.sapari.io

Restart the backend (docker compose restart backend or the env's equivalent). Verify the startup log:

docker logs sapari-backend --since=1m | grep media_token
# Expect: media_token: active=v1 registry=[v1:<8-char-hex>]

Record the fingerprint (the <8-char-hex> after v1:). It will be compared against the Worker's log in step 5.

2. Cloudflare Worker secret¶

From your laptop — not the server — in worker/:

npx wrangler secret put MEDIA_TOKEN_SECRET_V1 --env <staging|production>
# Paste the SAME value as MEDIA_TOKEN_SECRET from step 1.

The --env flag selects which Worker (sapari-proxy-staging or sapari-proxy-production). The secret name must be MEDIA_TOKEN_SECRET_V1 — the Worker's registry builder in worker/src/kid-registry.ts reads that exact name.

Verify:

npx wrangler secret list --env <staging|production>
# Expect: includes MEDIA_TOKEN_SECRET_V1

If the list is empty, the secret put didn't stick — re-run. If the list shows a V1 but playback still fails, the fingerprint diff in step 5 will catch a value mismatch.

3. Worker code deploy¶

From worker/ on your laptop:

npm run deploy:<staging|production>

The deploy output will include Uploaded sapari-proxy-<env> and Current Version ID: <uuid>. It may also say No deploy targets — this is not an error. With workers_dev = false and no [[routes]] in wrangler.toml, wrangler has nothing to auto-promote. The upload succeeded; the version is queued.

Promote the version to 100%:

npx wrangler versions deploy --env <staging|production>
# Interactive — select the latest version ID, confirm.

Skip this step if wrangler deploy already reported the version as active (shown as (100%) at the top of the versions deploy prompt when you run it). In that case, Ctrl-C out.

4. Add route patterns in the dashboard¶

For a first-time env provisioning, both routes need to be added. For an existing env that already proxies /api/*, only the /media/v1/* route is missing.

In the Cloudflare dashboard: Workers & Pages → sapari-proxy-<env> → Settings → Domains & Routes → Add → Route:

• Zone: sapari.io
• Route: <env-domain>/api/*
• Route: <env-domain>/media/v1/*

Where <env-domain> is staging.sapari.io or app.sapari.io.

Add both routes. Save. DNS does not need to change — the routes attach to the existing custom-domain DNS records.

The dashboard should also show the env.ASSETS service binding pointing at the sapari-<env> Pages project. If it doesn't — see worker/README.md for the recovery procedure.

5. Verify fingerprints match¶

From your laptop:

npx wrangler tail --env <staging|production>

In a browser, reload a page on the env domain that triggers a Worker request (e.g. load a clip in the editor — the /api/v1/projects/.../clips/.../proxy call will hit the Worker). The tail should print:

media_token: active=v1 registry=[v1:<8-char-hex>]

The fingerprint hex must be identical to the backend's log from step 1. If it diverges, the secret value on the Worker does not match the backend — re-run step 2 carefully (the value must be byte-identical, no trailing newline, no base64-decoding in between).

6. Smoke in the browser¶

Load a clip on the env domain. The <video> element should request <env-domain>/media/v1/<jwt> and receive a 200 (or 206 for range requests) with a video/* Content-Type. If it receives Content-Type: text/html and the SPA's index.html body, the route pattern from step 4 is missing or mis-scoped.

If it receives 401 with an empty body, the fingerprints match (step 5 passed) but verification failed anyway — check the wrangler tail for the sanitized reject reason (unknown-kid, invalid: <reason>, etc.).

Deploy an update (Worker code change only)¶

For iterative changes — bug fix, new route handler, logging tweak — after the initial setup is working:

cd worker/
npm run deploy:<env>
npx wrangler versions deploy --env <env>    # only if wrangler deploy reports No deploy targets

Then wrangler tail and reload to confirm the change is live. No dashboard steps needed unless you're adding a new route pattern.

Rolling back¶

npx wrangler rollback --env <env>

Picks the previous deployment. Or use the dashboard: Workers → sapari-proxy-<env> → Deployments → pick an earlier version → Deploy.

Rolling back reverts Worker code only. Route patterns, custom domain, ASSETS binding, and secrets all persist across code rollbacks — they're managed separately.

Troubleshooting¶

Match the symptom to the diagnosis. Each row includes the specific log or UI check that confirms it.

Symptom: /media/v1/<jwt> returns 200 with Content-Type: text/html and the SPA body¶

Cause: The request isn't reaching the Worker — Pages is serving its index.html SPA fallback.

Confirm: In wrangler tail, no /media/v1/* entries appear even though the browser is hitting the URL.

Fix: Add the <env-domain>/media/v1/* route pattern in the dashboard (step 4 above). No redeploy needed; route patterns take effect on the next request.

Symptom: curl https://staging.sapari.io/media/v1/<anything> returns 302 to sapari-io.cloudflareaccess.com¶

Cause: Cloudflare Access (Zero Trust) is gating staging.sapari.io and intercepts requests before they reach the Worker. Unauthenticated probes get redirected to the CF Access login page; the Worker never runs and wrangler tail shows nothing for the request. This is by design — defense in depth — but easy to mistake for "the Worker is broken."

Confirm: Response has location: https://sapari-io.cloudflareaccess.com/cdn-cgi/access/login/staging.sapari.io?... and a CF_AppSession cookie is being set.

Fix options: 1. Real-browser test: log in to staging.sapari.io once, then exercise /media/v1/* from DevTools: fetch('/media/v1/<jwt>', { credentials: 'include' }). The CF_AppSession cookie rides with the request. 2. Service token (for automated probes): create a service token in Cloudflare Zero Trust → Access → Service Tokens, then curl -H "CF-Access-Client-Id: <id>" -H "CF-Access-Client-Secret: <secret>" .... Rotate periodically. 3. Skip the manual probe entirely: mint spans + Worker reject logs accumulate naturally from real browser traffic. Waiting 24h is often faster than setting up service-token auth.

Production may or may not have the same Access policy — confirm in the Cloudflare Zero Trust dashboard → Access → Applications before assuming.

Symptom: /media/v1/<jwt> returns 401 with an empty body¶

Cause A: Worker has no matching secret for the token's kid.

Confirm: npx wrangler secret list --env <env> is empty or missing MEDIA_TOKEN_SECRET_V1.

Fix: npx wrangler secret put MEDIA_TOKEN_SECRET_V1 --env <env> with the same value as the server's MEDIA_TOKEN_SECRET.

Cause B: Secret exists but fingerprint doesn't match backend.

Confirm: Backend log fingerprint (docker logs sapari-backend | grep media_token) and Worker cold-start log fingerprint (wrangler tail) differ.

Fix: Re-run wrangler secret put with the exact server value. Values must be byte-identical. Common mistakes: trailing newline from cat, extra base64-decode step, wrong .env line parsed.

Cause C: Token legitimately expired (TTL is 300 seconds).

Confirm: Browser network tab shows the /proxy mint call returning a token, and the subsequent /media/v1/<jwt> call 401-ing within seconds. That's not expiry — that's A or B. Genuine expiry only happens after 5+ minutes of idle.

Fix: The frontend retry-as-contract handler refetches and retries transparently — the user sees at most a brief buffering overlay while the new source loads. Live on desktop and mobile (SwipeReview, FocusMode) as of Stage 4. Asset playback also routes through the Worker (/media/v1/<jwt> with the assets bucket dispatched via the JWT's bkt claim), but MobileAssetEditor does not yet mount the retry-as-contract layer for asset video tags — refresh on JWT expiry there relies on a fresh URL fetch on the next interaction.

Symptom: wrangler deploy says "No deploy targets for sapari-proxy-"¶

Not an error. With workers_dev = false and no [[routes]] in wrangler.toml, wrangler has no auto-promotion target. The upload succeeded.

Action: Run npx wrangler versions deploy --env <env>. If the interactive prompt shows the latest version already at (100%), hit Ctrl-C — nothing to do.

Symptom: Backend fails to start after deploying new code¶

Cause: MEDIA_TOKEN_SECRET is missing, empty, or shorter than 32 bytes.

Confirm: docker logs sapari-backend | tail -20 shows a ValueError from MediaTokenService.__init__.

Fix: Add or correct MEDIA_TOKEN_SECRET in /home/deploy/sapari/.env, generated via openssl rand -base64 32. Restart the backend.

Symptom: wrangler secret list --env <env> shows unexpected secrets / missing expected secrets¶

Secrets are per-Worker, per-env. --env staging lists secrets on sapari-proxy-staging; --env production lists secrets on sapari-proxy-production. They're separate. A secret put against the wrong env is a silent success against a different Worker.

Fix: Always verify the Worker name in the output (sapari-proxy-staging vs sapari-proxy-production) matches your intent.

Symptom: Deploy wipes the env.ASSETS Pages binding¶

Hasn't happened in practice, but possible if wrangler behavior changes. Wrangler typically leaves bindings it doesn't know about alone.

Fix: Dashboard → Workers → sapari-proxy-<env> → Settings → Variables and Secrets / Bindings → add Service binding named ASSETS → select the sapari-<env> Pages project.

Related¶

• worker/README.md — quick-reference deploy commands, the wrangler.toml design, and the dashboard-managed bindings list.
• media-token-rotation.md — annual and on-incident rotation procedure for the JWT signing secret.
• external-services.md — per-service provisioning runbook (Neon, R2, DNS, Stripe, etc.).
• architecture/decisions.md — rationale for the Worker-as-same-origin-proxy architecture.
• R2_MEDIA_PROXY_PLAN.md (repo root) — full Stage-0-through-Stage-6 migration plan for the R2 media proxy.