Storage¶

Sapari uses Cloudflare R2 for object storage. R2 is S3-compatible, so we use boto3 and standard S3 patterns. This page covers how we organize files and handle uploads/downloads.

Buckets¶

The canvas-media model uses two live buckets:

The unified media bucket collapses what used to be separate clip and asset stores into one — a single MediaFile row per unique file (deduplicated by sha256_hash or youtube_video_id) holds all of its artifact keys. Exports live in their own bucket so they can carry different retention policies and access controls.

The legacy STORAGE_BUCKET_RAW and STORAGE_BUCKET_ASSETS buckets (from the pre-canvas Clip/Asset model) are being decommissioned; they no longer back any main-path upload or read.

Key Structure¶

Files are organized with a prefix based on the first 2 characters of the UUID. This helps S3/R2 distribute files across partitions for better performance at scale.

# Media bucket (unified)
media/ab/ab123456-.../{original_filename}
media/ab/ab123456-.../audio
media/ab/ab123456-.../proxy.mp4
media/ab/ab123456-.../sprite.jpg
media/ab/ab123456-.../thumbnail.jpg

# Exports bucket
exports/{project_uuid}/{export_uuid}/Final Cut v1.mp4

The MediaFile model stores these paths in storage_key, audio_key, proxy_key, sprite_key, and thumbnail_key; sprite_seconds_per_tile (integer, nullable) persists the density chosen at generation time so the frontend can map source-time to sprite tile, and the peak-array timeline data lives inline in waveform_json.

Upload Flow¶

We use presigned URLs for uploads so files go directly to R2 without passing through our API servers. This is faster and cheaper.

# 1. Client requests presigned URL
POST /api/v1/media/presign
{
    "filename": "recording.mp4",
    "content_type": "video/mp4",
    "size_bytes": 104857600
}

# 2. Server creates MediaFile + MediaItem rows and returns presigned PUT URL
{
    "media_file_id": "ab123456-...",
    "media_item_id": "cd789012-...",
    "upload_url": "https://r2.cloudflare.../media/ab/ab123456.../recording.mp4?X-Amz-...",
    "content_type": "video/mp4",
    "expires_in": 3600
}

# 3. Client PUTs file bytes directly to R2
PUT {upload_url}
Content-Type: video/mp4
Body: <file bytes>

# 4. Client confirms upload completed
POST /api/v1/media/{media_file_id}/confirm

The presigned PUT is valid for 1 hour. R2 does not implement PostObject, so the size cap cannot be bound into the signature (S3-on-AWS would use presigned POST with content-length-range; that's unreachable here). On confirm, the backend HEADs the object and re-checks the user's storage quota against the actual uploaded size, not the client-declared size — this HEAD-based recheck is what actually enforces the cap. If the upload fails, the client can request a new PUT URL.

Download Flow¶

Downloads split into two paths depending on the content type.

Exports — presigned URL, direct to R2¶

One-shot downloads (exported videos) still use presigned R2 URLs:

# Request download URL for an export
GET /api/v1/exports/{uuid}/download

# Response
{
    "url": "https://r2.cloudflare.../exports/...",
    "expires_in": 3600,
    "filename": "Final Cut v1.mp4"
}

The frontend redirects to this URL or uses it in a download link. Export downloads to the Worker path remain a post-launch follow-up.

Media playback — Worker-fronted, JWT-authenticated¶

All media playback (main-video proxies, b-roll, overlays, thumbnails, scrub sprites) routes through a Cloudflare Worker at /media/v1/<jwt> rather than handing the browser a presigned R2 URL directly. Server-internal probes (ffprobe during artifact generation) still use presigned R2 — the Worker JWT model is browser-oriented. The backend mints a short-lived HS256 JWT (MEDIA_TOKEN_TTL_SECONDS, default 300); the Worker verifies it with a kid-based secret registry, fetches bytes from R2 via a native binding, and streams them back through Cloudflare's edge cache. Three reasons for the architecture:

1. Per-request authorization. Presigned URLs are valid-until-expiry for anyone who sees them. JWT + Worker verification lets ownership checks run on every request.
2. Edge caching. The Cache API stores byte-range responses per-colo. Repeat views share bandwidth rather than re-fetching from R2.
3. Shorter leak window. 5-minute JWTs limit blast radius if a URL ever escapes a log or screenshot.

# Request playback URL for a media item
GET /api/v1/media/{media_item_id}/proxy

# Response
{
    "url": "https://staging.sapari.io/media/v1/<jwt>",
    "expires_in": 300,
    "sprite": {
        "url": "https://staging.sapari.io/media/v1/<sprite-jwt>",
        "expires_in": 300,
        "tile_width_px": 160,
        "tile_height_px": 90,
        "tiles_per_row": 10,
        "total_tiles": 200,
        "seconds_per_tile": 1
    }
}

The sprite field is null until proxy generation completes; once the MediaFile has sprite_key + sprite_seconds_per_tile populated, the response includes a minted sprite URL alongside the proxy URL. Sprite responses ship with Cache-Control: public, max-age=31536000, immutable because the URL is content-addressable (media UUID + filename); proxy/original video responses keep the default 1-hour max-age.

Every media item — main video, b-roll, overlay, thumbnail — plays back through this single /media/{media_item_id}/proxy endpoint. The token's bkt claim is set to the unified media bucket; there is one route prefix and one bucket.

See docs/operations/cloudflare-workers.md for the deploy runbook and docs/operations/media-token-rotation.md for the annual rotation runbook. (The original staged migration and sprite-design plans were working documents and have been retired.)

Storage Quotas¶

Each tier has a storage quota enforced at presign time. Only user-uploaded files count toward the quota — YouTube imports and system-generated files (proxy, audio) are excluded.

How it works:

1. Presign — checks user.storage_used_bytes + request.size_bytes against tier quota. Rejects with 422 if over limit.
2. Confirm — atomically increments user.storage_used_bytes by file.size_bytes.
3. Delete — atomically decrements (with greatest(x, 0) guard) if the file is the last reference and not a YouTube import.
4. Reconciliation — daily cron (reconcile_storage_usage, 3 AM) recalculates from actual SUM queries and corrects drift.

The cached counter (User.storage_used_bytes) avoids expensive JOIN queries on every presign. Exports are excluded from the upload quota (handled separately by tier-based retention).

Tier quotas are defined in MB (entitlement/constants.py: TIER_STORAGE_MB) and converted to bytes using BYTES_PER_MB from common/constants.py at presign time.

Local Development¶

For local development, we use MinIO as an S3-compatible object store. Docker Compose sets it up automatically.

There's a quirk with presigned URLs in Docker: the URL generated inside the container points to http://minio:9000 (the Docker network hostname), but browsers need http://localhost:9000.

The StorageClient handles this by remapping URLs:

# Internal (Docker network)
http://minio:9000/bucket/key?signature...

# Remapped for browser
http://localhost:9000/bucket/key?signature...

This happens automatically based on the STORAGE_PUBLIC_ENDPOINT env var.

/media/v1/<jwt> in local dev¶

Staging and production route /media/v1/* to a Cloudflare Worker that verifies the JWT and streams bytes from R2. Locally there is no Worker, so the backend itself stands in: GET /media/v1/{token} in interfaces/main.py verifies the same token via MediaTokenService.verify() and 302-redirects to a browser-reachable presigned MinIO URL. The route is gated to ENVIRONMENT in {DEVELOPMENT, LOCAL} — production returns 404 so a misrouted request never falls through. Vite proxies /media/v1 to the backend in frontend/vite.config.ts so the frontend's media playback paths look identical to the deployed app. The proxy is scoped to /media/v1 rather than /media because the SPA serves its own media-library route at /media — a broader prefix would swallow that navigation and hand it to the backend.

Storage Client¶

The StorageClient class wraps boto3 and provides a clean interface:

from src.infrastructure.storage import get_storage_client

storage = get_storage_client()

# Generate presigned PUT URL (R2 doesn't implement PostObject, so size cannot be signed in)
upload = await storage.generate_upload_url(
    bucket=settings.STORAGE_BUCKET_MEDIA,
    key="media/ab/abc123/video.mp4",
    content_type="video/mp4",
    expires_in=3600,
)
# Returns PresignedUpload(url=..., key=..., bucket=..., content_type=..., expires_in=...)

# Read object metadata (for post-upload quota recheck — authoritative size enforcement)
metadata = await storage.head_object(
    bucket=settings.STORAGE_BUCKET_MEDIA,
    key="media/ab/abc123/video.mp4",
)
# Returns ObjectMetadata(size_bytes=..., content_type=..., etag=...)

# Generate presigned download URL
url = await storage.generate_presigned_download(
    bucket=settings.STORAGE_BUCKET_EXPORTS,
    key="exports/proj123/exp456/output.mp4",
    expires_in=3600,
)

# Upload file directly
await storage.upload_file(
    bucket=settings.STORAGE_BUCKET_MEDIA,
    key="media/ab/abc123/audio",
    file_path=Path("/tmp/audio.wav"),
    content_type="audio/wav",
)

Key Files¶

← Data Flow Events →