VOD Deep Dive Part 7: CDN Distribution — Why It's Fast Everywhere

CDN architecture (Edge/Shield/Origin), caching strategies, request collapsing, signed URLs, pre-warming, JIT vs pre-packaging, multi-CDN strategies, HTTP/3, and cost estimation.

zhuermu · · 20 min
vodstreamingcdncloudfrontcaching

This is Part 7 of the VOD Streaming Deep Dive series.

Why Video Needs a CDN

Suppose your server is in Shanghai and a US user wants to watch:

US user ────🌊 Trans-Pacific undersea cable 🌊────► Shanghai server
                  (180ms+ latency)
                  (high packet loss)
                  (limited bandwidth)

100,000 concurrent US users at 2 Mbps each = 200 Gbps of cross-ocean traffic. Technically and financially impossible.

CDN (Content Delivery Network) solves this:

              ┌─── New York edge cache ──► US users (20ms)
Shanghai ───►
origin        └─── Frankfurt edge cache ──► EU users (10ms)

After the first origin fetch, popular videos "live" at global edge nodes

Think of a CDN as a global chain of convenience stores. Headquarters (origin) manufactures goods in one place but distributes to every city’s store (edge node). Users buy locally — faster and cheaper.

Three-Layer Architecture

 ┌─────────────────────────────────────────┐
 │            User                         │
 └──────────────┬──────────────────────────┘
                │ ① Request seg_42.m4s
                ▼
       ┌────────────────────┐
       │ Edge PoP           │   Closest to user, within tens of km
       │ (hundreds globally)│   Caches popular content
       └────────────┬───────┘
                    │ Cache miss → go upstream
                    ▼
       ┌────────────────────┐
       │  Mid-tier / Shield │   Regional aggregation layer
       │  (a few per region)│   Reduces origin load
       └────────────┬───────┘
                    │ Cache miss → origin fetch
                    ▼
       ┌────────────────────┐
       │      Origin        │   Object storage (S3/OSS/COS)
       │                    │   or Packager service (MediaPackage)
       └────────────────────┘

Cache Hit Ratio is the most important CDN metric. Target: > 95%.

Origin Pull Flow

1. User → Edge: GET /video/seg_42.m4s
2. Edge checks local cache:
   ├── HIT → return immediately ✅
   └── MISS:
       3. Edge → Shield: GET /video/seg_42.m4s
       4. Shield checks cache:
          ├── HIT → return to Edge → Edge caches → return to user
          └── MISS:
              5. Shield → Origin: GET /video/seg_42.m4s
              6. Origin responds → Shield caches → Edge caches → user

First user triggers full chain (slow). Every subsequent user hits the edge cache (fast). This is why the first viewer of a newly published show has the worst experience — and why pre-warming matters.

Caching Strategy

CDN caching behavior is controlled by HTTP headers:

File typeCache-ControlRationale
Manifest (.m3u8/.mpd)max-age=60 to max-age=300May update (ad insertion, subtitle changes)
Init segment (.mp4)max-age=31536000 (1 year)Immutable
Media segment (.m4s/.ts)max-age=31536000 (1 year)Immutable
Subtitles (.vtt)max-age=3600 (1 hour)May be revised
Thumbnailsmax-age=86400 (1 day)May update

Critical pitfall: If signed URL parameters (?token=xxx) are included in the cache key, every user gets a unique cache entry and the hit rate drops to zero. Always exclude signing parameters from the cache key.

Request Collapsing

When 100 users simultaneously request seg_42.m4s and all miss the cache:

Without collapsing: 100 origin requests → origin overloaded.

With collapsing (default on modern CDNs): The edge recognizes it’s the same object, sends 1 origin request, then distributes the response to all 100 users.

This is a lifesaver during launch events and new episode premieres.

Signed URLs and Anti-Hotlinking

CDN segment URLs are public. Someone could embed them on their own site for free. Signed URLs prevent this:

https://d1234.cloudfront.net/video/seg_42.m4s
  ?Expires=1715084800
  &Signature=nitfHRCrtziwO2HwPf...
  &Key-Pair-Id=APKAEIBAERJR2EXAMPLE

The CDN edge validates: Is it expired? Is the signature correct? Are parameters untampered? Failure → 403 Forbidden.

Other anti-hotlinking methods: Referer whitelist, User-Agent checks, IP geo-restrictions, rate limiting, and signed cookies (more convenient than signed URLs when loading many segments).

Pre-Warming

When a new show launches, all segments are cold (not in edge caches). First-wave users all miss → poor experience.

Pre-warming: After CMS publish, proactively push content to edge nodes:

CMS publish hook → CDN prewarm API
  ├── POST /prewarm { urls: ["…/seg_0001.m4s", ..., "…/seg_0010.m4s"] }
  └── CDN dispatches edge nodes to origin-fetch

Minutes later: global edges have the segments
First-wave users hit cache ✅

Don’t prewarm every segment — focus on: init segment, first 5–10 media segments, all quality tiers for those segments.

JIT Packaging vs Pre-Packaging

Pre-packaging (Offline)

Generate all HLS/DASH/CMAF manifests and segments after transcoding → store in S3 → CDN serves directly.

Pros: Simple, maximum cache hits. Cons: Higher storage (multiple format copies), inflexible (adding a subtitle requires re-packaging).

JIT Packaging (Just-in-Time)

S3 stores only source fMP4. When a user requests a manifest, the origin service generates it on-the-fly.

Pros: Store once, flexible (add DRM or subtitles without re-packaging). Cons: First hit is slower (origin CPU), origin must handle load.

Representative implementations: AWS MediaPackage-VOD, Unified Origin, open-source mp4box.

Recommendation: Small catalog with high traffic → pre-packaging. Large catalog with long tail → JIT packaging.

Multi-CDN Strategy

Why one CDN isn’t enough:

  • Single point of failure: CDN goes down, you go dark
  • Geographic coverage: No single CDN is best everywhere
  • Pricing leverage: Multiple vendors compete for your business
  • Performance: Route to whichever is fastest right now

Routing Methods

DNS-based: 70% traffic to CloudFront, 20% to Cloudflare, 10% to regional CDN.

Client-side probing: App pings all CDNs at startup, picks the fastest; re-probes every 5 minutes.

Managed steering: Cedexis, Conviva Traffic Steering, Route 53 latency-based routing.

Failover

Player logic: if CDN A fails 3 times consecutively → switch to CDN B → report alert.

HTTP/2, HTTP/3, and QUIC

ProtocolKey benefit for video
HTTP/1.1Baseline. Head-of-line blocking is the main bottleneck.
HTTP/2Multiplexing: concurrent requests on one TCP connection. Major improvement for downloading multiple segments.
HTTP/3 / QUICUDP-based. Eliminates TCP head-of-line blocking. 0-RTT reconnection. Dramatic improvement on weak/lossy mobile networks.

All major CDNs (CloudFront, Cloudflare, Akamai) support HTTP/3. Enabling it improves startup time and rebuffer rate, especially on cellular networks.

Cost Estimation

Typical Bandwidth Pricing (after volume discounts)

Region$/GB
North America / Europe$0.005–$0.010
Latin America$0.020–$0.050
Middle East$0.050–$0.080
Asia-Pacific / India$0.015–$0.040

Quick Estimate

Assumptions:
  DAU = 1 million
  Average watch time = 30 min/day
  Average bitrate = 1.2 Mbps (720p H.264)

Per-user daily traffic = 1.2 Mbps × 1800s ÷ 8 ≈ 270 MB
Daily total = 1M × 270 MB = 270 TB
Monthly total = 270 TB × 30 = 8.1 PB

At $0.02/GB (weighted average):
  Monthly CDN cost ≈ 8.1 PB × 1000 × $0.02 = $162,000

Cost Reduction Levers

  1. Better codecs: HEVC saves 37% vs H.264; AV1 saves another 20–30% → directly reduces the bill
  2. Per-title encoding: Custom ladder per title saves 10–20%
  3. Lower default tier: Default to 720p on mobile (40% savings vs 1080p)
  4. Multi-CDN bidding
  5. Volume discounts: > 1 PB/month unlocks significant tiers
  6. Private CDN: Netflix Open Connect places servers inside ISP data centers

Key Takeaways

  1. CDN = cache content close to users in “convenience stores.”
  2. Three-layer architecture: Edge → Shield → Origin.
  3. Cache hit ratio > 95% is the target. Exclude signing parameters from cache keys.
  4. Pre-warm new content to avoid first-wave cold misses.
  5. JIT Packaging saves storage; Pre-packaging is simpler.
  6. Multi-CDN improves resilience, coverage, and pricing.
  7. HTTP/3 significantly improves weak-network performance.
  8. CDN bandwidth is the dominant VOD cost: codec efficiency + tier design + multi-CDN bidding are the three biggest levers.

Previous: Part 6: Adaptive Bitrate Streaming

Next: Part 8: DRM Content Protection