5  Why Caddy + Envoy

Note

Caddy sidecars for per-pod TLS and forward auth. Envoy as the sole rootful component for L4 SNI passthrough and L7 load balancing.

5.1 The two-layer ingress model

  • Outer layer: Envoy runs as the only rootful container, binds ports 80 and 443 on the host
  • Inner layer: Caddy runs as a sidecar in each pod, terminates TLS and handles forward auth
  • Envoy does L4 SNI passthrough: it reads the TLS ClientHello, routes by hostname, and forwards the encrypted connection to the correct pod’s Caddy
  • Each Caddy sidecar independently manages its own certificate via DNS-01 with Cloudflare

5.2 Why not a single reverse proxy

  • A single Nginx/Caddy/Traefik at the edge means one rootful process with access to every TLS private key
  • The sidecar model distributes trust: each pod’s Caddy only holds one key, runs as one service user
  • Adding a service means adding a pod with its own Caddy; no central config to update (Envoy’s config is the exception)

5.3 Caddy sidecars

  • Each pod includes a caddy-cloudflare container (Caddy built with the Cloudflare DNS plugin)
  • Caddyfile pattern: {fqdn} block with tls { dns cloudflare {env.CF_API_TOKEN} } and reverse_proxy localhost:{port}
  • For services behind Authentik: forward_auth directive sends subrequests to the Authentik outpost
  • CF_API_TOKEN is a podman secret mounted into the Caddy container

5.4 Envoy ingress

  • Envoy config is a static YAML with one SNI filter chain per FQDN, each routing to 127.0.0.1:{port} on the host
  • HTTP :80 redirects to HTTPS
  • For GPU inference (vLLM): Envoy does L7 with STRICT_DNS, health checks, and circuit breakers to support multiple backend instances
  • Envoy is the only service that needs to see all hostnames; everything else is decentralized

5.5 DNS

  • All *.dunn.dev records point to the host’s public IP via Cloudflare DNS
  • Internal services resolve to 127.0.0.1 via /etc/hosts entries managed by pyinfra (services that need to reach each other, e.g., Caddy → Authentik for forward auth)
  • Tailscale for administrative access to services that should not be public

5.6 Forward auth flow

  • Caddy’s forward_auth sends a subrequest to the Authentik outpost at https://auth.dunn.dev/outpost.goauthentik.io/auth/caddy
  • Authentik validates the session cookie; if missing or expired, redirects to login
  • After authentication, Authentik sets headers (X-authentik-username, etc.) that Caddy forwards to the upstream app
  • Per-service authorization is configured in Authentik as application-specific policies