jd five

Load Balancers & Proxies

Learning goals

By the end of this chapter you should be able to:

  • Distinguish Layer 4 (L4) and Layer 7 (L7) load balancing
  • Explain what a reverse proxy does and why teams put one in front of apps
  • Describe TLS termination at the edge and what changes on the backend hop
  • Reason about health checks and why “all nodes green” still fails users

Production traffic rarely goes straight from the Internet to one lonely server. Something in the middle distributes, protects, or transforms connections — that something is usually a load balancer or reverse proxy.

Plain English: why not one server?

A popular API might run dozens of identical app instances. Users should not pick which machine they hit. A load balancer (LB) sits in front and spreads incoming connections or requests across healthy backends.

Benefits beyond “more CPUs”:

  • High availability — if one instance dies, traffic goes elsewhere.
  • Rolling deploys — drain or remove nodes without a global outage.
  • SSL/TLS at the edge — one place to manage certificates (ties to the PKI chapter).
  • Security filtering — WAF, rate limits, IP allowlists (often L7).

Precise term: load balancer = distribution policy + health awareness. Reverse proxy = client talks to proxy; proxy talks to origin servers (often the same box in cloud offerings).

Layer 4 vs Layer 7: what the LB understands

Networking people label layers roughly like the OSI model. For interviews, two levels dominate:

Layer 4 (transport) — “I see IP and port”

The LB makes decisions using TCP/UDP information:

  • Source/destination IP and port
  • Protocol (TCP vs UDP)
  • Sometimes connection rate, SYN cookies

It does not parse HTTP. It might forward the raw TCP stream to a chosen backend (TCP pass-through) or NAT the connection.

Good for: extreme throughput, non-HTTP protocols (database proxies with care), TLS pass-through (encrypted blob forwarded; backend terminates TLS).

Examples (conceptual): AWS Network Load Balancer (NLB) for TCP/UDP, HAProxy in mode tcp.

Layer 7 (application) — “I understand HTTP”

The LB terminates the client connection (often), parses HTTP (method, path, headers, cookies), and routes based on that:

  • /api/* → API pool
  • Host: tenant-a.example.com → tenant A cluster
  • Header X-Version: canary → canary pool

Can inject headers (X-Forwarded-For, X-Request-Id), gzip, redirect HTTP→HTTPS.

Good for: path-based routing, sticky sessions via cookies, WAF integration.

Examples: AWS Application Load Balancer (ALB), nginx as HTTP reverse proxy, Envoy with HTTP filters.

Question L4 L7
Routes on URL path? No Yes
Sees HTTP headers? No Yes
Typical TLS terminate at LB? Optional pass-through Very common
Overhead Lower Higher (parse + logic)

Memory hook: L4 = packets/connections; L7 = requests.

Reverse proxy vs forward proxy

Reverse proxy: clients on the Internet hit the proxy; proxy fetches from your servers. Users often do not know backend hostnames. CDNs and LBs act as reverse proxies.

Forward proxy: your users (or apps) go through a proxy to reach the Internet — corporate egress, “squid” style. Different use case.

Interviewers usually mean reverse when they ask about nginx in front of Node or Java.

TLS termination: where HTTPS ends

Three common patterns:

  1. Terminate at LB — Client ↔ LB is HTTPS (LB holds public cert). LB ↔ app might be HTTP on a private network or re-encrypted HTTPS (TLS re-encrypt).

  2. Pass-through — LB forwards encrypted TCP; app terminates TLS. LB never sees plaintext HTTP. Harder L7 routing unless you use SNI routing at L4.

  3. End-to-end TLS to app — Client HTTPS to LB, LB HTTPS to backend with different cert. More CPU, stronger isolation between networks.

Why terminate early:

  • Central cert management (ACME at edge).
  • Hardware/crypto offload.
  • WAF needs plaintext HTTP headers/body.

Tradeoff: plaintext on the internal hop must ride a trusted network (VPC, mTLS between LB and app in strict shops).

Headers clients lose unless proxy adds them:

  • X-Forwarded-For — original client IP (trust only from your LB).
  • X-Forwarded-Proto: https — so apps generate correct absolute URLs.
  • Forwarded — standardized alternative (RFC 7239).

Algorithms (just enough): round robin, least connections, weighted, consistent hashing. Sticky sessions via cookie tie a user to one backend — uneven load unless session state lives in Redis.

Health checks: the LB’s “are you alive?” probe

LBs periodically probe backends:

  • HTTP GET /health expecting 200
  • TCP connect to port 8080
  • gRPC health service

If probes fail threshold → backend drained from pool.

Common mistakes:

  • Health check hits /health but app hangs on real DB calls — superficial green, user traffic 500s.
  • Check from LB security group blocked by backend firewall.
  • Graceful shutdown: app must fail health before killing in-flight work, or use connection draining window.

Kubernetes tie-in: readinessProbe removes pod from Service endpoints; livenessProbe restarts it.

Common mistakes

  1. Assuming L7 LB can route on URL while in TCP pass-through mode — encrypted traffic hides the path.
  2. Trusting X-Forwarded-For from the public Internet — only trust hops you control.
  3. Health endpoint shares fate with broken dependency — too deep checks flap entire fleet.
  4. No connection draining on deploy — cut active checkout flows mid-request.
  5. Confusing NLB/ALB/CLB product names without naming L4 vs L7 behavior.

Practice checkpoint

  1. L4 LB — can it route GET /api vs GET /static differently without TLS termination? Answer: not by reading HTTP; only if TLS terminates or SNI/other L4 tricks used — generally need L7 or terminate TLS first.

  2. Where is the certificate the browser validates if TLS terminates at the LB? Answer: the LB’s certificate, not the backend app’s (unless pass-through).

  3. Why add X-Forwarded-Proto? Answer: backend sees HTTP to LB but client used HTTPS; app needs to know original scheme for redirects and cookies.

  4. Backend fails TCP health but returns 200 on /health — what happened? Answer: misconfigured probe (wrong port/path), or firewall allows HTTP path but not probe source — debug probe from LB subnet.

  5. Sticky sessions — one downside? Answer: uneven load; backend loss drops sessions unless state is external.

Interview angles

  • L4 vs L7 with one concrete routing example each.
  • TLS termination tradeoffs: ops simplicity vs end-to-end encryption.
  • Single point of failure? LBs are often paired or managed with anycast/VIP failover.
  • Reverse proxy benefits: caching (CDN chapter), compression, rate limiting, hiding origin topology.
  • Blue/green and canary at L7: weight traffic by header or percentage.

Next: /networking/learn/systems/cdn-and-edge/