Load Balancers: The Next Step After Mastering Docker

When One Container Is No Longer Enough

Remember that day when you ran your first app in Docker and felt like a DevOps god? docker run, docker-compose up — and everything works. Beautiful. But then real traffic comes: one container starts choking, users complain about slowness, and you realize you need to scale somehow.

You spin up a second container. A third. And then the question arises: how do you distribute traffic between them? Which container gets the next request? What happens if one crashes? This is where Load Balancer comes in — a tool that knows the answers to these questions.

Why You Need This in Real Life

Let's skip the theory. Here are three practical scenarios where a load balancer solves your problems:

Scenario 1: E-commerce on Friday Night

You run an online store. On weekdays — 100 RPS, one container handles it. Friday night — 800 RPS, and your single container dies. With a load balancer, you run 5 app copies, and it distributes the load. Result: store works, you make money.

Scenario 2: Zero-Downtime Updates

You need to update your code. Without a load balancer: stop container → deploy new version → start. Site is down for 30-60 seconds. With a load balancer: start new containers → verify → switch traffic → kill old ones. Downtime = 0.

Scenario 3: Fault Tolerance Out of the Box

One container crashes (OOM, network failure, cosmic rays). Without a load balancer — users see 502. With a load balancer — it sends requests only to healthy instances, problem is invisible from outside.

How It Works Under the Hood

Think of a Load Balancer as a smart taxi dispatcher. Clients (requests) come to it, and it decides which driver (container) gets the next order. While doing this, it:

1. Checks driver health (health checks) — won't send a client to a broken car
2. Distributes fairly — won't give one driver 10 orders while others sit idle
3. Remembers context (session affinity) — if a client started a conversation with one driver, keeps them together

Technically it looks like this:

User → Load Balancer (port 80/443) → Backend servers (containers)
              ↓
      Health checks every N seconds
              ↓
      Routing by algorithm

Load Balancing Strategies (Which to Choose)

Load balancers use different algorithms to distribute load. Choice depends on your scenario:

Round Robin

What: Requests go in order: 1st container → 2nd → 3rd → back to 1st.

When to use: All containers are identical, requests are roughly the same complexity. The simplest and most popular option.

Example: API for a mobile app where each request is roughly the same load.

Least Connections

What: New request goes to the container with the fewest active connections.

When to use: Requests of varying complexity — some finish in 50ms, others in 5 seconds.

Example: SaaS platform where you have fast GET requests and heavy POSTs with data processing.

IP Hash

What: Client IP is hashed, and requests from one IP always go to the same container.

When to use: You have application-level state (in-memory cache, sessions), and want to minimize problems with it.

Example: Legacy app with in-memory sessions (though it's better to move sessions to Redis).

Weighted Round Robin

What: Like round robin, but some containers get more requests than others.

When to use: Your containers run on different hardware (e.g., 1 powerful server + 2 weak ones), or you're doing canary deployment (new version gets 10% traffic).

Example: Gradual transition to new version: old version gets 90% requests, new one — 10%.

Popular Load Balancers: Which to Choose

Nginx — Swiss Army Knife

Pros:

• Fast, battle-tested for years
• Can be used as HTTP server + load balancer + reverse proxy
• Huge community, tons of guides
• Can serve static files, cache, terminate SSL

Cons:

• Configuration via text files (not the most convenient for automation)
• For live config reload need nginx -s reload

When to choose: Default choice for most tasks. Proven solution that works everywhere.

Minimal config example:

upstream backend {
    server app1:8000;
    server app2:8000;
    server app3:8000;
}
 
server {
    listen 80;
    server_name example.com;
 
    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

HAProxy — When You Need Power

Pros:

• Extremely fast (handles 100k+ RPS on a single machine)
• Advanced balancing algorithms and health checks
• TCP and HTTP load balancing (can balance PostgreSQL, Redis, not just HTTP)
• Detailed statistics and metrics out of the box

Cons:

• Just a load balancer, doesn't serve static files or terminate SSL (well, it can, but that's not its main role)
• Config syntax is specific, harder for beginners

When to choose: High-load systems when you need maximum performance and detailed control.

Config example:

frontend http_front
    bind *:80
    default_backend app_servers
 
backend app_servers
    balance roundrobin
    option httpchk GET /health
    server app1 app1:8000 check
    server app2 app2:8000 check
    server app3 app3:8000 check

Traefik — Modern and Cloud-Native

Pros:

• Automatic service discovery (integration with Docker, Kubernetes)
• Automatic SSL certificates via Let's Encrypt
• Beautiful web interface with real-time stats
• Configuration via Docker labels or Kubernetes annotations

Cons:

• Heavier than Nginx/HAProxy on resources
• Can be overkill for simple tasks
• Fewer tutorials and articles than for Nginx

When to choose: Microservices architecture, Docker Swarm, Kubernetes. When you want automation and don't want to manually edit configs.

Example with Docker Compose:

version: "3"
 
services:
  traefik:
    image: traefik:v2.10
    command:
      - "--providers.docker=true"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
 
  app:
    image: myapp:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app.rule=Host(`example.com`)"
    deploy:
      replicas: 3

Practical Example: Running Nginx + 3 Containers

Let's manually build a minimal setup with Nginx and three app copies.

docker-compose.yml:

version: "3.8"
 
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app1
      - app2
      - app3
 
  app1:
    image: hashicorp/http-echo
    command: ["-text", "Hello from app1"]
    expose:
      - "5678"
 
  app2:
    image: hashicorp/http-echo
    command: ["-text", "Hello from app2"]
    expose:
      - "5678"
 
  app3:
    image: hashicorp/http-echo
    command: ["-text", "Hello from app3"]
    expose:
      - "5678"

nginx.conf:

events {
    worker_connections 1024;
}
 
http {
    upstream backend {
        server app1:5678;
        server app2:5678;
        server app3:5678;
    }
 
    server {
        listen 80;
 
        location / {
            proxy_pass http://backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
 
        location /health {
            access_log off;
            return 200 "OK\n";
            add_header Content-Type text/plain;
        }
    }
}

Running:

docker-compose up

Testing:

# Make several requests
curl http://localhost
# Hello from app1
 
curl http://localhost
# Hello from app2
 
curl http://localhost
# Hello from app3
 
curl http://localhost
# Hello from app1 (cycle repeats)

Health Checks — How the Balancer Knows About Problems

The load balancer isn't psychic. You need to explicitly tell it how to check backend health. Usually it's an HTTP endpoint that returns 200 OK if everything's fine.

Simple health check endpoint (Python/FastAPI):

from fastapi import FastAPI
 
app = FastAPI()
 
@app.get("/health")
async def health():
    # Here you can check DB, Redis, etc.
    return {"status": "ok"}

Nginx configuration:

upstream backend {
    server app1:8000 max_fails=3 fail_timeout=30s;
    server app2:8000 max_fails=3 fail_timeout=30s;
}

HAProxy configuration (more advanced):

backend app_servers
    option httpchk GET /health
    http-check expect status 200
    server app1 app1:8000 check inter 5s fall 3 rise 2
    server app2 app2:8000 check inter 5s fall 3 rise 2

Explanation:

• check inter 5s — check every 5 seconds
• fall 3 — mark server as down after 3 failed checks
• rise 2 — mark server as up after 2 successful checks

Common Mistakes and How to Avoid Them

Mistake 1: Forgot About Sticky Sessions

Problem: User logged in on app1, next request went to app2, no session there → logout.

Solution: Either use IP Hash / Cookie-based sticky sessions, or (better) externalize sessions to Redis/PostgreSQL.

Mistake 2: Wrong Headers

Problem: App doesn't see real client IP, sees balancer IP. Analytics and logging break.

Solution: Configure proxy headers correctly:

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

And teach your app to read X-Real-IP or X-Forwarded-For.

Mistake 3: No Load Balancer Monitoring

Problem: Load balancer crashed, and you learned about it from users.

Solution: Monitor:

• Load balancer availability (uptime check)
• Number of healthy backends
• Latency and error rate
• Throughput (RPS)

Use Prometheus + Grafana or cloudwatch/datadog if in the cloud.

When You DON'T Need a Load Balancer

A load balancer isn't a silver bullet. Here's when it's overkill:

1. You have one container and stable load — why complicate things?
2. Local development — docker-compose up without nginx is simpler and faster
3. Stateful services without replication — PostgreSQL, Redis master — nothing to balance (though read-replicas can be balanced)

What's Next: Path to Kubernetes

Load Balancer is the bridge between "I know Docker" and "I know production". Next steps:

1. Scaling automation: now you manually write app1, app2, app3. In Kubernetes it's replicas: 3.
2. Service Discovery: now you manually specify backend addresses. In Kubernetes services find each other automatically.
3. Rolling updates and rollbacks: now you manually switch versions. In Kubernetes it's one command.

But all this is evolution of the same ideas you learned today. Kubernetes is just a load balancer + orchestrator + automation on steroids.

Conclusions

What to remember:

1. A load balancer is a proxy that distributes requests among app copies
2. Algorithm choice depends on your scenario (round robin is default for most)
3. Nginx — universal choice, HAProxy — for high loads, Traefik — for cloud-native
4. Health checks are a mandatory part of setup, otherwise balancer won't know about problems
5. This isn't the end goal, but a stepping stone to orchestrators (Docker Swarm, Kubernetes)

See also:

• Load Testing — how to verify balancing works
• Proxmox — where to deploy your infrastructure