Migrating Apps from Docker Swarm to Kubernetes

The old setup was Docker Swarm with Traefik handling TLS termination and routing. The new setup is Kubernetes with Cilium Gateway API doing the same job. The apps themselves don't change — same container images, same ports. It's just the plumbing around them that's different.

What I'm migrating

Three web apps, all simple HTTP servers on port 80:

The old Traefik config used labels on Docker services to define routing rules. In Kubernetes, that translates to HTTPRoutes attached to the Gateway.

Pull secret for GitHub Container Registry

The images live in GitHub Container Registry (ghcr.io), which is private. Kubernetes needs credentials to pull them.

Create a GitHub Personal Access Token: GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic). Only read:packages scope is needed.

Store it as a Kubernetes secret:

apiVersion: v1
kind: Secret
metadata:
  name: ghcr-pull-secret
  namespace: apps
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: |
    {
      "auths": {
        "ghcr.io": {
          "username": "<your-github-username>",
          "password": "<your-token>"
        }
      }
    }

This file goes in k8s/ghcr-pull-secret.yaml and is gitignored. Apply it:

kubectl apply -f k8s/ghcr-pull-secret.yaml

Every Deployment that pulls from ghcr.io references this secret via imagePullSecrets.

App manifests

Each app is a Deployment + Service + HTTPRoute. The pattern is identical for all three, just different images and hostnames. Here's the structure for one:

Deployment + Service (apps/app-name.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-name
  namespace: apps
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app-name
  template:
    metadata:
      labels:
        app: app-name
    spec:
      imagePullSecrets:
        - name: ghcr-pull-secret
      containers:
        - name: app-name
          image: ghcr.io/user/image:tag
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: app-name
  namespace: apps
spec:
  selector:
    app: app-name
  ports:
    - port: 80
      targetPort: 80

HTTPRoute (apps/app-name-httproute.yaml):

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app-name-route
  namespace: apps
spec:
  parentRefs:
    - name: main-gateway
      namespace: gateway
  hostnames:
    - "subdomain.yourdomain.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: app-name
          port: 80

The hostnames field is where the magic happens — Gateway API routes traffic to the right service based on the Host header. This replaces Traefik's Host() rule.

Apply all three:

kubectl apply -f apps/main-site.yaml -f apps/main-site-httproute.yaml
kubectl apply -f apps/blog.yaml -f apps/blog-httproute.yaml
kubectl apply -f apps/webapp.yaml -f apps/webapp-httproute.yaml

The zero-downtime DNS swap

This is the nice part. With Cloudflare in front of both the old and new infrastructure, migration is just a DNS change. The old Docker Swarm keeps serving traffic until the A records point somewhere else.

The plan:

1. Deploy everything on the new k8s cluster (done above)
2. Test directly against the new cluster, bypassing DNS:

   curl -kv --resolve yourdomain.com:443:<node-ip> https://yourdomain.com

3. In Cloudflare DNS, change A records from old server IPs to new node IPs
4. Verify through Cloudflare: curl https://yourdomain.com
5. Once confirmed working, decommission the old servers

Since Cloudflare proxies all traffic (orange cloud), visitors never see the origin IPs. They connect to Cloudflare's edge, Cloudflare connects to whichever origin IPs the A records point to. Swap the records, traffic flows to the new cluster. No downtime because DNS changes behind Cloudflare are instant from the visitor's perspective — there's no TTL to wait for.

Load balancing with multiple A records

For basic redundancy, add two A records for the same hostname pointing to two different node IPs. Cloudflare round-robins between them. This is free and requires no special configuration — just two A records with orange cloud enabled.

For health-check-based failover (Cloudflare stops routing to a dead node), you'd need the paid Load Balancer ($5/month). Not necessary to start with.

Network policy

The existing fromEntities: ingress policy in the apps namespace already covers these new deployments. Any pod in the apps namespace automatically gets ingress from the Gateway and can talk to other pods in the same namespace and DNS. No policy changes needed.

Files