Gateway API and Ingress Setup

Setting up Cilium Gateway API to route external traffic to services based on hostname.

Goal

blog.yourdomain.com ─┐                                    ┌─→ blog service
app.yourdomain.com   ├─→ Cloudflare LB → 2 nodes → Gateway API ─┼─→ app service
yourdomain.com ──────┘                                    └─→ main service

Architecture

1. Cloudflare - DNS + Load balancer (free tier, 2 node IPs)

   • All domains point to same LB
   • SSL termination at Cloudflare (Full Strict mode later)
   • Routes to 2 of 3 node IPs

2. Cilium Gateway API - Ingress controller (hostNetwork mode)

   • Listens directly on node IPs port 80 (443 later for TLS)
   • Routes by Host header to correct service
   • Uses Envoy under the hood

3. HTTPRoutes - Per-service routing rules

   • Match hostname → route to Service

Step 1: Install Gateway API CRDs

Gateway API is a Kubernetes standard (not Cilium-specific). CRDs must be installed first.

Version: v1.4.1 (latest stable, Dec 2025)

Downloaded from:

curl -LO https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.4.1/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml

Important: Cilium 1.19 requires grpcroutes CRD even if you don't use gRPC. Without it: "Required GatewayAPI resources are not found".

Apply:

kubectl apply -f k8s/gateway-api-crds/gateway.networking.k8s.io_gatewayclasses.yaml
kubectl apply -f k8s/gateway-api-crds/gateway.networking.k8s.io_gateways.yaml
kubectl apply -f k8s/gateway-api-crds/gateway.networking.k8s.io_httproutes.yaml
kubectl apply -f k8s/gateway-api-crds/gateway.networking.k8s.io_referencegrants.yaml
kubectl apply -f k8s/gateway-api-crds/gateway.networking.k8s.io_grpcroutes.yaml

Step 2: Enable Gateway API in Cilium (hostNetwork mode)

For bare metal with public IPs, use hostNetwork mode. This makes the Gateway listen directly on node IPs instead of through a LoadBalancer (which requires cloud provider or MetalLB).

Add to k8s/cilium-values.yaml:

gatewayAPI:
  enabled: true
  hostNetwork:
    enabled: true

# Required for binding to port 80 (privileged port)
envoy:
  securityContext:
    capabilities:
      keepCapNetBindService: true  # Pass capability to forked Envoy process
      envoy:
      - NET_ADMIN
      - SYS_ADMIN
      - NET_BIND_SERVICE

Important: keepCapNetBindService: true is required! Without it, Envoy can't bind to ports < 1024:

cannot bind '0.0.0.0:80': Permission denied

Upgrade Cilium:

helm upgrade cilium cilium/cilium --namespace kube-system --values k8s/cilium-values.yaml
kubectl rollout restart daemonset cilium-envoy -n kube-system

Step 3: Create Gateway

File: k8s/gateway.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: gateway
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: main-gateway
  namespace: gateway
spec:
  gatewayClassName: cilium
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: All

Apply:

kubectl apply -f k8s/gateway.yaml

Known issue: With hostNetwork mode, Gateway shows PROGRAMMED: False and AddressNotAssigned. This is a cosmetic bug (GitHub #42786). The Gateway actually works - test with:

curl -v http://<node-ip>:80
# Should return: HTTP/1.1 404 Not Found, server: envoy

Step 4: Create HTTPRoutes

Each service gets an HTTPRoute. Example:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: demo-route
  namespace: app
spec:
  parentRefs:
  - name: main-gateway
    namespace: gateway
  hostnames:
    - "demo.yourdomain.com"  # Optional: match specific hostname
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: demo
      port: 80

Troubleshooting

"Waiting for controller"

Missing CRDs. Check operator logs:

kubectl logs -n kube-system deployment/cilium-operator | grep -i gateway

"Permission denied" binding port 80

Need keepCapNetBindService: true and NET_BIND_SERVICE capability in envoy securityContext.

Gateway shows PROGRAMMED: False

Known bug with hostNetwork mode. Test directly:

curl http://<node-ip>:80

Check Envoy logs

kubectl logs -n kube-system ds/cilium-envoy --tail=50 | grep -i -E "(bind|permission|listen|error)"

Files

• k8s/gateway-api-crds/ - CRD definitions (v1.4.1)
• k8s/gateway.yaml - Gateway resource
• k8s/demo-app.yaml - Demo app deployment
• k8s/demo-httproute.yaml - Demo HTTPRoute

Network Policy for Apps

Key insight: Cilium assigns a special ingress identity to Gateway API traffic. Use fromEntities: ingress to allow it:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: apps-namespace-policy
  namespace: apps
spec:
  endpointSelector: {}
  ingress:
    - fromEndpoints:
        - matchLabels:
            io.kubernetes.pod.namespace: apps
    - fromEntities:
        - ingress  # Gateway API traffic
    - fromEndpoints:
        - matchLabels:
            io.kubernetes.pod.namespace: kube-system
  egress:
    - toEndpoints:
        - matchLabels:
            io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
    - toEndpoints:
        - matchLabels:
            io.kubernetes.pod.namespace: apps
    - toEntities:
        - kube-apiserver

Do NOT use host or remote-node entities for Gateway API with hostNetwork - they don't work as expected.

Step 5: Add TLS Listener (Cloudflare Origin Cert)

See dedicated article: Adding TLS to Gateway API with Cloudflare Origin Certificates

Port 80 listener was removed — Gateway is HTTPS-only (port 443). Cloudflare handles HTTP→HTTPS redirect on the edge.

Next Steps

• Test with demo app
• Network policy with fromEntities: ingress
• Add TLS listener (port 443) with Cloudflare origin certs
• Configure Cloudflare DNS + LB
• Test hostname-based routing