Distribute traffic across clusters with F5 BIG-IP

This guide describes how to use an F5 BIG-IP system as the external load balancer for NGINX Gateway Fabric Gateways in two clusters, with TLS termination and traffic distribution between them.

Overview

In this guide, you configure an ExternalLoadBalancer resource that puts BIG-IP in front of Gateways in two clusters. BIG-IP terminates client TLS and re-encrypts toward NGINX, runs your health monitors and iRules, and spreads traffic across both clusters.

The intended use case is a single hostname and certificate served by backends in more than one cluster, such as an active-active deployment or a migration between clusters. Clients see one address, and traffic moves between clusters without a DNS change.

See How configuration reaches BIG-IP.

Before you begin

You need:

• Two Kubernetes clusters, referred to in this guide as cluster A and cluster B.
• An F5 BIG-IP system running version 17.1.0.3 or later, and an account on it with administrator privileges.
• Network access from cluster A to the BIG-IP system, and from BIG-IP to the nodes of both clusters.
• Python 3.14 or later.

Both clusters run NGINX Gateway Fabric and serve traffic. Cluster A also runs F5 Container Ingress Services, which owns the BIG-IP configuration and reaches cluster B over a kubeconfig, so every step that touches BIG-IP is run against cluster A.

This guide installs the AS3 extension, F5 Container Ingress Services, NGINX Gateway Fabric, and cert-manager. cert-manager issues the certificate the Gateway presents on its HTTPS listener, and is installed in both clusters.

The shell commands in this guide read the following environment variables, so set them once in the shell you work from and the commands can be copied as they appear:

export BIGIP_ADDRESS="192.0.2.10:443"
export BIGIP_USERNAME="admin"
export BIGIP_PASSWORD="<your-password>"
export VIRTUAL_SERVER_ADDRESS="192.0.2.100"

• BIGIP_ADDRESS is the BIG-IP management address, including the port. BIG-IP listens on 443 by default.
• BIGIP_USERNAME and BIGIP_PASSWORD are your BIG-IP credentials.
• VIRTUAL_SERVER_ADDRESS is a free IPv4 address on the BIG-IP subnet, which BIG-IP listens on.

NGINX_POD_NAME is set later, and is the name of an NGINX Pod in the cluster you are reading logs from.

Prepare BIG-IP

In this section you install the AS3 extension and create the BIG-IP objects this guide depends on: a partition for F5 Container Ingress Services to own, an iRule, and the SSL profiles and health monitors the ExternalLoadBalancer refers to by path.

AS3 extension

F5 Container Ingress Services configures BIG-IP by posting AS3 declarations, so AS3 must be installed before anything else. Follow Downloading and installing the BIG-IP AS3 package in the F5 documentation, then return here.

Partition

Create a partition named k8s for F5 Container Ingress Services to own:

curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" -X POST "https://$BIGIP_ADDRESS/mgmt/tm/auth/partition" \
  -H "Content-Type: application/json" -d '{"name":"k8s"}'

The response describes the new partition:

{
    "name": "k8s",
    "fullPath": "k8s",
    "defaultRouteDomain": 0
}

F5 Container Ingress Services manages the full contents of its partition. The partition cannot be Common, because Container Ingress Services must not modify shared configuration.

SSL Profiles

This guide uses the two SSL profiles that ship with BIG-IP:

• /Common/clientssl carries the certificate BIG-IP presents to clients, and terminates their TLS connections.
• /Common/serverssl re-encrypts traffic on the connection BIG-IP opens to NGINX. It does not validate the backend certificate, so the self-signed certificate the Gateway presents is accepted.

Both are suitable for testing. In production, replace them with profiles carrying your own certificates, and configure peer verification on the server SSL profile if the backend certificate must be validated.

HTTP iRules

Create an iRule named gatewaylink_irule, which inserts a response header:

when HTTP_RESPONSE {
  HTTP::header insert "X-GatewayLink" "true"
}

The iRule runs on every HTTP response BIG-IP sends back to a client and adds an X-GatewayLink: true header to it. Only a Layer 7 virtual server runs HTTP-event iRules, so the header appearing in a response confirms both that BIG-IP built a Layer 7 virtual server and that the iRule is attached to it. You check for the header in Verify the configuration.

To create the iRule run the following command:

curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" -X POST "https://$BIGIP_ADDRESS/mgmt/tm/ltm/rule" \
  -H "Content-Type: application/json" -d '{
    "name": "gatewaylink_irule",
    "apiAnonymous": "when HTTP_RESPONSE { HTTP::header insert \"X-GatewayLink\" \"true\" }"
  }'

Health Monitors

This guide uses two health monitors that ship with BIG-IP, so there is nothing to create:

• /Common/http checks the HTTP pool members by sending a request and waiting for a response.
• /Common/tcp checks the HTTPS pool members by opening a TCP connection, without inspecting encrypted traffic.

BIG-IP marks a pool member offline when its monitor fails and stops sending traffic to it, so each virtual server is checked in a way that suits the traffic it carries.

Connect cluster A to cluster B

F5 Container Ingress Services runs in cluster A and reaches cluster B over a kubeconfig. Build that kubeconfig, because Container Ingress Services needs it at install time.

On cluster B, grant F5 Container Ingress Services read access:

kubectl apply -f - <<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: bigip-ctlr-clusterrole
rules:
  - apiGroups: [""]
    resources: ["nodes", "services", "endpoints", "namespaces", "pods", "secrets", "configmaps"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["cis.f5.com"]
    resources: ["*"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["discovery.k8s.io"]
    resources: ["endpointslices"]
    verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bigip-ctlr
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: bigip-ctlr-clusterrole-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: bigip-ctlr-clusterrole
subjects:
  - kind: ServiceAccount
    name: bigip-ctlr
    namespace: kube-system
EOF

Generate a kubeconfig from the service account. The server address must be the cluster B API server as reached from cluster A, so take it from the node rather than from the local kubeconfig, which records https://127.0.0.1:6443:

TOKEN=$(kubectl create token bigip-ctlr -n kube-system --duration=8760h)
CA=$(kubectl get cm kube-root-ca.crt -n kube-system -o jsonpath='{.data.ca\.crt}' | base64 -w0)
APISERVER=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}' | awk '{print $1}')

cat > remote-kubeconfig.yaml <<EOF
apiVersion: v1
kind: Config
clusters:
- name: remote
  cluster:
    server: https://${APISERVER}:6443
    certificate-authority-data: ${CA}
users:
- name: bigip-ctlr
  user:
    token: ${TOKEN}
contexts:
- name: remote
  context:
    cluster: remote
    user: bigip-ctlr
current-context: remote
EOF

Verify the kubeconfig works:

KUBECONFIG=remote-kubeconfig.yaml kubectl get nodes

The command lists the nodes of cluster B:

NAME    STATUS   ROLES                  AGE   VERSION
vm2     Ready    control-plane,master   14d   v1.31.5+k3s1

Configure cluster A

Run these steps against cluster A. Only this cluster runs F5 Container Ingress Services and owns the BIG-IP configuration.

Generate secret to connect to cluster B

Copy the remote-kubeconfig.yaml file you generated on cluster B over to cluster A, then create a Secret from it. Create the Secret before installing F5 Container Ingress Services, because it reads the kubeconfig at startup.

kubectl create secret generic remote-kubeconfig -n kube-system \
  --from-file=kubeconfig=remote-kubeconfig.yaml

Important

The Secret name and namespace must match the secret value in the extended-spec-config ConfigMap created in the next section, and the key inside the Secret must be kubeconfig. Nothing validates these names. F5 Container Ingress Services starts normally, builds only local pools, and reports the mismatch in its log:

Copy

error occurred while fetching Secret: remote-kubeconfig for the cluster: remote, Error: secrets "remote-kubeconfig" not found

Install F5 Container Ingress Services

Install the F5 Container Ingress Services custom resource definitions:

kubectl apply -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/v2.20.4/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml

Confirm the IngressLink custom resource definition is installed:

kubectl get crd ingresslinks.cis.f5.com

NAME                        CREATED AT
ingresslinks.cis.f5.com     2026-08-05T01:40:54Z

Create a ConfigMap named extended-spec-config:

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: extended-spec-config
  namespace: kube-system
  labels:
    f5nr: "true"
data:
  extendedSpec: |
    mode: default
    externalClustersConfig:
    - clusterName: remote
      secret: kube-system/remote-kubeconfig
EOF

F5 Container Ingress Services reads its multi-cluster configuration from this ConfigMap. The f5nr: "true" label is required. The clusterName value is how you refer to cluster B in the ExternalLoadBalancer resource later.

Deploy F5 Container Ingress Services:

helm repo add f5-stable https://f5networks.github.io/charts/stable
helm repo update

helm install f5-cis f5-stable/f5-bigip-ctlr -n kube-system \
  --set bigip_secret.create=true \
  --set bigip_secret.username="$BIGIP_USERNAME" \
  --set bigip_secret.password="$BIGIP_PASSWORD" \
  --set rbac.create=true \
  --set serviceAccount.create=true \
  --set namespace=kube-system \
  --set args.bigip_url="$BIGIP_ADDRESS" \
  --set args.bigip_partition=k8s \
  --set args.pool_member_type=nodeport \
  --set args.custom_resource_mode=true \
  --set args.insecure=true \
  --set args.log_level=DEBUG \
  --set args.log-as3-response=true \
  --set args.multi-cluster-mode=standalone \
  --set args.local-cluster-name=local \
  --set args.extended-spec-configmap=kube-system/extended-spec-config

These fields must be set according to your own setup:

• args.bigip_url must include the port. F5 Container Ingress Services assumes 443, so omitting a non-default port causes connection failures.
• args.custom_resource_mode=true is required. Without it, F5 Container Ingress Services never watches IngressLink resources.
• args.multi-cluster-mode=standalone is required for a Layer 7 virtual server and TLS termination.
• args.local-cluster-name is required whenever multi-cluster mode is set, and must match multiCluster.localClusterName in the ExternalLoadBalancer.
• args.extended-spec-configmap points to the ConfigMap created earlier, in <NAMESPACE>/<NAME> form. F5 Container Ingress Services reads the list of external clusters and their kubeconfig Secrets from it, so without this value it has no way to reach cluster B.
• args.pool_member_type must match the type of the Gateway’s Service. Use nodeport with NodePort, or cluster with ClusterIP.
• args.log-as3-response=true logs the BIG-IP response to each declaration, which is useful for troubleshooting.

Confirm F5 Container Ingress Services reached BIG-IP and accepted the mode:

kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -E "authn/login|multi-cluster-mode"

The log shows a successful login and the configured multi-cluster mode:

[DEBUG] [BIGIP] postConfig request: POST https://192.0.2.10:443/mgmt/shared/authn/login 200 OK
[DEBUG] Multi-cluster-mode: standalone, local cluster name: local

Set up for both clusters

Apply the following resources to both clusters. Use the same Gateway name and the same listeners in each, so the data plane Services carry matching labels and expose the same ports. F5 Container Ingress Services builds one virtual server per Service port and pools every cluster behind that virtual server.

Install the custom resource definitions

Install the F5 Container Ingress Services custom resource definitions in both clusters, including cluster B, which does not run F5 Container Ingress Services:

kubectl apply -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/v2.20.4/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml

Confirm the IngressLink custom resource definition is installed:

kubectl get crd ingresslinks.cis.f5.com

NAME                        CREATED AT
ingresslinks.cis.f5.com     2026-08-05T01:40:54Z

With external load balancer support enabled, NGINX Gateway Fabric watches IngressLink resources on startup in every cluster it runs in. A cluster without the custom resource definition leaves the control plane unable to start, and its Pod restarts continuously:

no matches for kind "IngressLink" in version "cis.f5.com/v1"
failed to start control loop: failed to wait for provisioner-IngressLink caches to sync

Install NGINX Gateway Fabric

Install NGINX Gateway Fabric with external load balancer support enabled.

Using Helm, set the nginxGateway.externalLoadBalancer.enable=true value. Using Kubernetes manifests, add the --external-load-balancer flag to the nginx-gateway container arguments.

Create a Gateway

Create an NginxProxy resource named gatewaylink-proxy, which exposes the readiness probe:

kubectl apply -f - <<EOF
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
  name: gatewaylink-proxy
spec:
  kubernetes:
    deployment:
      container:
        readinessProbe:
          expose: true
          port: 8081
          path: /nginx-ready
    service:
      type: NodePort
      externalTrafficPolicy: Local
EOF

This NginxProxy configures the Gateway that references it with the settings BIG-IP depends on:

• readinessProbe.expose puts the readiness port on the data plane Service. F5 Container Ingress Services builds its health monitor against the Service port named health.
• readinessProbe.path sets the readiness path to /nginx-ready, which is the path the generated health monitor requests. NGINX Gateway Fabric serves its readiness endpoint at /readyz by default.
• service.type sets the type of the Gateway’s Service, and must match the F5 Container Ingress Services pool_member_type.
• service.externalTrafficPolicy: Local preserves the client source address, and means only nodes running an NGINX Pod advertise the endpoint, so BIG-IP health checks reach a node that answers.

Issue the listener certificate

The Gateway presents a certificate on its HTTPS listener, read from a Secret named nginx-tls. This guide uses cert-manager and a local certificate authority to issue it, so install both in each cluster.

Install cert-manager onto the cluster using Helm with Gateway API features enabled.

• Add the Helm repository.

  Copy

  helm repo add jetstack https://charts.jetstack.io
  helm repo update

• Install cert-manager, and enable the GatewayAPI feature gate:

  Copy

  helm install \
    cert-manager jetstack/cert-manager \
    --namespace cert-manager \
    --create-namespace \
    --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \
    --set config.kind="ControllerConfiguration" \
    --set config.enableGatewayAPI=true \
    --set crds.enabled=true

Create a self-signed ClusterIssuer, a CA Certificate, and a CA-backed ClusterIssuer. cert-manager uses the resulting local-ca-issuer to sign certificates in any namespace:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-cluster-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: local-ca
  namespace: cert-manager
spec:
  isCA: true
  commonName: LocalCA
  secretName: local-ca-secret
  issuerRef:
    name: selfsigned-cluster-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: local-ca-issuer
spec:
  ca:
    secretName: local-ca-secret
EOF

Create the nginx-tls Secret by requesting a certificate from the local certificate authority:

kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: nginx-tls
  namespace: default
spec:
  secretName: nginx-tls
  issuerRef:
    name: local-ca-issuer
    kind: ClusterIssuer
  commonName: cafe.example.com
  dnsNames:
  - cafe.example.com
EOF

Confirm the Secret exists before continuing:

kubectl get secret nginx-tls -n default

Create a Gateway named gateway with an HTTP listener and an HTTPS listener:

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gateway
spec:
  gatewayClassName: nginx
  infrastructure:
    parametersRef:
      name: gatewaylink-proxy
      group: gateway.nginx.org
      kind: NginxProxy
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      hostname: cafe.example.com
    - name: https
      port: 443
      protocol: HTTPS
      hostname: cafe.example.com
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: nginx-tls
EOF

The Gateway has two listeners, so the data plane Service exposes ports 80 and 443. Keep the HTTP listener: it gives BIG-IP a plain HTTP path for health checks, and lets you verify traffic without TLS during setup.

Confirm the Gateway is Accepted and Programmed:

kubectl describe gateways.gateway.networking.k8s.io gateway

Verify the status is Accepted and Programmed, and that both listeners appear:

Status:
  Conditions:
    Message:               The Gateway is accepted
    Reason:                Accepted
    Status:                True
    Type:                  Accepted

Create coffee application and routes

Create the coffee application by copying and pasting the following block into your terminal:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coffee
spec:
  replicas: 2
  selector:
    matchLabels:
      app: coffee
  template:
    metadata:
      labels:
        app: coffee
    spec:
      containers:
      - name: coffee
        image: nginxdemos/nginx-hello:plain-text
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: coffee
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: coffee
EOF

Create an HTTPRoute named coffee that attaches to the Gateway and routes /coffee to that Service:

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: coffee
spec:
  parentRefs:
  - name: gateway
  hostnames:
  - "cafe.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    backendRefs:
    - name: coffee
      port: 80
EOF

Confirm the application is running and the route is attached:

kubectl get pods,svc,httproute

The output lists two coffee Pods, the coffee Service, and the HTTPRoute:

NAME                          READY   STATUS    RESTARTS   AGE
pod/coffee-7dd75bc79b-cqvb7   1/1     Running   0          77s
pod/coffee-7dd75bc79b-t4xkn   1/1     Running   0          77s

NAME             TYPE        CLUSTER-IP     PORT(S)   AGE
service/coffee   ClusterIP   10.43.174.12   80/TCP    77s

NAME                                         HOSTNAMES               AGE
httproute.gateway.networking.k8s.io/coffee   ["cafe.example.com"]    32s

Create the ExternalLoadBalancer

On cluster A only, create an ExternalLoadBalancer resource named gateway-elb:

kubectl apply -f - <<EOF
apiVersion: gateway.nginx.org/v1alpha1
kind: ExternalLoadBalancer
metadata:
  name: gateway-elb
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: gateway
  gatewayLink:
    virtualServerAddress: "$VIRTUAL_SERVER_ADDRESS"
    partition: k8s
    host: cafe.example.com
    tls:
      reference: bigip
      clientSSLs:
        - /Common/clientssl
      serverSSLs:
        - /Common/serverssl
    monitors:
      - name: /Common/http
        reference: bigip
      - name: /Common/tcp
        reference: bigip
    iRules:
      - /Common/gatewaylink_irule
    multiCluster:
      localClusterName: local
      remoteClusters:
        - clusterName: remote
EOF

This example configures BIG-IP TLS termination, hostname matching, health monitors, iRules, and multi-cluster traffic distribution.

Set localClusterName to the value passed to F5 Container Ingress Services as args.local-cluster-name. Set each clusterName under remoteClusters to a clusterName from the extended spec ConfigMap.

Important

The multiCluster field is required when F5 Container Ingress Services runs in multi-cluster mode.

Confirm NGINX Gateway Fabric Accepted the resource:

kubectl describe externalloadbalancers.gateway.nginx.org gateway-elb

Verify the status is Accepted:

Status:
  Controllers:
    Conditions:
      Message:               The ExternalLoadBalancer is accepted
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.nginx.org/nginx-gateway-controller

Verify the configuration

Confirm the IngressLink was written and accepted:

kubectl describe ingresslink gateway-nginx

F5 Container Ingress Services writes this status after posting the AS3 declaration. A status of OK means BIG-IP accepted the declaration, and Vs Address is the address the virtual server listens on:

Status:
  Last Updated:  2026-08-05T02:23:16Z
  Status:        OK
  Vs Address:    192.0.2.100

Send a request through BIG-IP:

curl -kv --resolve cafe.example.com:443:$VIRTUAL_SERVER_ADDRESS https://cafe.example.com/coffee

The request returns 200 OK, with the X-GatewayLink header added by the iRule and a response body from the backend application:

< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: text/plain
< X-GatewayLink: true

Server address: 10.42.0.19:8080
Server name: coffee-7b9578cff9-272gf
URI: /coffee

Confirm traffic is distributed across both clusters. The example application returns the name of the Pod that served each request:

for i in $(seq 1 20); do
  curl -sk --resolve cafe.example.com:443:$VIRTUAL_SERVER_ADDRESS https://cafe.example.com/coffee | grep "Server name"
done | sort | uniq -c

The output counts each Pod that answered:

   7 Server name: coffee-7b9578cff9-272gf
   6 Server name: coffee-7b9578cff9-qqxzg
   6 Server name: coffee-7b9578cff9-4mxb9
   1 Server name: coffee-7b9578cff9-tpb2c

Match those names against the Pods in each cluster to see which cluster served which request.

Confirm BIG-IP is terminating client TLS. BIG-IP decrypts the client connection and opens a separate connection to NGINX, so NGINX logs the BIG-IP request rather than the client one:

export NGINX_POD_NAME=$(kubectl get pods -l app.kubernetes.io/name=gateway-nginx -o jsonpath='{.items[0].metadata.name}')
kubectl logs $NGINX_POD_NAME -c nginx | grep coffee | tail -1

The log records the BIG-IP self-IP address as the client, and the request arriving over HTTP/1.1:

192.0.2.10 - - [05/Aug/2026:02:41:18 +0000] "GET /coffee HTTP/1.1" 200 158 "-" "curl/8.5.0"

Pass TLS through to NGINX

Omitting the tls field from the ExternalLoadBalancer moves TLS termination to NGINX. BIG-IP builds a TCP virtual server and forwards the encrypted stream without decrypting it, so clients see the certificate from the Gateway certificateRefs Secret and no SSL profiles are needed on BIG-IP.

Use this when the certificate and private key must stay inside the cluster. Note that BIG-IP cannot read a stream it does not decrypt, so hostname matching and HTTP-event iRules are not available in this configuration.

Troubleshooting

No IngressLink is created

Confirm the --external-load-balancer flag is set on the control plane deployment. Helm ignores values a chart does not define, so a chart without external load balancer support renders a deployment without the flag:

kubectl get deploy -n nginx-gateway ngf-nginx-gateway-fabric \
  -o jsonpath='{.spec.template.spec.containers[?(@.name=="nginx-gateway")].args}'

The IngressLink has no status

F5 Container Ingress Services writes this status, so an empty status means it has not processed the resource. Wait up to two minutes for reconciliation, then check its logs:

kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr

No address is allocated

Confirm F5 Container Ingress Services was deployed with args.ipam=true, then check the F5 IPAM Controller logs:

kubectl logs -n kube-system -l app=f5-ipam-controller --tail=20

A label that does not match a configured pool is reported directly:

[PROV] IPAM LABEL: gatewaylink Not Found

Set ipamLabel on the ExternalLoadBalancer to a pool name from the args.ip_range map used when installing the F5 IPAM Controller.

The AS3 declaration is rejected

Read the BIG-IP response in the F5 Container Ingress Services logs, which usually names the problem:

kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -E "AS3\]\[POST\]|response:"

F5 Container Ingress Services reports that AS3 is not installed

The Pod is in CrashLoopBackOff and its logs contain [ERROR] AS3 RPM is not installed on BIGIP. F5 Container Ingress Services infers this from a 404 on the AS3 endpoint, so it also appears when AS3 is installed but not serving. See Troubleshooting in the F5 documentation.

After restoring AS3, delete the Pod so it retries without waiting out its backoff:

kubectl delete pod -n kube-system -l app=f5-cis-f5-bigip-ctlr

A pool is empty

Confirm the type of the Gateway’s Service matches the F5 Container Ingress Services pool_member_type, and that the Gateway has a listener on the port the pool was built for. A missing or invalid certificateRefs Secret leaves an HTTPS listener unprogrammed, so the Service never exposes port 443:

kubectl get svc gateway-nginx -o jsonpath='{.spec.type}{"\n"}{.spec.ports}'
kubectl describe gateways.gateway.networking.k8s.io gateway

NGINX logs show an internal address as the client

The client address travels inside the PROXY protocol header. NGINX reads it only when both the connection address and the address inside the header are trusted, so an internal address in the log means the header never arrived or was discarded.

Confirm the iRule is attached. Creating an iRule on BIG-IP does not attach it to anything:

curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" "https://$BIGIP_ADDRESS/mgmt/tm/ltm/virtual" \
  | python3 -c 'import sys,json
for v in json.load(sys.stdin)["items"]:
    print(v["fullPath"], "->", v.get("rules", "no rules"))'

If the iRule is attached, confirm the trusted addresses:

kubectl exec $NGINX_POD_NAME -c nginx -- grep set_real_ip_from /etc/nginx/conf.d/http.conf

Set trustedAddresses on the NginxProxy resource to the subnet of the IP address which the BIG-IP system uses to send traffic to NGINX.

A configured field has no effect

Kubernetes discards fields that are not in the installed custom resource definition schema without reporting an error, so both controllers report success while the field never arrives. Check where the field stops:

export FIELD_NAME="ipamLabel"

kubectl get crd ingresslinks.cis.f5.com -o yaml | grep -A5 "$FIELD_NAME"
kubectl logs -n nginx-gateway deploy/ngf-nginx-gateway-fabric | grep "unknown field"
kubectl get ingresslink gateway-nginx -o jsonpath='{.spec}' | python3 -m json.tool

An unknown field message means the installed custom resource definition is older than the NGINX Gateway Fabric release. Install a matching version.

The control plane restarts continuously in cluster B

The NGINX Gateway Fabric Pod reports CrashLoopBackOff, and its logs end with a cache sync failure:

no matches for kind "IngressLink" in version "cis.f5.com/v1"
failed to start control loop: failed to wait for provisioner-IngressLink caches to sync

The F5 Container Ingress Services custom resource definitions are missing from that cluster. With external load balancer support enabled, NGINX Gateway Fabric watches IngressLink resources on startup, whether or not Container Ingress Services runs there.

• Install the custom resource definitions in the affected cluster:

kubectl apply -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/v2.20.4/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml

• Delete the Pod so it restarts immediately rather than waiting out its backoff:

kubectl delete pod -n nginx-gateway -l app.kubernetes.io/name=nginx-gateway-fabric

The remote cluster has no pool

Only _local pools exist on BIG-IP, and traffic never reaches cluster B. F5 Container Ingress Services could not load the cluster B kubeconfig, so it has no endpoints to pool. Start with its log, which names the cause directly:

kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -i "MultiCluster"

• Confirm the Secret exists under the name and namespace the extended-spec-config ConfigMap refers to. A Secret created under a different name is reported as missing:

error occurred while fetching Secret: remote-kubeconfig for the cluster: remote, Error: secrets "remote-kubeconfig" not found

• Confirm the token is still valid. A token issued for a ServiceAccount that has since been deleted and recreated is rejected:

the server has asked for the client to provide credentials

Regenerate the kubeconfig on cluster B and recreate the Secret.

• Confirm the Secret holding the cluster B kubeconfig parses. A kubeconfig with broken indentation is stored without complaint and fails only when F5 Container Ingress Services loads it:

kubectl get secret remote-kubeconfig -n kube-system -o jsonpath='{.data.kubeconfig}' | base64 -d > /tmp/check.yaml
KUBECONFIG=/tmp/check.yaml kubectl get nodes

The command lists the cluster B nodes. An error such as mapping values are not allowed in this context means the file is malformed, so regenerate it and recreate the Secret.

• Confirm the clusterName in the extended spec ConfigMap matches the clusterName under remoteClusters in the ExternalLoadBalancer.
• Restart F5 Container Ingress Services after replacing the Secret, because it reads the kubeconfig at startup:

kubectl rollout restart deploy/f5-cis-f5-bigip-ctlr -n kube-system

The remote pool member is down

The _remote pool exists but its member reports offline, so all traffic goes to the local cluster.

• Confirm the cluster B data plane Service exposes the same ports as cluster A. A missing HTTPS listener leaves nothing listening on the 443 NodePort:

kubectl get svc gateway-nginx -o jsonpath='{range .spec.ports[*]}{.name} {.port}:{.nodePort}{"\n"}{end}'

• Confirm the nginx-tls Secret exists in cluster B. Without it the HTTPS listener is not programmed and NGINX never listens on 443.
• Restart the data plane after creating a certificate. NGINX does not load a certificate created after the Pod started, and the Gateway reports every condition as healthy while the listener is missing from the configuration:

kubectl rollout restart deploy/gateway-nginx -n default

Requests fail with a connection reset

A request through BIG-IP fails with Recv failure: Connection reset by peer. NGINX Gateway Fabric enables HTTP/2 by default. BIG-IP SSL profiles do not negotiate HTTP/2 unless configured to, so BIG-IP sends HTTP/1.1 into a connection NGINX set up for HTTP/2.

• Set disableHTTP2: true on the NginxProxy resource, or use a BIG-IP SSL profile with HTTP/2 enabled. Confirm the setting reached the data plane rather than trusting the resource:

kubectl exec $NGINX_POD_NAME -c nginx -- grep "listen 443" /etc/nginx/conf.d/http.conf

The absence of an http2 token on the listen line means HTTP/2 is off.

Remove the configuration

Delete the ExternalLoadBalancer so F5 Container Ingress Services deletes the objects it created on BIG-IP:

kubectl delete externalloadbalancer gateway-elb

Confirm the virtual servers are gone:

curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" "https://$BIGIP_ADDRESS/mgmt/tm/ltm/virtual" | python3 -m json.tool | grep fullPath

References

• F5 IngressLink documentation: the F5 Container Ingress Services resource that NGINX Gateway Fabric generates.
• F5 Application Services 3 Extension reference: the declaration format F5 Container Ingress Services posts to BIG-IP.
• F5 Container Ingress Services: the F5 Container Ingress Services source and custom resource definitions.
• F5 IPAM Controller: allocates virtual server addresses when using the ipamLabel field instead of a fixed address.
• F5 BIG-IP iControl REST API: the API used by the curl commands in this guide.
• BIG-IP Virtual Edition on Amazon Web Services
• BIG-IP Virtual Edition on Microsoft Azure
• BIG-IP Virtual Edition on Google Cloud Platform
• F5 Container Ingress Services multi-cluster guide: multi-cluster deployment topologies.