Installation with Kubernetes manifests

Overview

Learn how to install, upgrade, and uninstall NGINX Gateway Fabric using Kubernetes manifests.

Before you begin

To complete this guide, you’ll need to install:

kubectl, a command-line interface for managing Kubernetes clusters.

Deploy NGINX Gateway Fabric

Deploying NGINX Gateway Fabric with Kubernetes manifests takes only a few steps. With manifests, you can configure your deployment exactly how you want. Manifests also make it easy to replicate deployments across environments or clusters, ensuring consistency.

If you’d like to use NGINX Plus:
1. To pull from the F5 Container registry, configure a docker registry secret using your JWT token from the MyF5 portal by following the instructions from here. Make sure to specify the secret in the imagePullSecrets field of the nginx-gateway ServiceAccount.
2. Alternatively, pull an NGINX Gateway Fabric image with NGINX Plus and push it to your private registry by following the instructions from here.
3. Update the nginx container’s image field of the nginx-gateway Deployment accordingly.

1. Install the Gateway API resources

Note:
The Gateway API resources from the standard channel must be installed before deploying NGINX Gateway Fabric. If they are already installed in your cluster, please ensure they are the correct version as supported by the NGINX Gateway Fabric - see the Technical Specifications.

To install the Gateway API resources, run the following:

kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v1.4.0" | kubectl apply -f -

Note:
If you plan to use the edge version of NGINX Gateway Fabric, you can replace the version in ref with main, for example ref=main.

Alternatively, you can install the Gateway API resources from the experimental channel. Installing Gateway API resources from the experimental channel includes everything in the standard release channel plus additional experimental resources and fields. NGINX Gateway Fabric currently supports a subset of the additional features provided by the experimental channel. To install from the experimental channel, run the following:

kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v1.4.0" | kubectl apply -f -

Note:
To learn more about what Gateway API resources NGINX Gateway Fabric currently supports, visit our Gateway API Compatibility document.

2. Deploy the NGINX Gateway Fabric CRDs

Stable release

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/crds.yaml

Edge version

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/main/deploy/crds.yaml

3. Deploy NGINX Gateway Fabric

Note:
By default, NGINX Gateway Fabric is installed in the nginx-gateway namespace. You can deploy in another namespace by modifying the manifest files.

Deploys NGINX Gateway Fabric with NGINX OSS.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/default/deploy.yaml

Deploys NGINX Gateway Fabric with NGINX OSS and an AWS Network Load Balancer service.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/aws-nlb/deploy.yaml

Deploys NGINX Gateway Fabric with NGINX OSS and nodeSelector to deploy on Linux nodes.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/azure/deploy.yaml

Deploys NGINX Gateway Fabric with NGINX Plus. The image is pulled from the NGINX Plus Docker registry, and the imagePullSecretName is the name of the secret to use to pull the image. The secret must be created in the same namespace as the NGINX Gateway Fabric deployment.

Important:
Ensure that you Enable Usage Reporting and update the necessary fields before applying.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/nginx-plus/deploy.yaml

Deploys NGINX Gateway Fabric with NGINX OSS and experimental features.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/experimental/deploy.yaml

Note:
Requires the Gateway APIs installed from the experimental channel.

Deploys NGINX Gateway Fabric with NGINX Plus and experimental features. The image is pulled from the NGINX Plus Docker registry, and the imagePullSecretName is the name of the secret to use to pull the image. The secret must be created in the same namespace as the NGINX Gateway Fabric deployment.

Important:
Ensure that you Enable Usage Reporting and update the necessary fields before applying.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/nginx-plus-experimental/deploy.yaml

Note:
Requires the Gateway APIs installed from the experimental channel.

Deploys NGINX Gateway Fabric with NGINX OSS using a Service type of NodePort.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/nodeport/deploy.yaml

Deploys NGINX Gateway Fabric with NGINX OSS on OpenShift.

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/openshift/deploy.yaml

4. Verify the Deployment

To confirm that NGINX Gateway Fabric is running, check the pods in the nginx-gateway namespace:

kubectl get pods -n nginx-gateway

The output should look similar to this (note that the pod name will include a unique string):

NAME                             READY   STATUS    RESTARTS   AGE
nginx-gateway-5d4f4c7db7-xk2kq   2/2     Running   0          112s

5. Access NGINX Gateway Fabric

There are two options for accessing NGINX Gateway Fabric depending on the type of LoadBalancer service you chose during installation:

If the LoadBalancer type is NodePort, Kubernetes will randomly allocate two ports on every node of the cluster. To access the NGINX Gateway Fabric, use an IP address of any node of the cluster along with the two allocated ports.

Tip:
Read more about the type NodePort in the Kubernetes documentation.
If the LoadBalancer type is LoadBalancer:
- For GCP or Azure, Kubernetes will allocate a cloud load balancer for load balancing the NGINX Gateway Fabric pods. Use the public IP of the load balancer to access NGINX Gateway Fabric.
- For AWS, Kubernetes will allocate a Network Load Balancer (NLB) in TCP mode with the PROXY protocol enabled to pass the client’s information (the IP address and the port).
Use the public IP of the load balancer to access NGINX Gateway Fabric. To get the public IP which is reported in the EXTERNAL-IP column:
- For GCP or Azure, run:
```
kubectl get svc nginx-gateway -n nginx-gateway
```
- In AWS, the NLB (Network Load Balancer) DNS (directory name system) name will be reported by Kubernetes instead of a public IP. To get the DNS name, run:
```
kubectl get svc nginx-gateway -n nginx-gateway
```
  Note:
  
  We recommend using the NLB DNS whenever possible, but for testing purposes, you can resolve the DNS name to get the IP address of the load balancer:
  nslookup <dns-name>
Tip:

Learn more about type LoadBalancer in the Kubernetes documentation.

For AWS, additional options regarding an allocated load balancer are available, such as its type and SSL termination. Read the Kubernetes documentation to learn more.

Important:
By default Helm and manifests configure NGINX Gateway Fabric on ports 80 and 443, affecting any gateway listeners on these ports. To use different ports, update the configuration. NGINX Gateway Fabric requires a configured gateway resource with a valid listener to listen on any ports.

NGINX Gateway Fabric uses the created service to update the Addresses field in the Gateway Status resource. Using a LoadBalancer service sets this field to the IP address and/or hostname of that service. Without a service, the pod IP address is used.

This gateway is associated with the NGINX Gateway Fabric through the gatewayClassName field. The default installation of NGINX Gateway Fabric creates a GatewayClass with the name nginx. NGINX Gateway Fabric will only configure gateways with a gatewayClassName of nginx unless you change the name via the --gatewayclass command-line flag.

Upgrade NGINX Gateway Fabric

Tip:
For guidance on zero downtime upgrades, see the Delay Pod Termination section below.

To upgrade NGINX Gateway Fabric and get the latest features and improvements, take the following steps:

Upgrade Gateway API resources:
- Verify that your NGINX Gateway Fabric version is compatible with the Gateway API resources. Refer to the Technical Specifications for details.
- Review the release notes for any important upgrade-specific information.
- To upgrade the Gateway API resources, run:
```
kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v1.4.0" | kubectl apply -f -
```
  or, if you installed the from the experimental channel:
```
kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v1.4.0" | kubectl apply -f -
```

Upgrade NGINX Gateway Fabric CRDs:

To upgrade the Custom Resource Definitions (CRDs), run:

kubectl apply -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/crds.yaml

Upgrade NGINX Gateway Fabric deployment:

Select the deployment manifest that matches your current deployment from the table above in the Deploy NGINX Gateway Fabric section and apply it.

Delay pod termination for zero downtime upgrades

To avoid client service interruptions when upgrading NGINX Gateway Fabric, you can configure PreStop hooks to delay terminating the NGINX Gateway Fabric pod, allowing the pod to complete certain actions before shutting down. This ensures a smooth upgrade without any downtime, also known as a zero downtime upgrade.

For an in-depth explanation of how Kubernetes handles pod termination, see the Termination of Pods topic on their official website.

Note:
Keep in mind that NGINX won’t shut down while WebSocket or other long-lived connections are open. NGINX will only stop when these connections are closed by the client or the backend. If these connections stay open during an upgrade, Kubernetes might need to shut down NGINX forcefully. This sudden shutdown could interrupt service for clients.

Follow these steps to configure delayed pod termination:

Open the deploy.yaml for editing.

Add delayed shutdown hooks:

In the deploy.yaml file, add lifecycle: preStop hooks to both the nginx and nginx-gateway container definitions. These hooks instruct the containers to delay their shutdown process, allowing time for connections to close gracefully. Update the sleep value to what works for your environment.

<...>
name: nginx-gateway
<...>
lifecycle:
  preStop:
    exec:
      command:
      - /usr/bin/gateway
      - sleep
      - --duration=40s # This flag is optional, the default is 30s
<...>
name: nginx
<...>
lifecycle:
  preStop:
    exec:
      command:
      - /bin/sleep
      - "40"
<...>

Set the termination grace period:
- Set terminationGracePeriodSeconds to a value that is equal to or greater than the sleep duration specified in the preStop hook (default is 30). This setting prevents Kubernetes from terminating the pod before before the preStop hook has completed running.
```
terminationGracePeriodSeconds: 50
```
Save the changes.

See Also:

For additional information on configuring and understanding the behavior of containers and pods during their lifecycle, refer to the following Kubernetes documentation:

Container Lifecycle Hooks

Pod Lifecycle

Uninstall NGINX Gateway Fabric

Follow these steps to uninstall NGINX Gateway Fabric and Gateway API from your Kubernetes cluster:

Uninstall NGINX Gateway Fabric:

To remove NGINX Gateway Fabric and its custom resource definitions (CRDs), run:

kubectl delete namespace nginx-gateway
kubectl delete cluterrole nginx-gateway
kubectl delete clusterrolebinding nginx-gateway

kubectl delete -f https://raw.githubusercontent.com/nginxinc/nginx-gateway-fabric/v1.4.0/deploy/crds.yaml

Remove the Gateway API resources:
- Warning:
  This will remove all corresponding custom resources in your entire cluster, across all namespaces. Double-check to make sure you don’t have any custom resources you need to keep, and confirm that there are no other Gateway API implementations active in your cluster.
  
  To uninstall the Gateway API resources, run the following:
```
kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v1.4.0" | kubectl delete -f -
```
  Alternatively, if you installed the Gateway APIs from the experimental channel, run the following:
```
kubectl kustomize "https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v1.4.0" | kubectl delete -f -
```

Last modified August 20, 2024