Gateway architecture

Learn about the architecture and design principles of NGINX Gateway Fabric: a Kubernetes Gateway API implementation which uses NGINX as the data plane.

This document is intended for:

  • Cluster Operators who want to understand how NGINX Gateway Fabric works in production, how it manages traffic, and how to troubleshoot failures.

  • Application Developers who would like to use NGINX Gateway Fabric to expose and route traffic to their applications within Kubernetes.

The reader needs to be familiar with core Kubernetes concepts, such as pods, deployments, services, and endpoints. For an understanding of how NGINX itself works, you can read the “Inside NGINX: How We Designed for Performance & Scale” blog post.

If you are interested in contributing to the project or learning about its internal implementation details, please see the Developer Architecture Guide.

NGINX Gateway Fabric Deployment Model and Architectural Overview

NGINX Gateway Fabric splits its architecture into two main parts to provide better security, flexibility, and reliability:

Control Plane: Centralized Management

The control plane operates as a Deployment, serving as a Kubernetes controller built with the controller-runtime library. It manages all aspects of resource provisioning and configuration for the NGINX data planes by watching Gateway API resources and other Kubernetes objects such as Services, Endpoints, and Secrets.

Key functionalities include:

  • Dynamic provisioning: When a new Gateway resource is created, the control plane automatically provisions a dedicated NGINX Deployment and exposes it using a Service.
  • Configuration management: Kubernetes and Gateway API resources are translated into NGINX configurations, which are securely delivered to the data plane pods via a gRPC connection to the NGINX Agent.
  • Secure communication: By default, the gRPC connection uses self-signed certificates generated during installation. Integration with cert-manager is also supported for optional certificate management.

Data Plane: Autonomous Traffic Management

Each NGINX data plane pod can be provisioned as an independent Deployment or DaemonSet containing an nginx container. This container runs both the nginx process and the NGINX agent, which is responsible for:

  • Applying configurations: The agent receives updates from the control plane and applies them to the NGINX instance.
  • Handling reloads: NGINX Agent handles configuration reconciliation and reloading NGINX, eliminating the need for shared volumes or Unix signals between the control plane and data plane pods.

Gateway Resource Management

Users can have multiple gateways running side-by-side in the same cluster. This supports flexible operation and isolation across Gateways:

  • Concurrent Gateways: Multiple Gateway objects can run simultaneously within a single installation.
  • 1:1 resource mapping: Each Gateway resource corresponds uniquely to a dedicated data plane deployment, ensuring clear delineation of ownership and operational segregation.

High-level overview of NGINX Gateway Fabric in execution

This figure depicts an example of NGINX Gateway Fabric exposing three web applications within a Kubernetes cluster to clients on the internet:

graph LR
    %% Nodes and Relationships
    subgraph KubernetesCluster["Kubernetes Cluster"]
        subgraph NamespaceNGF["Namespace: nginx-gateway"]
            NGFControlPlanePod["NGINX Gateway Fabric Control Plane Pod"]
            NGFControlPlanePod --> KubernetesAPI["Kubernetes API"]
        end
        subgraph ApplicationsNamespaceA["Namespace: applications"]
            subgraph DataplaneComponentsA["Dataplane Components"]
                GatewayA["Gateway A
Listener: *.example.com"]
                subgraph NGINXDataPlanePodA["NGINX Data Plane Pod"]
                    subgraph NGINXContainerA["NGINX Container"]
                        NGINXProcessA["NGINX Process"]
                        NGINXAgentA["NGINX Agent"]
                    end
                end
            end
            subgraph HTTPRouteAAndApplications["HTTPRoutes and Applications"]
                HTTPRouteA["HTTPRoute A
Host: a.example.com"]
                HTTPRouteB["HTTPRoute B
Host: b.example.com"]
                ApplicationA["Application A
Pods: 2"]
                ApplicationB["Application B
Pods: 1"]
            end
        end
        subgraph ApplicationsNamespaceB["Namespace: applications-2"]
            subgraph DataplaneComponentsB["Dataplane Components"]
                GatewayB["Gateway B
Listener: *.other-example.com"]
                subgraph NGINXDataPlanePodB["NGINX Data Plane Pod"]
                    subgraph NGINXContainerB["NGINX Container"]
                        NGINXProcessB["NGINX Process"]
                        NGINXAgentB["NGINX Agent"]
                    end
                end
            end
            subgraph HTTPRouteBandApplications["HTTPRoutes and Applications"]
                HTTPRouteC["HTTPRoute C
Host: c.other-example.com"]
                ApplicationC["Application C
Pods: 1"]
            end
        end
    end
    subgraph UsersAndClients["Users and Clients"]
        UserOperator["Cluster Operator"]
        UserDevA["Application Developer A"]
        UserDevB["Application Developer B"]
        ClientA["Client A"]
        ClientB["Client B"]
    end
    subgraph SharedInfrastructure["Public Endpoint"]
        PublicEndpoint["TCP Load Balancer / NodePort"]
    end
    %% Updated Traffic Flow
    ClientA-->|a.example.com|PublicEndpoint
    ClientB-->|c.other-example.com|PublicEndpoint
    PublicEndpoint==>NGINXProcessA
    PublicEndpoint==>NGINXProcessB
    NGINXProcessA==>ApplicationA
    NGINXProcessA==>ApplicationB
    NGINXProcessB==>ApplicationC
    %% Kubernetes Configuration Flow
    HTTPRouteA-->GatewayA
    HTTPRouteB-->GatewayA
    HTTPRouteC-->GatewayB
    UserOperator-->KubernetesAPI
    NGFControlPlanePod--gRPC-->NGINXAgentA
    NGFControlPlanePod--gRPC-->NGINXAgentB
    NGINXAgentA-->NGINXProcessA
    NGINXAgentB-->NGINXProcessB
    UserDevA-->KubernetesAPI
    UserDevB-->KubernetesAPI
    %% Styling
    style UserOperator fill:#66CDAA,stroke:#333,stroke-width:2px
    style GatewayA fill:#66CDAA,stroke:#333,stroke-width:2px
    style GatewayB fill:#66CDAA,stroke:#333,stroke-width:2px
    style NGFControlPlanePod fill:#66CDAA,stroke:#333,stroke-width:2px
    style NGINXProcessA fill:#66CDAA,stroke:#333,stroke-width:2px
    style NGINXProcessB fill:#66CDAA,stroke:#333,stroke-width:2px
    style KubernetesAPI fill:#9370DB,stroke:#333,stroke-width:2px
    style HTTPRouteAAndApplications fill:#E0FFFF,stroke:#333,stroke-width:2px
    style HTTPRouteBandApplications fill:#E0FFFF,stroke:#333,stroke-width:2px
    style UserDevA fill:#FFA07A,stroke:#333,stroke-width:2px
    style HTTPRouteA fill:#FFA07A,stroke:#333,stroke-width:2px
    style HTTPRouteB fill:#FFA07A,stroke:#333,stroke-width:2px
    style ApplicationA fill:#FFA07A,stroke:#333,stroke-width:2px
    style ApplicationB fill:#FFA07A,stroke:#333,stroke-width:2px
    style ClientA fill:#FFA07A,stroke:#333,stroke-width:2px
    style UserDevB fill:#87CEEB,stroke:#333,stroke-width:2px
    style HTTPRouteC fill:#87CEEB,stroke:#333,stroke-width:2px
    style ApplicationC fill:#87CEEB,stroke:#333,stroke-width:2px
    style ClientB fill:#87CEEB,stroke:#333,stroke-width:2px
    style PublicEndpoint fill:#FFD700,stroke:#333,stroke-width:2px
The figure does not show many of the necessary Kubernetes resources the Cluster Operators and Application Developers need to create, like deployment and services.

The figure shows:

Category Description
Namespaces - Namespace: nginx-gateway: Contains the NGINX Gateway Fabric Control Plane Pod, responsible for managing Gateway API configurations and provisioning NGINX Data Plane Pods.
- Namespace: applications: Contains Gateway A for *.example.com, handling Application A and Application B.
- Namespace: applications-2: Contains Gateway B for *.other-example.com, handling Application C.
Users - Cluster Operator: Sets up the NGINX Gateway Fabric Control Plane Pod and manages Gateway API resources by provisioning Gateways (A and B).
- Developers A & B: Developers deploy their applications and create HTTPRoutes associated with their Gateways.
Clients - Client A: Interacts with Application A through a.example.com.
- Client B: Interacts with Application C through c.other-example.com.
NGINX Gateway Fabric Control Plane Pod The control plane pod, deployed in the nginx-gateway namespace, communicates with the Kubernetes API to:
- Fetch Gateway API resources.
- Dynamically provision and configure NGINX Data Plane Pods.
- Deliver traffic routing and configuration updates to NGINX Agent instances over gRPC.
Gateways - Gateway A: Listens for requests under *.example.com. Routes:
   • HTTPRoute A: Routes requests to a.example.com into Application A.
   • HTTPRoute B: Routes requests to b.example.com into Application B.
- Gateway B: Listens for requests under *.other-example.com. Routes:
   • HTTPRoute C: Routes requests to c.other-example.com into Application C.
Applications - Application A: Deployed by Developer A (2 pods), routed by Gateway A via HTTPRoute A.
- Application B: Deployed by Developer A (1 pod), routed by Gateway A via HTTPRoute B.
- Application C: Deployed by Developer B (1 pod), routed by Gateway B via HTTPRoute C.
NGINX Data Plane Pods - NGINX Data Plane Pod A: Handles traffic routed from Gateway A:
   • NGINX Process A: Forwards requests to Application A and Application B as defined in Gateway A’s HTTPRoutes.
   • NGINX Agent A: Receives configuration updates from the NGINX Gateway Fabric Control Plane Pod via gRPC.
- NGINX Data Plane Pod B: Manages traffic routed from Gateway B:
   • NGINX Process B: Forwards requests to Application C as defined in Gateway B’s HTTPRoute.
   • NGINX Agent B: Receives configuration updates via gRPC from the NGINX Gateway Fabric Control Plane Pod.
Traffic Flow - Client A:
   1. Sends requests to a.example.com via the Public Endpoint.
   2. Requests are routed by Gateway A and processed by NGINX Process A.
   3. Traffic is forwarded to Application A.
- Client B:
   1. Sends requests to c.other-example.com via the Public Endpoint.
   2. Requests are routed by Gateway B and processed by NGINX Process B.
   3. Traffic is forwarded to Application C.
Public Endpoint A shared entry point (TCP Load Balancer or NodePort) that exposes the NGINX Data Plane externally to forward client traffic into the cluster.
Kubernetes API Acts as the central hub for resource management:
- Fetches Gateway API resources for Gateway A and Gateway B.
- Facilitates NGINX configuration updates via the NGINX Gateway Fabric Control Plane Pod.

Color Coding :

  • Cluster Operator resources (e.g., NGINX Gateway Fabric, NGINX Pods and Gateways) are marked in green.
  • Resources owned by Application Developer A (e.g., HTTPRoute A, Application A) are marked in orange.
  • Resources owned by Application Developer B (e.g., HTTPRoute B, Application C) are marked in blue.

NGINX Gateway Fabric: Component Communication Workflow 

graph LR
    %% Main Components
    subgraph nginx-gateway["Namespace: nginx-gateway"]
        NGFPod[NGINX Gateway Fabric Control Plane Pod]
    end
    
    subgraph NGINXDataPlane["NGINX Data Plane Pod"]
        NGINXAgent[NGINX Agent]
        NGINXMaster[NGINX Master]
        NGINXWorker[NGINX Worker]
        ConfigFiles[Config Files]
    end

    F5Telemetry[F5 Telemetry Service]
    PrometheusMonitor[Prometheus]
    KubernetesAPI[Kubernetes API]
    Client[Client]
    BackendApplication[Backend Application]

    %% High-Level Configuration Flow
    KubernetesAPI -->|"(1) Updates Resources"| NGFPod
    NGFPod -->|"(2) Sends Configuration Metadata via gRPC"| NGINXAgent
    NGINXAgent -->|"(3) Validates & Writes Configuration"| ConfigFiles
    NGINXAgent -->|"(4) Signals NGINX Master to Reload"| NGINXMaster

    %% Prometheus Monitoring
    PrometheusMonitor -->|"(5) Fetches Metrics from NGINX"| NGINXWorker

    %% Telemetry Data
    NGFPod -->|"(6) Sends Telemetry Data"| F5Telemetry

    %% Client Traffic Flow
    Client -->|"(7) Sends Traffic"| NGINXWorker
    NGINXWorker -->|"(8) Routes Traffic"| BackendApplication

    %% Styling
    classDef important fill:#66CDAA,stroke:#333,stroke-width:2px;
    classDef metrics fill:#FFC0CB,stroke:#333,stroke-width:2px;
    classDef io fill:#FFD700,stroke:#333,stroke-width:2px;
    class KubernetesAPI,NGFPod important;
    class PrometheusMonitor,F5Telemetry metrics;
    class NGINXAgent,NGINXMaster,NGINXWorker,ConfigFiles io;
    class Client,BackendApplication important;

The following table describes the connections, preceeded by their types in parentheses. For brevity, the suffix “process” has been omitted from the process descriptions.

# Component/Protocol Description
1 Kubernetes API (HTTPS) Kubernetes API → NGINX Gateway Fabric Control Plane Pod: The NGINX Gateway Fabric Control Plane Pod (in the nginx-gateway namespace) watches the Kubernetes API for updates to Gateway API resources (e.g., Gateways, HTTPRoutes), fetching the latest configuration to manage routing and traffic control.
2 gRPC NGINX Gateway Fabric Control Plane Pod → NGINX Agent: The NGINX Gateway Fabric Control Plane Pod processes Gateway API resources, generates NGINX configuration settings, and securely delivers them to the NGINX Agent inside the NGINX Data Plane Pod via gRPC.
3 File I/O NGINX Agent → Config Files: The NGINX Agent (within the NGINX Data Plane Pod) validates the configuration metadata received from the Control Plane Pod and writes it to NGINX configuration files within the pod. These files store dynamic routing rules and traffic settings.
4 Signal NGINX Agent → NGINX Master: After writing the configuration files, the NGINX Agent signals the NGINX Master process to reload the configuration. This ensures the NGINX Data Plane Pod immediately applies the updated routes and settings.
5 HTTP/HTTPS Prometheus → NGINX Worker: Prometheus collects runtime metrics (e.g., traffic statistics, request rates, and active connections) from the NGINX Worker process, which is part of the NGINX Data Plane Pod. The /metrics endpoint exposes these metrics for monitoring and observability.
6 HTTPS NGINX Gateway Fabric Control Plane Pod → F5 Telemetry Service: The NGINX Gateway Fabric Control Plane Pod sends telemetry data (e.g., API requests handled, usage metrics, performance stats, error rates) to the external F5 Telemetry Service for centralized monitoring and diagnostics.
7 HTTP/HTTPS Client → NGINX Worker: Clients send incoming application traffic (e.g., HTTP/HTTPS requests) to the NGINX Worker process within the NGINX Data Plane Pod. These requests are typically routed through a shared Public Endpoint (e.g., LoadBalancer or NodePort) before reaching the NGINX Data Plane.
8 HTTP/HTTPS NGINX Worker → Backend Application: The NGINX Worker process forwards client traffic to the appropriate backend application services (e.g., Pods) as defined in the routing rules and configuration received from the Control Plane Pod.

Additional features and enhancements when using NGINX Plus

NGINX Gateway Fabric supports both NGINX Open Source and NGINX Plus. While the previous diagram shows NGINX Open Source, using NGINX Plus provides additional capabilities, including:

  • The ability for administrators to connect to the NGINX Plus API on port 8765 (restricted to localhost by default).
  • Dynamic updates to upstream servers without requiring a full reload:
    • Changes to upstream servers, such as application scaling (e.g., adding or removing pods in Kubernetes), can be applied using the NGINX Plus API.
    • This reduces the frequency of configuration reloads, minimizing potential disruptions and improving system stability during updates.

These features enable reduced downtime, improved performance during scaling events, and more fine-grained control over traffic management.

Resilience and fault isolation

This architecture separates the control plane and data plane, creating clear operational boundaries that improve resilience and fault isolation:

Control plane resilience

In the event of a control plane failure or downtime:

  • Existing data plane pods continue serving traffic using their last-valid cached configurations.
  • Updates to routes or Gateways are temporarily paused, but stable traffic delivery continues without degradation.
  • Recovery restores functionality, resynchronizing configuration updates seamlessly.

Data plane resilience

If a data plane pod encounters an outage or restarts:

  • Only routes tied to the specific linked Gateway object experience brief disruptions.
  • Configurations automatically resynchronize with the data plane upon pod restart, minimizing the scope of impact.
  • Other data plane pods remain unaffected and continue serving traffic normally.

Pod readiness

The control plane (nginx-gateway) and data plane (nginx) containers provide a readiness endpoint at /readyz. A readiness probe periodically checks this endpoint during startup. The probe reports 200 OK when:

  • The control plane is ready to configure the NGINX data planes.
  • The data plane is ready to handle traffic.

This marks the pods ready ensuring traffic is routed to healthy pods, ensuring reliable startup and smooth operations.