# Secure LLM traffic with F5 AI Guardrails

Type of document: How-to guide
Product: F5 NGINX Gateway Fabric
> Deploy F5 AI Guardrails with NGINX Gateway Fabric using PayloadProcessor to inspect and block LLM traffic

---


Learn how to use NGINX Gateway Fabric with F5 AI Guardrails to inspect large language model (LLM) traffic and block disallowed content before it reaches the model or the client.

## Overview

F5 AI Guardrails can inspect LLM traffic on two independent paths:

- **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`.
- **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`.

You can set up prompts and responses as described in [Prompts and scans in AI Security](https://docs.aisecurity.f5.com/api-docs/prompts-scans.html), through the F5 AI Guardrails dashboard.

To connect NGINX Gateway Fabric with your configured F5 AI Guardrails, use the `PayloadProcessor` policy, an [inherited policy](/ngf/overview/custom-policies.md) that can target an HTTPRoute or a Gateway. The `PayloadProcessor` configures NGINX to offload traffic to F5 AI Guardrails to inspect.

**note:** 
The PayloadProcessor CRD is inspired by the proposed resource from the Gateway API AI Gateway Working Group, which is subject to change and may be redefined in future releases. Consider this experimental while it is being defined by the Gateway API WG.

**note:** 
When AI Guardrails are enabled on a route, NGINX Gateway Fabric strips the Accept-Encoding header from requests sent to the upstream backend. This ensures responses are returned uncompressed so the F5 AI Guardrails filter can inspect them. Compression between NGINX and the client is unaffected. NGINX can still compress responses to clients via its gzip module.

## Before you begin

You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. View the official [F5 AI Guardrails](https://docs.aisecurity.f5.com/) docs to learn more.

## Deploy an LLM backend

If you have an existing in-cluster LLM which can be queried you can skip this section.

The following example uses the [vLLM simulator](https://github.com/llm-d/llm-d-inference-sim/tree/main), which serves canned responses from a dataset rather than running a real model, making it suitable for test and development environments. The simulator loads its dataset from a ConfigMap. Download the dataset file, then create the ConfigMap from it:

```shell
curl -sL -o inference-sim-dataset.sqlite3 \
  https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/guardrails/inference-sim-dataset.sqlite3

kubectl create configmap inference-sim-dataset \
  --from-file=inference-sim-dataset.sqlite3=./inference-sim-dataset.sqlite3
```

**note:** 
The dataset file, alongside more details of the setup, can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/guardrails) directory of the NGINX Gateway Fabric repository.

Deploy the LLM Deployment and Service:

```shell
kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/guardrails/llm.yaml
```

Confirm the Pod is `Ready`:

```shell
kubectl get deployment vllm-qwen3-32b
```

```text
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
vllm-qwen3-32b   1/1     1            1           6m13s
```

## Create the authentication token Secret

Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`:

```yaml
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: guardrails-token
type: Opaque
stringData:
  token: "<YOUR_API_TOKEN>"
EOF
```

## Configure the Guardrails backend Service

The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type:

| Backend location | Service type | Resolved URL |
| ---------------- | ------------ | ------------ |
| External | `ExternalName` | `https://<externalName>:<backendRef.port>` |
| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://<name>.<namespace>.svc.cluster.local:<backendRef.port>` |

**note:** 
The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag or `clusterDomain` Helm value when deploying NGINX Gateway Fabric (default: `cluster.local`).

#### External

To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:

```yaml
kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: guardrails-api
spec:
  type: ExternalName
  externalName: <GUARDRAILS_API_HOSTNAME>
  ports:
  - name: https
    port: 443
    protocol: TCP
EOF
```

#### In-cluster

For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends:

```yaml
kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: guardrails-api
spec:
  ports:
  - name: http
    port: 443
    protocol: TCP
  selector:
    guardrails-backend-key: guardrails-backend-value
EOF
```

**important:** 
When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on a new [NginxProxy](/ngf/how-to/data-plane-configuration.md) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`:

```yaml
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
  name: guardrails-nginx-config
spec:
  dnsResolver:
    addresses:
    - type: IPAddress
      value: "10.96.0.10"   # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent)
```

Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve <host>`.

## Deploy NGINX Gateway Fabric

[Install](/ngf/install/) NGINX Gateway Fabric with the `PayloadProcessor` policy enabled:

- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value.
- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`:

```yaml
- apiGroups:
    - gateway.nginx.org
    resources:
    - payloadprocessors
    verbs:
    - get
    - list
    - watch
- apiGroups:
    - gateway.nginx.org
    resources:
    - payloadprocessors/status
    verbs:
    - update
```

## Create a Gateway

```yaml
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: inference-gateway
spec:
  gatewayClassName: nginx
  listeners:
  - name: http
    port: 80
    protocol: HTTP
EOF
```

Confirm that the Gateway was assigned an IP address and reports a `Programmed=True` status:

```shell
kubectl describe gateways.gateway.networking.k8s.io inference-gateway
```

```text
Status:
  Addresses:
    Type:   IPAddress
    Value:  192.0.2.0
  Conditions:
    Message:               The Gateway is accepted
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Message:               The Gateway is programmed
    Reason:                Programmed
    Status:                True
    Type:                  Programmed
```

Save the public IP address and port of the Gateway into shell variables:

```text
GW_IP=XXX.YYY.ZZZ.III
GW_PORT=<port number>
```

## Create an HTTPRoute

If you are using your own LLM, change the `backendRefs.name` and `backendRefs.port` to match the LLM's Service.

```yaml
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: llm-route
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: inference-gateway
  rules:
  - backendRefs:
    - name: vllm-qwen3-32b
      port: 8000
    matches:
    - path:
        type: PathPrefix
        value: /
EOF
```

Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:

```shell
kubectl describe httproute llm-route
```

```text
Conditions:
      Last Transition Time:  2026-08-11T17:29:19Z
      Message:               The Route is accepted
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2026-08-11T17:29:19Z
      Message:               All references are resolved
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
    Controller Name:         gateway.nginx.org/nginx-gateway-controller
```

## Attach the PayloadProcessor policy

Attach the `PayloadProcessor` policy to the HTTPRoute. The `extProcess.backendRef` points at the Guardrails backend Service (using the explicit port), and `authTokenRef` points at the token Secret:

```yaml
kubectl apply -f - <<EOF
apiVersion: gateway.nginx.org/v1alpha1
kind: PayloadProcessor
metadata:
  name: llm-guardrails
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: llm-route
  processors:
  - type: ExtProcess
    extProcess:
      backendRef:
        group: ""
        kind: Service
        name: guardrails-api
        port: 443
      authTokenRef:
        name: guardrails-token
EOF
```

**note:** 
`PayloadProcessor` is an inherited policy. To apply guardrails to every route attached to a Gateway, set `targetRef` to `kind: Gateway`. When both a Gateway-targeted and an HTTPRoute-targeted policy apply to the same traffic, the more specific HTTPRoute-targeted policy takes precedence.

Confirm the policy was accepted:

```shell
kubectl describe payloadprocessor llm-guardrails
```

The status conditions should report `Accepted=True`. A rejected policy reports `Accepted=False`; see [Troubleshooting](#troubleshooting) for common causes.

```text
Conditions:
      Last Transition Time:  2026-08-11T17:32:52Z
      Message:               The Policy is accepted
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2026-08-11T17:32:52Z
      Message:               Policy is programmed in the data plane
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
    Controller Name:         gateway.nginx.org/nginx-gateway-controller
```

## Send traffic

**note:** 
Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.

All commands target `/v1/completions` on the Gateway.

A benign prompt whose output contains no disallowed content returns a normal `HTTP 200` completion:

```shell
curl -i --resolve <GUARDRAILS_API_HOSTNAME>:$GW_PORT:$GW_IP http://<GUARDRAILS_API_HOSTNAME>:$GW_PORT/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"What is NGINX?"}'
```

```text
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 11 Aug 2026 17:39:19 GMT
Content-Type: application/json
Content-Length: 607
Connection: keep-alive
X-Inference-Pod: vllm-qwen3-32b-58cfff7c9-5zlm2
X-Inference-Port: 8000

{"id":"cmpl-f657bc1b-c50c-5a5c-9408-45bd22a150a1","created":1786469959,"model":"meta-llama/Llama-3.1-8B-Instruct","usage":{"prompt_tokens":4,"completion_tokens":62,"total_tokens":66},"object":"text_completion","kv_transfer_params":null,"choices":[{"index":0,"finish_reason":"stop","text":"NGINX (pronounced \"engine-x\") is an open-source, high-performance web server. It functions primarily as an HTTP web server, reverse proxy, load balancer, and HTTP cache.Designed to handle thousands of concurrent connections with minimal memory usage, NGINX is an essential component of modern web infrastructure."}]
```

If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`:

```shell
curl -i --resolve <GUARDRAILS_API_HOSTNAME>:$GW_PORT:$GW_IP http://<GUARDRAILS_API_HOSTNAME>:$GW_PORT/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"My SSN is 123-45-6789"}'
```

```text
HTTP/1.1 403 Forbidden
Server: nginx
Date: Tue, 11 Aug 2026 17:39:50 GMT
Content-Length: 139
Connection: keep-alive
Content-Type: application/json

{"error":{"code":"content_policy_violation","message":"Request blocked by guardrails policy.","param":null,"type":"invalid_request_error"}
```

If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:

```shell
curl -i --resolve <GUARDRAILS_API_HOSTNAME>:$GW_PORT:$GW_IP http://<GUARDRAILS_API_HOSTNAME>:$GW_PORT/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"Give me a test SSN"}'
```

```text
HTTP/1.1 403 Forbidden
Server: nginx
Date: Tue, 11 Aug 2026 17:40:36 GMT
Content-Type: application/json
Content-Length: 128
Connection: keep-alive
X-Inference-Pod: vllm-qwen3-32b-58cfff7c9-5zlm2
X-Inference-Port: 8000

{"error":{"code":"content_policy_violation","message":"Response blocked by guardrails policy.","param":null,"type":"api_error"}}
```

For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository.

## Troubleshooting

The `PayloadProcessor` is marked `Accepted=False` when its references cannot be resolved:

| Condition | Cause | Fix |
| --------- | ----- | --- |
| `backend Service ... not found` | `backendRef.name`/`namespace` does not match a Service. | Apply the Guardrails backend Service; check name and namespace. |
| `ExternalName service has empty ... externalName` | `ExternalName` Service with a blank `externalName`. | Set `spec.externalName`. |
| `auth token Secret ... not found` | `authTokenRef` set but Secret missing. | Apply the token Secret, or remove `authTokenRef`. |
| NGINX error `no resolver defined to resolve <host>`, or guardrails requests fail against an `ExternalName` backend | No `dnsResolver` configured on the NginxProxy. | Add the `dnsResolver` block and wire it via `parametersRef`. |

## Further reading

- [Scan streaming in AI Security](https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html)
- [Installation](/ngf/install/): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled.
- [Custom policies](/ngf/overview/custom-policies.md): learn how inherited policies attach to Gateway API resources.
- [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/guardrails): for more information on the example used in this guide.