#Configuring Token Quotas

#Enable the global rate limit backend

The local rate limiter cannot accumulate response-derived token cost, so the gateway must use the Global Rate Limit service backed by Redis. Create a Redis instance from Cache Service for Redis and copy its access address from the instance detail page. Set that address in the envoy-gateway-config ConfigMap (namespace envoy-gateway-system), under data."envoy-gateway.yaml":

kubectl edit configmap envoy-gateway-config -n envoy-gateway-system

Add the rateLimit block under the top-level config (keep any existing keys such as gateway: and provider: intact):

rateLimit:
  backend:
    type: Redis
    redis:
      url: <redis-host>:<redis-port>  # access address of the Redis instance
      # tls:                          # uncomment if Redis requires TLS
      #   certificateRef:
      #     name: redis-client-cert

• <redis-host>:<redis-port>: the access address copied from the Redis instance detail page.
• For a password-protected Redis, also create an Opaque Secret with key redis-username and redis-password, then reference it via rateLimit.backend.redis.auth.passwordRef. See the Envoy Gateway rate-limit docs for the full schema.

The Envoy Gateway control plane reads this bootstrap configuration only at startup and does not hot-reload it, so restart its Deployment to apply the change, then confirm the dedicated envoy-ratelimit Deployment is healthy:

kubectl rollout restart deployment envoy-gateway -n envoy-gateway-system
kubectl rollout status  deployment envoy-gateway   -n envoy-gateway-system
kubectl rollout status  deployment envoy-ratelimit -n envoy-gateway-system
# the envoy-ratelimit Deployment is created by the EG controller the first time
# rateLimit.backend is set; if it never appears, the config above was not parsed

If the Gateway was already running before the rate limit backend was enabled, also restart its data plane so the proxy picks up the rate limit service:

kubectl rollout restart deployment -n envoy-gateway-system \
  -l gateway.envoyproxy.io/owning-gateway-name=<gateway-name>

#Capture token usage on the route

Add llmRequestCosts to the AIGatewayRoute so the gateway writes token counts into Envoy dynamic metadata (the per-request scratch space filters use to talk to each other) under the namespace io.envoy.ai_gateway. The rate-limit filter reads from this namespace in the next step.

apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: AIGatewayRoute
metadata:
  name: <aigatewayroute-name>
  namespace: <your-namespace>
spec:
  # ... existing parentRefs and rules ...
  llmRequestCosts:
    - metadataKey: llm_input_token
      type: InputToken
    - metadataKey: llm_output_token
      type: OutputToken
    - metadataKey: llm_total_token
      type: TotalToken

• metadataKey: the key under io.envoy.ai_gateway where the count is written. Pick any name; the BackendTrafficPolicy below must reference the same string.

• type: InputToken counts the prompt, OutputToken counts the completion, TotalToken is the sum. Use CEL for a custom formula — for example, charge output tokens 3× because they are slower:

  - metadataKey: llm_weighted_cost
    type: CEL
    cel: "input_tokens + output_tokens * 3"

Apply and confirm the route is still accepted (the new field should not break translation):

kubectl get aigatewayroute <aigatewayroute-name> -n <your-namespace> \
  -o jsonpath='{.status.conditions[?(@.type=="Accepted")].status}'
# expect: True

#Define the token budget by identity

Attach a BackendTrafficPolicy with a Global rate limit. Set the request cost to 0 and the response cost to the captured token metadata, so that only tokens count against the limit. Use clientSelectors to scope the budget per identity and per model.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: maas-token-quota
  namespace: <your-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: <aigatewayroute-name>
  rateLimit:
    type: Global
    global:
      rules:
        - clientSelectors:
            - headers:
                - name: x-user-id       # identity header from the SecurityPolicy
                  type: Distinct
                - name: x-ai-eg-model
                  type: Exact
                  value: my-llm
          limit:
            requests: 200000             # 200k tokens per window
            unit: Hour
          cost:
            request:
              from: Number
              number: 0                   # requests do not count
            response:
              from: Metadata              # tokens count
              metadata:
                namespace: io.envoy.ai_gateway
                key: llm_total_token

• x-user-id with type: Distinct gives each caller an independent counter, which produces a per-user quota. Use x-user-group to aggregate a department against one budget, or match a specific group such as premium with type: Exact for tiered limits.
• limit.requests is interpreted as a token budget here, because the cost is sourced from token metadata. With 200000 tokens/hour and a typical chat call costing roughly 1.5–2k tokens, expect ~100–130 calls/hour per caller before throttling kicks in.
• cost.request.number: 0 means a request that fails to reach the upstream (e.g. malformed body) consumes no quota. Set it to 1 if you want pre-flight throttling on call count as well.
• cost.response.metadata.key must match a metadataKey declared on the route.