Enabling Rate Limits using Envoy

    1. Setup Istio in a Kubernetes cluster by following the instructions in the .

    2. Deploy the Bookinfo sample application.

    Envoy supports two kinds of rate limiting: global and local. Global rate limiting uses a global gRPC rate limiting service to provide rate limiting for the entire mesh. Local rate limiting is used to limit the rate of requests per service instance. Local rate limiting can be used in conjunction with global rate limiting to reduce load on the global rate limiting service.

    In this task you will configure Envoy to rate limit traffic to a specific path of a service using both global and local rate limits.

    Envoy can be used to for your mesh. Global rate limiting in Envoy uses a gRPC API for requesting quota from a rate limiting service. A reference implementation of the API, written in Go with a Redis backend, is used below.

    1. Use the following configmap to to rate limit requests to the path /productpage at 1 req/min and all other requests at 100 req/min.

    2. Create a global rate limit service which implements Envoy’s rate limit service protocol. As a reference, a demo configuration can be found , which is based on a reference implementation provided by Envoy.

      1. $ kubectl apply -f @samples/ratelimit/rate-limit-service.yaml@

    3. Apply an EnvoyFilter to the ingressgateway to enable global rate limiting using Envoy’s global rate limit filter.

      The first patch inserts the envoy.filters.http.ratelimit filter into the HTTP_FILTER chain. The rate_limit_service field specifies the external rate limit service, rate_limit_cluster in this case.

      The second patch defines the rate_limit_cluster, which provides the endpoint location of the external rate limit service.

      1. $ kubectl apply -f - <<EOF
      2. apiVersion: networking.istio.io/v1alpha3
      3. kind: EnvoyFilter
      4. metadata:
      5.   name: filter-ratelimit
      6.   namespace: istio-system
      7. spec:
      8.   workloadSelector:
      9.     # select by label in the same namespace
      10.     labels:
      11.       istio: ingressgateway
      12.   configPatches:
      13.     # The Envoy config you want to modify
      14.     - applyTo: HTTP_FILTER
      15.       match:
      16.         context: GATEWAY
      17.         listener:
      18.           filterChain:
      19.             filter:
      20.               name: "envoy.filters.network.http_connection_manager"
      21.               subFilter:
      22.                 name: "envoy.filters.http.router"
      23.       patch:
      24.         # Adds the Envoy Rate Limit Filter in HTTP filter chain.
      25.         value:
      26.           name: envoy.filters.http.ratelimit
      27.           typed_config:
      28.             "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
      29.             # domain can be anything! Match it to the ratelimter service config
      30.             domain: productpage-ratelimit
      31.             failure_mode_deny: true
      33.             rate_limit_service:
      34.               grpc_service:
      35.                 envoy_grpc:
      36.                   cluster_name: outbound|8081||ratelimit.default.svc.cluster.local
      37.                   authority: ratelimit.default.svc.cluster.local
      38.               transport_api_version: V3
      39. EOF

    4. Apply another EnvoyFilter to the ingressgateway that defines the route configuration on which to rate limit. This adds rate limit actions for any route from a virtual host named *.80.

    Envoy supports local rate limiting of L4 connections and HTTP requests. This allows you to apply rate limits at the instance level, in the proxy itself, without calling any other service.

    The following EnvoyFilter enables local rate limiting for any traffic through the productpage service. The HTTP_FILTER patch inserts the envoy.filters.http.local_ratelimit into the HTTP connection manager filter chain. The local rate limit filter’s token bucket is configured to allow 10 requests/min. The filter is also configured to add an x-local-rate-limit response header to requests that are blocked.

    The statistics mentioned on the are disabled by default. You can enable them with the following annotations during deployment:

    1. template:
    2.   metadata:
    3.     annotations:
    4.       proxy.istio.io/config: |-
    5.         proxyStatsMatcher:
    6.           inclusionRegexps:
    7.           - ".*http_local_rate_limit.*"
    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: EnvoyFilter
    4. metadata:
    5.   name: filter-local-ratelimit-svc
    6.   namespace: istio-system
    7. spec:
    8.   workloadSelector:
    9.     labels:
    10.       app: productpage
    11.   configPatches:
    12.     - applyTo: HTTP_FILTER
    13.       match:
    14.         listener:
    15.           filterChain:
    16.             filter:
    17.               name: "envoy.filters.network.http_connection_manager"
    18.       patch:
    19.         operation: INSERT_BEFORE
    20.         value:
    21.           name: envoy.filters.http.local_ratelimit
    23.             "@type": type.googleapis.com/udpa.type.v1.TypedStruct
    24.             type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
    25.             value:
    26.               stat_prefix: http_local_rate_limiter
    27.               token_bucket:
    28.                 max_tokens: 10
    29.                 tokens_per_fill: 10
    30.                 fill_interval: 60s
    31.               filter_enabled:
    32.                 runtime_key: local_rate_limit_enabled
    33.                 default_value:
    34.                   numerator: 100
    35.                   denominator: HUNDRED
    36.               filter_enforced:
    37.                 runtime_key: local_rate_limit_enforced
    38.                 default_value:
    39.                   numerator: 100
    40.                   denominator: HUNDRED
    41.               response_headers_to_add:
    42.                 - append: false
    43.                   header:
    44.                     key: x-local-rate-limit
    45.                     value: 'true'
    46. EOF

    The following EnvoyFilter enables local rate limiting for any traffic to port 80 of the productpage service. Unlike the previous configuration, there is no token_bucket included in the HTTP_FILTER patch. The token_bucket is instead defined in the second (HTTP_ROUTE) patch which includes a typed_per_filter_config for the envoy.filters.http.local_ratelimit local envoy filter, for routes to virtual host inbound|http|9080.

    Send traffic to the Bookinfo sample. Visit http://$GATEWAY_URL/productpage in your web browser or issue the following command:

    1. $ curl -s "http://$GATEWAY_URL/productpage" -o /dev/null -w "%{http_code}\n"
    2. 429

    $GATEWAY_URL is the value set in the Bookinfo example.

    You will see the first request go through but every following request within a minute will get a 429 response.

    Although the global rate limit at the ingress gateway limits requests to the productpage service at 1 req/min, the local rate limit for productpage instances allows 10 req/min. To confirm this, send internal productpage requests, from the ratings pod, using the following curl command:

    1. $ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -s productpage:9080/productpage -o /dev/null -w "%{http_code}\n"
    2. 429

    You should see no more than 10 req/min go through per instance.