How to configure global rate limit

    After installation, you can see that the following two NSs have been added to the cluster, and the Dubbo and Thrift demo applications are installed in these two NSs. You can choose either of them to test.

    What is global rate limiting?

    Unlike , when using a global rate limiting, all service instances share a single rate limiting quota. A global rate limiting server is used to ensure that. When receiving a request, Sidecar Proxy on the server side will first send a rate limiting query request to rate limiting server, which will read the rules in its own configuration file, then determine whether the request triggers a rate limiting condition according to the rules, finally return result to Sidecar Proxy. Based on the result of this rate limiting, Sidecar Proxy decides whether to reject this request or to continue processing it.

    In global rate limiting, the rate limiting decision is made at the global rate limiting server with a fixed quota, therefore, the number of client requests that are allowed to send to the server won’t change even the number of service instances increases. Global rate limiting also introduces an additional hop at the request path, resulting in more latency. When there’s a large number of client requests, the global rate limiting server may become a potential bottleneck. Besides, global rate limiting is more complicated to deploy and manage than local limiting.

    If you just want to keep the number of requests sent to a single service instance within a reasonable limit, you should use local rate limiting. That’s because local rate limiting is handled separately at each service instance’s Sidecar Proxy, local rate limiting is more precise and reasonable for controlling inbound requests to service instances in this scenario. Used along with an appropriate HPA policy trigger by rate limiting metrics, we can keep the server cluster healthy by horizontally scaling the server cluster when all the existing service instances reach their limiting quota.

    Deploy the rate limiting server

    The rate limiting server has already been deployed in the example application, so you don’t need to deploy the rate limiting server by youself.

    The rate limiting rules for global rate limiting need to be set in the rate limiting server side. The following rate limiting rule has already been set in the demo, which allows 5 request per minute for the sayHello interface.

    2. descriptors:
    3.   - key: method
    4.     value: "sayHello"
    5.     rate_limit:
    6.       unit: minute
    7.       requests_per_unit: 5

    The deployment scripts can be found here:

    Once rate limiting is enabled for a service via MetaRouter, the service’s Sidecar Proxy will initiate a rate limiting request to the rate limiting server upon receipt of the request and will decide whether to continue processing the request or reject it based on the response of the rate limiting server.

    If the match condition is present, only requests that match the condition will be sent to the rate limiting server. If the match condition is not set, then all requests for that service will be sent to the rate limiting server. The match condition can be used to filter out requests locally that do not require rate limiting, improving the efficiency of request processing.

    Using the aerakictl command to view the client’s application log, you can see that the client can only successfully send out 5 requests per minute.

    1. ➜  ~ aerakictl_app_log client meta-thrift -f --tail 10
    2. Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
    3. Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
    4. Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
    5. Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
    6. Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
    7. org.apache.thrift.TApplicationException: meta protocol local rate limit: request '6' has been rate limited
    8.         at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
    9.         at org.aeraki.HelloService$Client.recv_sayHello(HelloService.java:61)
    10.         at org.aeraki.HelloService$Client.sayHello(HelloService.java:48)
    11.         at org.aeraki.HelloClient.main(HelloClient.java:44)
    12. Connected to thrift-sample-server
    13. org.apache.thrift.TApplicationException: meta protocol local rate limit: request '7' has been rate limited
    14. ...

    Understand what happened

    In the configuration sent to the Sidecar Proxy, Aeraki sets up the MetaProtocol Proxy in the FilterChain corresponding to the service in the VirtualInbound Listener.

    Aeraki translates MetaRouter to global rate limiting filter route configuration, which is distributed to the MetaProtocol Proxy via Aeraki.

    The configuration of the sidecar proxy for the service can be viewed with the following command：

    1. {
    2.  "typed_config": {
    3.   "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
    4.   "type_url": "type.googleapis.com/aeraki.meta_protocol_proxy.v1alpha.MetaProtocolProxy",
    6.    "stat_prefix": "inbound|9090||",
    7.    "application_protocol": "thrift",
    8.    "route_config": {
    9.     "name": "inbound|9090||",
    10.     "routes": [
    11.      {
    12.       "route": {
    13.        "cluster": "inbound|9090||"
    14.       }
    15.      }
    16.     ]
    17.    },
    18.    "codec": {
    19.     "name": "aeraki.meta_protocol.codec.thrift"
    20.    },
    21.    "meta_protocol_filters": [
    22.     {
    23.      "name": "aeraki.meta_protocol.filters.ratelimit",
    24.      "config": {
    25.       "@type": "type.googleapis.com/aeraki.meta_protocol_proxy.filters.ratelimit.v1alpha.RateLimit",
    26.       "match": {
    27.        "metadata": [
    28.         {
    29.          "name": "method",
    31.        ]
    32.       },
    33.       "domain": "production",
    34.       "timeout": "0.100s",
    35.       "failure_mode_deny": true,
    36.       "rate_limit_service": {
    37.        "grpc_service": {
    38.         "envoy_grpc": {
    39.          "cluster_name": "outbound|8081||rate-limit-server.meta-thrift.svc.cluster.local"
    40.         }
    41.        }
    42.       },
    43.       "descriptors": [
    44.        {
    45.         "property": "method",
    46.         "descriptor_key": "method"
    47.        }
    48.       ]
    49.      }
    50.     },
    51.     {
    52.      "name": "aeraki.meta_protocol.filters.router"
    53.     }
    54.    ]
    55.   }
    57. }

    Last modified May 9, 2022: