Plugin Development - Caching Custom Entities

    Think of an api-key authentication plugin that needs to validate the api-key on every request, thus loading the custom credential object from the datastore on every request. When the client provides an api-key along with the request, normally you would query the datastore to check if that key exists, and then either block the request or retrieve the Consumer ID to identify the user. This would happen on every request, and it would be very inefficient:

    • Querying the datastore adds latency on every request, making the request processing slower.
    • The datastore would also be affected by an increase of load, potentially crashing or slowing down, which in turn would affect every Kong node.

    To avoid querying the datastore every time, we can cache custom entities in-memory on the node, so that frequent entity lookups don’t trigger a datastore query every time (only the first time), but happen in-memory, which is much faster and reliable that querying it from the datastore (especially under heavy load).

    Once you have defined your custom entities, you can cache them in-memory in your code by using the kong.cache module provided by the :

    There are 2 levels of cache:

    1. L1: Lua memory cache - local to an Nginx worker process. This can hold any type of Lua value.
    2. L2: Shared memory cache (SHM) - local to an Nginx node, but shared between all the workers. This can only hold scalar values, and hence requires (de)serialization of a more complex types such as Lua tables.

    When data is fetched from the database, it will be stored in both caches. If the same worker process requests the data again, it will retrieve the previously deserialized data from the Lua memory cache. If a different worker within the same Nginx node requests that data, it will find the data in the SHM, deserialize it (and store it in its own Lua memory cache) and then return it.

    This module exposes the following functions:

    Bringing back our authentication plugin example, to lookup a credential with a specific api-key, we would write something similar to:

    1. -- handler.lua
    2. local BasePlugin = require "kong.plugins.base_plugin"
    5. local kong = kong
    8. local function load_credential(key)
    9.   local credential, err = kong.db.keyauth_credentials:select_by_key(key)
    10.   if not credential then
    11.     return nil, err
    12.   end
    13.   return credential
    14. end
    17. local CustomHandler = BasePlugin:extend()
    20. CustomHandler.PRIORITY = 1010
    23. function CustomHandler:new()
    25. end
    28. function CustomHandler:access(config)
    29.   CustomHandler.super.access(self)
    31.   -- retrieve the apikey from the request querystring
    32.   local key = kong.request.get_query_arg("apikey")
    34.   local credential_cache_key = kong.db.keyauth_credentials:cache_key(key)
    36.   -- We are using cache.get to first check if the apikey has been already
    37.   -- stored into the in-memory cache. If it's not, then we lookup the datastore
    38.   -- and return the credential object. Internally cache.get will save the value
    39.   -- in-memory, and then return the credential.
    40.   local credential, err = kong.cache:get(credential_cache_key, nil,
    41.                                          load_credential, credential_cache_key)
    42.   if err then
    43.     kong.log.err(err)
    44.     return kong.response.exit(500, {
    45.       message = "Unexpected error"
    46.     })
    47.   end
    49.   if not credential then
    50.     -- no credentials in cache nor datastore
    51.       message = "Invalid authentication credentials"
    52.     })
    53.   end
    55.   -- set an upstream header if the credential exists and is valid
    56.   kong.service.request.set_header("X-API-Key", credential.apikey)
    57. end
    60. return CustomHandler

    Note that in the above example, we use various components from the Plugin Development Kit to interact with the request, cache module, or even produce a response from our plugin.

    The cache is used in several places in the . Give that file a look in order to see how an official plugin uses the cache.

    Every time a cached custom entity is updated or deleted in the datastore (i.e. using the Admin API), it creates an inconsistency between the data in the datastore, and the data cached in the Kong nodes’ memory. To avoid this inconsistency, we need to evict the cached entity from the in-memory store and force Kong to request it again from the datastore. We refer to this process as cache invalidation.

    If you want your cached entities to be invalidated upon a CRUD operation rather than having to wait for them to reach their TTL, you have to follow a few steps. This process can be automated for most entities, but manually subscribing to some CRUD events might be required to invalidate some entities with more complex relationships.

    Cache invalidation can be provided out of the box for your entities if you rely on the cache_key property of your entity’s schema. For example, in the following schema:

    We can see that we declare the cache key of this API key entity to be its key attribute. We use key here because it has a unique constraints applied to it. Hence, the attributes added to cache_key should result in a unique combination, so that no two entities could yield the same cache key.

    Adding this value allows you to use the following function on the DAO of that entity:

    1. cache_key = kong.db.<dao>:cache_key(arg1, arg2, arg3, ...)

    Where the arguments must be the attributes specified in your schema’s cache_key property, in the order they were specified. This function then computes a string value cache_key that is ensured to be unique.

    For example, if we were to generate the cache_key of an API key:

    1. local cache_key = kong.db.keyauth_credentials:cache_key("abcd")

    If the cache_key is generated like so and specified in an entity’s schema, cache invalidation will be an automatic process: every CRUD operation that affects this API key will be make Kong generate the affected cache_key, and broadcast it to all of the other nodes on the cluster so they can evict that particular value from their cache, and fetch the fresh value from the datastore on the next request.

    When a parent entity is receiving a CRUD operation (e.g. the Consumer owning this API key, as per our schema’s consumer_id attribute), Kong performs the cache invalidation mechanism for both the parent and the child entity.

    Note: Be aware of the negative caching that Kong provides. In the above example, if there is no API key in the datastore for a given key, the cache module will store the miss just as if it was a hit. This means that a “Create” event (one that would create an API key with this given key) is also propagated by Kong so that all nodes that stored the miss can evict it, and properly fetch the newly created API key from the datastore.

    See the Clustering Guide to ensure that you have properly configured your cluster for such invalidation events.

    In some cases, the cache_key property of an entity’s schema is not flexible enough, and one must manually invalidate its cache. Reasons for this could be that the plugin is not defining a relationship with another entity via the traditional foreign = "parent_entity:parent_attribute" syntax, or because it is not using the cache_key method from its DAO, or even because it is somehow abusing the caching mechanism.

    In those cases, you can manually setup your own subscriber to the same invalidation channels Kong is listening to, and perform your own, custom invalidation work.

    To listen on invalidation channels inside of Kong, implement the following in your plugin’s init_worker handler:

    1. function MyCustomHandler:init_worker()
    2.   -- listen to all CRUD operations made on Consumers
    3.   kong.worker_events.register(function(data)
    5.   end, "crud", "consumers")
    7.   -- or, listen to a specific CRUD operation only
    8.   kong.worker_events.register(function(data)
    9.     kong.log.inspect(data.operation)  -- "update"
    10.     kong.log.inspect(data.old_entity) -- old entity table (only for "update")
    11.     kong.log.inspect(data.entity)     -- new entity table
    12.     kong.log.inspect(data.schema)     -- entity's schema
    13.   end, "crud", "consumers:update")
    14. end

    Once the above listeners are in place for the desired entities, you can perform manual invalidations of any entity that your plugin has cached. For instance:

    1. kong.worker_events.register(function(data)
    2.   if data.operation == "delete" then
    3.     local cache_key = data.entity.id
    4.     kong.cache:invalidate("prefix:" .. cache_key)

    Next: