Asynchronous search

Asynchronous search in OpenSearch lets you send search requests that run in the background. You can monitor the progress of these searches and get back partial results as they become available. After the search finishes, you can save the results to examine at a later time.

Introduced 1.0

To perform an asynchronous search, send requests to , with your query in the request body:

Sample request

  1. POST _plugins/_asynchronous_search/?pretty&size=10&wait_for_completion_timeout=1ms&keep_on_completion=true&request_cache=false
  2. {
  3.   "aggs": {
  4.     "city": {
  5.       "terms": {
  6.         "field": "city",
  7.         "size": 10
  8.       }
  9.     }
  10.   }
  11. }

Sample response

  1. {
  2.   "*id*": "FklfVlU4eFdIUTh1Q1hyM3ZnT19fUVEUd29KLWZYUUI3TzRpdU5wMjRYOHgAAAAAAAAABg==",
  3.   "state": "RUNNING",
  4.   "start_time_in_millis": 1599833301297,
  5.   "expiration_time_in_millis": 1600265301297,
  6.   "response": {
  7.     "took": 15,
  8.     "timed_out": false,
  9.     "terminated_early": false,
  10.     "num_reduce_phases": 4,
  11.     "_shards": {
  12.       "total": 21,
  13.       "successful": 4,
  14.       "skipped": 0,
  15.       "failed": 0
  16.     },
  17.     "hits": {
  18.       "total": {
  19.         "value": 807,
  20.         "relation": "eq"
  21.       },
  22.       "max_score": null,
  23.       "hits": []
  24.     },
  25.     "aggregations": {
  26.       "city": {
  27.         "doc_count_error_upper_bound": 16,
  28.         "buckets": [
  29.           {
  30.             "key": "downsville",
  31.             "doc_count": 1
  32.           },
  33.         ....
  34.         ....
  35.         ....
  36.           {
  37.             "key": "blairstown",
  38.             "doc_count": 1
  39.           }
  41.       }
  42.     }
  43.   }
  44. }

Response parameters

Introduced 1.0

After you submit an asynchronous search request, you can request partial responses with the ID that you see in the asynchronous search response.

Sample response

  1. {
  2.   "id": "Fk9lQk5aWHJIUUltR2xGWnpVcWtFdVEURUN1SWZYUUJBVkFVMEJCTUlZUUoAAAAAAAAAAg==",
  3.   "state": "STORE_RESIDENT",
  4.   "start_time_in_millis": 1599833907465,
  5.   "expiration_time_in_millis": 1600265907465,
  6.   "response": {
  7.     "took": 83,
  8.     "timed_out": false,
  9.     "_shards": {
  10.       "total": 20,
  11.       "successful": 20,
  12.       "skipped": 0,
  13.       "failed": 0
  14.     },
  15.     "hits": {
  16.       "total": {
  17.         "value": 1000,
  18.         "relation": "eq"
  19.       },
  20.       "max_score": 1,
  21.       "hits": [
  22.         {
  23.           "_index": "bank",
  24.           "_id": "1",
  25.           "_score": 1,
  26.           "_source": {
  27.             "email": "amberduke@abc.com",
  28.             "city": "Brogan",
  29.             "state": "IL"
  30.           }
  31.         },
  32.        {....}
  33.       ]
  34.     "aggregations": {
  35.       "city": {
  36.         "doc_count_error_upper_bound": 0,
  37.         "sum_other_doc_count": 997,
  38.         "buckets": [
  40.             "key": "belvoir",
  41.             "doc_count": 2
  42.           },
  43.           {
  44.             "key": "aberdeen",
  45.             "doc_count": 1
  46.           },
  47.           {
  48.             "key": "abiquiu",
  49.             "doc_count": 1
  50.           }
  51.         ]
  52.       }
  53.     }
  54.   }
  55. }

After the response is successfully persisted, you get back the STORE_RESIDENT state in the response.

For asynchronous searches with keep_on_completion as true and a sufficiently long keep_alive time, you can keep polling the IDs until the search finishes. If you don’t want to periodically poll each ID, you can retain the results in your cluster with the keep_alive parameter and come back to it at a later time.

Introduced 1.0

To delete an asynchronous search:

  1. DELETE _plugins/_asynchronous_search/<ID>?pretty
  • If the search is still running, OpenSearch cancels it.
  • If the search is complete, OpenSearch deletes the saved results.

Sample response

You can use the stats API operation to monitor asynchronous searches that are running, completed, and/or persisted.

  1. GET _plugins/_asynchronous_search/stats

Sample response

  1. {
  2.   "_nodes": {
  3.     "total": 8,
  4.     "successful": 8,
  5.     "failed": 0
  6.   },
  7.   "cluster_name": "264071961897:asynchronous-search",
  8.   "nodes": {
  9.     "JKEFl6pdRC-xNkKQauy7Yg": {
  10.       "asynchronous_search_stats": {
  11.         "submitted": 18236,
  12.         "initialized": 112,
  13.         "search_failed": 56,
  14.         "search_completed": 56,
  15.         "rejected": 18124,
  16.         "persist_failed": 0,
  17.         "cancelled": 1,
  18.         "running_current": 399,
  19.         "persisted": 100
  20.       }
  21.     }
  22.   }
  23. }

Response parameters