Term-level queries

The following table describes the differences between them:

OpenSearch uses a probabilistic ranking framework called Okapi BM25 to calculate relevance scores. To learn more about Okapi BM25, see Wikipedia.

Assume that you have the complete works of Shakespeare indexed in an OpenSearch cluster. We use a term-level query to search for the phrase “To be, or not to be” in the field:

Sample response

  1. {
  2.   "took" : 3,
  3.   "timed_out" : false,
  4.   "_shards" : {
  5.     "total" : 1,
  6.     "successful" : 1,
  7.     "skipped" : 0,
  8.     "failed" : 0
  9.   },
  10.   "hits" : {
  11.     "total" : {
  12.       "value" : 0,
  13.       "relation" : "eq"
  14.     },
  15.     "max_score" : null,
  16.     "hits" : [ ]
  17.   }
  18. }

We don’t get back any matches (hits). This is because the term “To be, or not to be” is searched literally in the inverted index, where only the analyzed values of the text fields are stored. Term-level queries aren’t suited for searching on analyzed text fields because they often yield unexpected results. When working with text data, use term-level queries only for fields mapped as keyword only.

Using a full-text query:

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "match": {
  5.       "text_entry": "To be, or not to be"
  6.     }
  7.   }
  8. }

The search query “To be, or not to be” is analyzed and tokenized into an array of tokens just like the text_entry field of the documents. The full-text query performs an intersection of tokens between our search query and the text_entry fields for all the documents, and then sorts the results by relevance scores:

Sample response

  1. {
  2.   "took" : 19,
  3.   "timed_out" : false,
  4.   "_shards" : {
  5.     "total" : 1,
  6.     "successful" : 1,
  7.     "skipped" : 0,
  8.     "failed" : 0
  9.   },
  10.   "hits" : {
  11.     "total" : {
  12.       "value" : 10000,
  13.       "relation" : "gte"
  14.     },
  15.     "max_score" : 17.419369,
  16.     "hits" : [
  17.       {
  18.         "_index" : "shakespeare",
  19.         "_id" : "34229",
  20.         "_score" : 17.419369,
  21.         "_source" : {
  22.           "type" : "line",
  23.           "line_id" : 34230,
  24.           "play_name" : "Hamlet",
  25.           "speech_number" : 19,
  26.           "line_number" : "3.1.64",
  27.           "speaker" : "HAMLET",
  28.           "text_entry" : "To be, or not to be: that is the question:"
  29.       },
  30.       {
  31.         "_index" : "shakespeare",
  32.         "_id" : "109930",
  33.         "_score" : 14.883024,
  34.         "_source" : {
  35.           "type" : "line",
  36.           "line_id" : 109931,
  37.           "play_name" : "A Winters Tale",
  38.           "speech_number" : 23,
  39.           "line_number" : "4.4.153",
  40.           "speaker" : "PERDITA",
  42.         }
  43.       },
  44.       {
  45.         "_index" : "shakespeare",
  46.         "_id" : "103117",
  47.         "_score" : 14.782743,
  48.         "_source" : {
  49.           "type" : "line",
  50.           "line_id" : 103118,
  51.           "play_name" : "Twelfth Night",
  52.           "speech_number" : 53,
  53.           "line_number" : "1.3.95",
  54.           "speaker" : "SIR ANDREW",
  55.           "text_entry" : "will not be seen; or if she be, its four to one"
  56.         }
  57.       }
  58.     ]
  59.   }
  60. }
  61. ...

For a list of all full-text queries, see .

If you want to query for an exact term like “HAMLET” in the speaker field and don’t need the results to be sorted by relevance scores, a term-level query is more efficient:

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "term": {
  5.       "speaker": "HAMLET"
  6.     }
  7.   }
  8. }

Sample response

The term-level queries are exact matches. So, if you search for “Hamlet”, you don’t get back any matches, because “HAMLET” is a keyword field and is stored in OpenSearch literally and not in an analyzed form. The search query “HAMLET” is also searched literally. So, to get a match on this field, we need to enter the exact same characters.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "term": {
  5.       "line_id": {
  6.         "value": "61809"
  7.       }
  8.     }
  9.   }
  10. }

Terms

Use the terms query to search for multiple terms in the same field.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "terms": {
  5.       "line_id": [
  6.         "61809",
  7.         "61810"
  8.       ]
  9.     }
  10.   }
  11. }

You get back documents that match any of the terms.

Use the ids query to search for one or more document ID values.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.       "values": [
  5.         34229,
  6.         91296
  7.       ]
  8.     }
  9.   }
  10. }

Range

Use the range query to search for a range of values in a field.

To search for documents where the line_id value is >= 10 and <= 20:

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "range": {
  5.       "line_id": {
  7.         "lte": 20
  8.       }
  9.     }
  10.   }
  11. }

Assume that you have a products index and you want to find all the products that were added in the year 2019:

Specify relative dates by using basic math expressions.

To subtract 1 year and 1 day from the specified date:

  1. GET products/_search
  2. {
  3.   "query": {
  4.     "range": {
  5.       "created": {
  6.         "gte": "2019/01/01||-1y-1d"
  7.       }
  8.     }
  9.   }
  10. }

The first date that we specify is the anchor date or the starting point for the date math. Add two trailing pipe symbols. You could then add one day (+1d) or subtract two weeks (-2w). This math expression is relative to the anchor date that you specify.

To find products added in the last year and rounded off by month:

  1. GET products/_search
  2. {
  3.   "query": {
  4.     "range": {
  5.       "created": {
  6.         "gte": "now-1y/M"
  7.       }
  8.     }
  9.   }
  10. }

The keyword now refers to the current date and time.

Use the prefix query to search for terms that begin with a specific prefix.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "prefix": {
  5.       "speaker": "KING"
  6.     }
  7.   }
  8. }

Exists

Use the exists query to search for documents that contain a specific field.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "exists": {
  5.       "field": "speaker"
  6.     }
  7.   }
  8. }

Use wildcard queries to search for terms that match a wildcard pattern.

To search for terms that start with H and end with Y:

If we change * to ?, we get no matches, because ? refers to a single character.

Wildcard queries tend to be slow because they need to iterate over a lot of terms. Avoid placing wildcard characters at the beginning of a query because it could be a very expensive operation in terms of both resources and time.

Regex

Use the regexp query to search for terms that match a regular expression.

  1. GET shakespeare/_search
  2. {
  3.   "query": {
  4.     "regexp": {
  5.       "play_name": "[a-zA-Z]amlet"
  6.     }
  7.   }
  8. }

A few important notes:

  • Regular expressions are applied to the terms in the field (i.e. tokens), not the entire field.
  • Regular expressions use the Lucene syntax, which differs from more standardized implementations. Test thoroughly to ensure that you receive the results you expect. To learn more, see .
  • regexp queries can be expensive operations and require the search.allow_expensive_queries setting to be set to true. Before making frequent queries, test their impact on cluster performance and examine alternative queries for achieving similar results.