elasticsearch


    sources:
      - type: elasticsearch
        name: elastic
        hosts:
          - "localhost:9200"
          - "localhost:9201"
        indices: ["blog*"]
        username: "bob"
        password: "bob"
        schema: ""
        sniff: false
        gzip: true
        includeFields: # pull selected field
          - Title
          - Author
        excludeFields: # exclude selected field
          - Content
        query: | # elastic query phrases
          {
            "match": {"Title": "bob"}
          }
        size: 10 # data size per fetch
        interval: 30s # pull data frequency
        timeout: 5s # pull timeout
        db: 
          flushTimeout: 2s # persistent the elastic pull location frequency
          cleanInactiveTimeout: 24h # delete the db record after the time
          cleanScanInterval: 1h # check the expired db record frequency

indices

`字段`	`类型`	`是否必填`	`默认值`	`含义`
index	string数组	必填		查询elasticsearch的index名称

username

`字段`	`类型`	`是否必填`	`默认值`	`含义`
username	string	非必填		消费elasticsearch的用户名

password

`字段`	`类型`	`是否必填`	`默认值`	`含义`
password	string	必填		消费elasticsearch的密码

gzip

`字段`	`类型`	`是否必填`	`默认值`	`含义`
gzip	bool	非必填	false	是否开启gzip压缩

includeFields

`字段`	`类型`	`是否必填`		`含义`
includeFields	string数组	非必填		只返回指定的_source字段

excludeFields

`字段`	`类型`	`是否必填`	`默认值`	`含义`
excludeFields	string数组	非必填		排除指定的_source字段

size

`字段`	`类型`	`是否必填`	`默认值`	`含义`
size	int	非必填	100	每次请求得到hits返回的个数

interval

`字段`	`类型`	`是否必填`	`默认值`	`含义`
interval	time.Duration	非必填	30s	定时请求elasticsearch的时间间隔

timeout

`字段`	`类型`	`是否必填`	`默认值`	`含义`
timeout	time.Duration	非必填	5s	请求的超时时间