Data Source 和 Sink 的容错保证
Sink | Guarantees | Notes |
---|---|---|
HDFS BucketingSink | exactly once | Implementation depends on Hadoop version |
Elasticsearch | at least once | |
Kafka producer | at least once/ exactly once | exactly once with transactional producers (v 0.11+) |
Cassandra sink | at least once / exactly once | exactly once only for idempotent updates |
AWS Kinesis Streams | at least once | |
File sinks | at least once | |
Socket sinks | at least once | |
Standard output | at least once | |
Redis sink | at least once |