Overview

Feature

Inlong sort standalone supports multi tenancy. An inlong sort standalone cluster can host the distribution tasks of different tenants. The distribution tasks are obtained from the inlong manager.
Each distribution task is responsible for distributing multiple data streams to a data store. Users only need to configure on the front page of inlong manager to specify the data streams to be distributed to a specific data store.
For example, the inlong data streams D1 and D2 are distributed to hive cluster H1, D1 is also distributed to elasticsearch cluster E1, and D2 is also distributed to CLS cluster C1. Then the inlong sort standalone cluster will receive three distribution tasks.

• H1 distributes task consumption D1 and D2 to hive cluster H1;
• C1 distributes the task consumption D2 and distributes it to CLS cluster C1.

• Inlong-tubemq
• Apache Pulsar

• Apache hive (currently only supports sequence file format)
• Apache Pulsar

support more types of cache layer message queues

Apache Kafka, etc.

support more kinds of data storage

support more file formats written to Apache hive

Orc file, etc.