Introduction to Log Receivers

    This tutorial gives a brief introduction about the general steps of adding log receivers in KubeSphere.

    • You need a user granted a role including the permission of Cluster Management. For example, you can log in to the console as directly or create a new role with the permission and assign it to a user.

    • Before adding a log receiver, you need to enable any of the Logging, Events or Auditing components. For more information, see Enable Pluggable Components.

    To add a log receiver:

    1. Log in to the web console of KubeSphere as admin.

    2. Click Platform in the upper-left corner and select Cluster Management.

      Note

      If you have enabled the , you can select a specific cluster.

    4. On the log receivers list page, click Add Log Receiver.

      Note

      • At most one receiver can be added for each receiver type.

    A default Elasticsearch receiver will be added with its service address set to an Elasticsearch cluster if , events, or auditing is enabled in ClusterConfiguration.

    An internal Elasticsearch cluster will be deployed to the Kubernetes cluster if neither externalElasticsearchUrl nor externalElasticsearchPort is specified in when , events, or auditing is enabled. The internal Elasticsearch cluster is for testing and development only. It is recommended that you configure an external Elasticsearch cluster for production.

    Log searching relies on the internal or external Elasticsearch cluster configured.

    If the default Elasticsearch log receiver is deleted, refer to Add Elasticsearch as a Receiver to add a new one.

    Kafka is often used to receive logs and serves as a broker to other processing systems like Spark. demonstrates how to add Kafka to receive Kubernetes logs.

    If you need to output logs to more places other than Elasticsearch or Kafka, you can add Fluentd as a log receiver. Fluentd has numerous output plugins which can forward logs to various destinations such as S3, MongoDB, Cassandra, MySQL, syslog, and Splunk. Add Fluentd as a Receiver demonstrates how to add Fluentd to receive Kubernetes logs.

    Container logs, resource events, and audit logs should be stored in different Elasticsearch indices to be searched in KubeSphere. The index is automatically generated in - format.

    You can turn a log receiver on or off without adding or deleting it. To turn a log receiver on or off:

    1. On the Log Receivers page, click a log receiver and go to the receiver’s detail page.

    2. Click More and select Change Status.

    3. Select Collecting or Disabled to turn the log receiver on or off.

    4. A log receiver’s status will be changed to Disabled if you turn it off, otherwise the status will be Collecting on the Log Receivers page.

    You can edit a log receiver or delete it:

    1. On the Log Receivers page, click a log receiver and go to the receiver’s detail page.

    3. Delete a log receiver by clicking Delete.