How to Set Up Kafka Cluster Monitoring Alerts

Kafka is an open-source distributed event and stream-processing platform built to process demanding real-time data feeds. It is inherently scalable, with high throughput and availability.

You can set alert policies based on your cluster performance metrics. Alert policies notify you when a performance metric rises above or falls below a threshold you set, and again when a metric returns to within the threshold. You can receive notifications via email or Slack.

Create an Alert Policy

To create an alert policy, click the name of your database cluster to go its Overview page, then click the Insights tab.

The Insights tab of a database cluster

Click the Manage alert policies button to open the Manage alert policies window.

Creating an alert policy

Fill out the sections in the Create alert policy tab to set up an alert:

  1. Specify metric & set threshold. You need to specify three things in this section:

    • The performance metric that you want to create an alert for, like CPU usage or disk usage.

    • The numerical threshold above or below which the alert condition triggers, such as above 70% usage or below 50% usage.

    • The length of time a threshold must be breached before the alert notification triggers, like 5 minutes or 1 day. The performance metric you’re alerting on is averaged over this interval to determine if it violates the threshold.

  2. Select Clusters or tags. Choose the clusters or tags that you want the alert policy to apply to. You can also choose All clusters.

  3. Send alerts via. Choose at least one method of notification. You can receive notifications via email (the email on your account and the email of any team members) or Slack.

    Slack notifications require Slack authentication. Click Connect Slack to authenticate. Once you are authenticated to Slack, you can select any Slack team, direct message group, or individual members to which you have access.

  4. Name and create alert policy. Give the alert policy a descriptive name, and then click Create alert policy.

It may take up to a few minutes for the newly-created alert policy to start evaluating data.

Manage Alert Policies and Triggered Alerts

From the Insights tab, click the Manage alert policies button to open the Manage alert policies window, then click on the Existing alert policies tab.

All of your existing alert policies here. If an alert is triggered, it appears in a Triggered Alerts section of this tab, and untriggered alerts are under Alert Policies.

Triggered alert policy

Click an alert policy’s More menu and select Edit to modify the alert policy. You can also delete an alert policy from the More menu, which permanently removes the alert policy without confirmation.