How to Resize Kafka Clusters

Validated on 8 Dec 2022 • Last edited on 18 Dec 2024

Kafka is an open-source distributed event and stream-processing platform built to process demanding real-time data feeds. It is inherently scalable, with high throughput and availability.

You can resize existing Kafka clusters at any time to add more CPUs, RAM, and storage.

Warning
You cannot reduce the amount of added storage in a cluster lower than the amount of storage you are currently using in your cluster.

Resize a Database Cluster Using the CLI

Note

To resize a database cluster using doctl, you need to provide a value for the --size flag, which specifies the cluster’s new configuration (number of CPUs, amount of RAM, and hard disk space). Use the doctl databases options slugs command to get a list of available values.

You can also review a list of available slugs for each engine on its pricing page.

How to Resize a Database Cluster Using the DigitalOcean CLI

  1. Install doctl, the DigitalOcean command-line tool.

  2. Create a personal access token and save it for use with doctl.

  3. Use the token to grant doctl access to your DigitalOcean account.

              doctl auth init

  4. Finally, run doctl databases resize. Basic usage looks like this, but you can read the usage docs for more details:

                doctl databases resize <database-cluster-id> [flags]

    The following example resizes a PostgreSQL or MySQL database to have two nodes, 16 vCPUs, 64 GB of memory, and 2048 GiB of storage space:

                  doctl databases resize ca9f591d-9999-5555-a0ef-1c02d1d1e352 --num-nodes 2 --size db-s-16vcpu-64gb --storage-size-mib 2048000 --wait true

Resize a Database Cluster Using the API

Note

To resize a database cluster using the API, you need to provide a value for the size field, which specifies the cluster’s configuration (number of CPUs, amount of RAM, and hard disk space). Use the /v2/databases/options endpoint to get a list of available values.

You can also review a list of available slugs for each engine on its pricing page.

How to Resize a Database Cluster Using the DigitalOcean API

  1. Create a personal access token and save it for use with the API.

  2. Send a PUT request to https://api.digitalocean.com/v2/databases/{database_cluster_uuid}/resize

    cURL

    Using cURL:

                    curl -X PUT \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $DIGITALOCEAN_TOKEN" \
-d '{"size":"db-s-4vcpu-8gb", "num_nodes":3, "storage_size_mib":163840}' \
"https://api.digitalocean.com/v2/databases/9cc10173-e9ea-4176-9dbc-a4cee4c4ff30/resize"

    Go

    Using Godo, the official DigitalOcean V2 API client for Go: 

                    import (
    "context"
    "github.com/digitalocean/godo"
)

func main() {
    pat := "mytoken"

    client := godo.NewFromToken(pat)
    ctx := context.TODO()

    resizeRequest := &godo.DatabaseResizeRequest{
        SizeSlug: "db-s-4vcpu-8gb",
        NumNodes: 3,
        StorageSizeMib: 163840,
    }
}

    Python

                    import os
from pydo import Client

client = Client(token=os.environ.get("DIGITALOCEAN_TOKEN"))

req = {
  "size": "db-s-4vcpu-8gb",
  "num_nodes": 3,
  "storage_size_mib": 163840
}

update_resp = client.databases.update_cluster_size(database_cluster_uuid="a7a8bas", body=req)

Resize a Kafka Cluster Using the Control Panel

To resize a Kafka database cluster, click the name of the cluster in the control panel to go to its Overview page, then click the Settings tab.

In the Cluster configuration section, click Edit. Select your cluster configuration and CPU option.

Cluster configuration section with additional nodes selected

Then, select your storage size. If you select a Dedicated CPU plan, you can also customize the number of nodes in the cluster to 3, 6, 9, or 15 nodes.

Once you have selected your new configuration and size, click Save to provision the new configuration. The provisioning takes several minutes but the total time depends on the size of the cluster.

Your cluster’s state changes from Active to Resizing until the process is done. You can expect no downtime and do not need to take action.

Control Panel Blog Pricing Careers Terms Privacy Status API Docs Tutorials Support