Kafka is an open-source distributed event and stream-processing platform built to process demanding real-time data feeds. It is inherently scalable, with high throughput and availability.
doctl
, you need to provide values for the --engine
, --region
, and --size
flags. Use the doctl databases options engines
, doctl databases options regions
, and doctl databases options slugs
commands, respectively, to get a list of available values.engine
, region
, and size
fields, which specify the database’s engine, its datacenter, and its configuration (number of CPUs, amount of RAM, and hard disk space). Use the /v2/databases/options
endpoint to get a list of available values.You can create a Kafka database cluster at any time from the Create menu by selecting Databases. This takes you to the Create a Database page.
In the Choose a datacenter section, select the datacenter for your database cluster.
This page lists the datacenters in which you currently have the most resources. The number of resources you have in each datacenter is listed to the right as X resources. Hover over this text to see the specific resources you have in that datacenter.
For the best performance, create your database in the same datacenter as your other DigitalOcean resources.
In the Choose a database engine section, choose Kafka.
We currently support Kafka v3.6 and v3.7. You cannot change the Kafka version after creating a cluster.
In the Choose a cluster configuration section, select the machine type and the number and size of the database nodes. If you select the Shared CPU plan, you also need to select a CPU option. If you select a Dedicated CPU plan, you can customize the number of nodes in the cluster to 3, 6, 9, or 15 nodes. Each option lists its combined monthly cost, equivalent hourly cost, and node specifications. For more options, click See all plans.
After creation, you can increase the number or size of database nodes at any time. However, you cannot downsize nodes to have less storage than the current storage the cluster is currently using.
Because Kafka is more resource-intensive than other engines, we recommend using shared CPU plans for development and testing workloads and using dedicated CPU plans for production workloads.
In the Choose a storage size section, you can increase your storage by clicking the up arrow or entering your desired storage amount, in increments of 10 GB, up to the maximum listed in the storage range. Additional storage you add to the cluster costs $0.20 per GB per month.
You can increase your storage at any time. You cannot reduce the amount of storage in a cluster below the amount of storage you are currently using in your cluster.
In the last section, Finalize and Create, choose the name for the cluster, the project to add it to, and any tags you want to use.
There are three sub-headers in this section:
Choose a name: You can leave the automatically-generated name for the database or choose a custom name. Names must be unique, be between 3 and 63 characters long, and only contain alphanumeric characters, dashes, and periods.
Select a project: You can leave the default project or choose another one.
Tags: You can add a tag by typing it into the text box and pressing enter. Tags can only contain letters, numbers, colons, dashes, and underscores.
After creation, you can always edit the database’s tags or move it to another project; however, its name cannot be changed.
When you’re ready, click the Create a Database Cluster button.
Clusters typically take five minutes or more to provision, but you can complete important configuration tasks such as restricting inbound connections while you wait.