For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Kimi K3 on 8x MI300X

This 1-Click image serves Kimi K3 (Moonshot AI), a frontier-scale Mixture-of-Experts model, on a single 8x AMD Instinct MI300X node. Inference runs on llama.cpp built with the ROCm/HIP backend (gfx942) against ROCm 7.14, using Unsloth’s Dynamic 2-bit UD-Q2_K_XL quant. That quant is chosen deliberately: at 861 GB it fits in the node’s aggregate 1536 GB of HBM while leaving room for a usable KV cache, where the next quant up (UD-Q4_K_XL, 1509 GB) would load but leave almost none.

Weights are baked in, so it is ready on first boot

All 19 UD-Q2_K_XL shards plus the vision projector are already in the image under /opt/models/Kimi-K3-GGUF. There is no first-boot download and no runtime dependency on Hugging Face being reachable. The build fails loudly if the bake is incomplete, so a published image always carries the full payload. Expect a couple of minutes for llama.cpp to load the weights onto the 8 GPUs after boot; the login banner tells you when the API is ready.

Two ways in: a public URL, or a private SSH tunnel

llama-server exposes an OpenAI-compatible API (chat completions plus a browser chat UI) bound to 127.0.0.1. On boot the droplet also brings up an ephemeral Cloudflare tunnel and prints the resulting public URL in the login banner, along with an API key generated uniquely for that droplet. No key is baked into the image, so nothing secret is published. Inference requests without the key are rejected. If you would rather not expose anything publicly, ignore the tunnel and reach the API over an SSH port-forward instead.

Thinking-only, so budget tokens accordingly

K3 always reasons before answering. Reasoning tokens count against max_tokens, so a small budget can be consumed entirely inside the thinking trace and return empty content. Use 2048 or more, and set depth with reasoning_effort.

Multimodal

The image builds llama-mtmd-cli and loads the MoonViT-V2 vision projector, so image inputs work through the same API.

8x only

The snapshot carries min_disk 2046 GB because of the baked weights, so DigitalOcean will only restore it onto the 8x MI300X SKU. That is also the only configuration with enough VRAM to serve this quant.

Software Included

Deploying this Offering using the Control Panel

Click the Deploy to DigitalOcean button to deploy this offering. If you aren’t logged in, this link will prompt you to log in with your DigitalOcean account.

[](https://cloud.digitalocean.com/gpus/new?appId=f2ef016bb8df96394b712619&image=Kimi K3 2-bit on 8x MI300X UD-Q2_K_XL on Ubuntu 24.04&type=applications)

Getting Started After Deploying Kimi K3 2-bit on 8x MI300X

After the droplet boots, SSH in as root. The login banner (MOTD) is the primary interface: it reports whether the model has finished loading, prints the public Cloudflare tunnel URL, and prints the API key generated for this specific droplet. Nothing is pre-shared, the key is created on first boot.

To use the public URL, open it in a browser for the built-in chat UI, or point any OpenAI client at <tunnel-url>/v1 and send the key as Authorization: Bearer <key>. Requests without the key return 401, so only share the URL with people you want using these GPUs. The hostname changes whenever the tunnel service restarts.

To keep everything private instead, run ssh -N -L 8080:127.0.0.1:8080 root@YOUR_DROPLET_IP and point your client at http://127.0.0.1:8080/v1.

Give requests a generous max_tokens (2048 or more): Kimi K3 spends tokens on hidden reasoning first, so a small budget can return empty content with the budget consumed. The reasoning text comes back separately as reasoning_content. Moonshot’s recommended sampling (temperature 1.0, top_p 0.95) is already set in the systemd unit. Context size defaults to 32768 tokens and is tunable via CTX_SIZE in /etc/default/kimi-server; K3 itself supports up to 1M tokens, and how far this node can be pushed depends on the VRAM left after the weights. Useful commands: systemctl status kimi-server, journalctl -fu kimi-server, systemctl status cloudflared-kimi, amd-smi monitor.