ROCm vLLM (Jupyter)

Leverage the power of DigitalOcean GPU droplets powered by AMD to deploy a ROCm + vLLM + JupyterLab IDE with Jupyter AI in minutes! Whether you’re using Jupyter Lab for data science or machine learning tasks, use this marketplace app to set up a powerful experimentation environment without the hassle of manual installation.

Software Included

Creating an App using the Control Panel

Click the Deploy to DigitalOcean button to create a Droplet based on this 1-Click App. If you aren’t logged in, this link will prompt you to log in with your DigitalOcean account.

Creating an App using the API

In addition to creating a Droplet from the ROCm vLLM (Jupyter) 1-Click App using the control panel, you can also use the DigitalOcean API. As an example, to create a 4GB ROCm vLLM (Jupyter) Droplet in the SFO2 region, you can use the following curl command. You need to either save your API access token) to an environment variable or substitute it in the command below.

curl -X POST -H 'Content-Type: application/json' \
         -H 'Authorization: Bearer '$TOKEN'' -d \
        '{"name":"choose_a_name","region":"sfo2","size":"s-2vcpu-4gb","image": "digitaloceanai-rocmvllmjupyter"}' \
        "https://api.digitalocean.com/v2/droplets"

Getting Started After Deploying ROCm vLLM (Jupyter)

JupyterLab

JupyterLab is an interactive development environment for working with notebooks, code, and data. It offers a flexible and powerful interface for data science and scientific computing, allowing you to create and share documents that contain live code, equations, visualizations, and narrative text.

Jupyter AI

Jupyter AI extends JupyterLab with capabilities for integrating machine learning and artificial intelligence workflows. It provides tools and extensions to leverage AI models directly within the Jupyter environment, making it easier to incorporate AI into your data science tasks.

Before You Deploy

Consider the following to ensure you choose the right configuration for your needs:

• Droplet Size: Select a Droplet size with sufficient CPU, memory, and storage based on your data processing requirements.
• Volume: Attach a Volume to your Droplet if you anticipate working with large datasets or need additional storage space.

Quickly Get Started With Your Jupyter Droplet

1. Access the Droplet Console:

   • Navigate to the Droplets page.
   • Locate your newly created Jupyter Droplet and click on its name.
   • Under the “Access” tab, select Console. This will open an in-browser terminal session connected to your droplet.
   • Log in as the root user using the password you set during droplet creation.

2. Login via SSH (Method 1: Using SSH Key):

• If you selected an SSH key during droplet creation, follow these steps:- Open your preferred SSH client (e.g., PuTTY, Terminal).

  • Use the droplet’s public IP address to log in as root:

  ssh root@your_droplet_public_IP

  • Ensure your SSH key is added to the SSH agent, or specify the key file directly:

  ssh -i /path/to/your/private_key root@your_droplet_public_IP

  • Once connected, you will be logged in as the root user without needing a password.

2. Check the Message of the Day (MOTD) for Jupyter Access Token:

   • Upon successful login via console or SSH, the Message of the Day (MOTD) will be displayed.
   • This message includes important information such as the Jupyter Lab access token. Take note of this token as you’ll need it to log in to Jupyter Lab.

5. Access the Jupyter Lab IDE:

   • Open a web browser and navigate to the URL for your Jupyter Lab instance, which will be http://your_droplet_public_IP.
   • This will bring you to the login page for Jupyter Lab.

6. Enter the Jupyter Lab Access Token:

   • On the login page, enter the access token that you noted from the MOTD.
   • This token is required for the first login to ensure secure access to your Jupyter Lab environment.

7. Start Using Jupyter Lab:

   • After entering the access token, you’ll be logged into Jupyter Lab’s web-based IDE.
   • You can now start creating, editing, and running Jupyter notebooks directly within your browser.