DigitalOcean Droplets are Linux-based virtual machines (VMs) that run on top of virtualized hardware. Each Droplet you create is a new server you can use, either standalone or as part of a larger, cloud-based infrastructure.
Live migration moves a running Droplet from one physical machine to another in its datacenter without requiring a reboot. We use live migration to minimize disruption and downtime during events like normal infrastructure and network maintenance, software upgrades, and hardware failures.
Live migrating a Droplet follows this general process:
When the migration is triggered, the system uses a scoring algorithm to find an available physical host in the same datacenter based on the Droplet’s size (CPU, memory, disk).
The system prepares the target hardware and begins copying the Droplet to its new location. This is more complicated than it may seem; because the Droplet is still running, its disk and memory may change during the migration.
To account for this, the system monitors the disk to track modified blocks. Once the initial copy is complete, it re-copies the modified blocks and repeats the process until all blocks are moved, at which point new writes will take place on the new hardware. If a Droplet is writing rapidly to disk during a migration, the system applies a temporary throttle when there are only a smaller number of modified blocks remaining so the migration can complete.
A very similar process occurs for memory pages, though automatic throttling in this case is significantly shorter than with disk throttling, and should be unnoticeable.
The system updates the Droplet’s records to point to its new location.
Once the migration is complete, we can access the original hardware for maintenance or other events without any effect on the Droplets that were originally running there.
Overall, live migrations have minimal impact on Droplets and users. They don’t change a Droplet’s data or configuration, like software or network settings, so IP addresses and internal data will remain the same. Because migrations are run at the block-level, there is also no risk of file loss or corruption. In the final stages of the transfer, there may be a momentary slowdown in the read/write performance of a Droplet’s disk.
Users typically don’t have to do anything before or after a live migration, but there are some exceptions:
If you’re concerned about disk throttling or prefer offline migration, you can shut down the Droplet before its migration window. Shutting down or otherwise reducing the load of a Droplet during its migration window isn’t necessary, but it will not negatively impact the migration.
If you have Droplets running workloads leveraging nested virtualization, you will need to restart any nested VMs after a live migration.
If a live migration fails, the source Droplets are unaffected. We will either attempt the live migration again or send a notification that we need to briefly shut down the Droplet to migrate it.
To see if a Droplet is scheduled for a live migration, you can send an HTTP request from the Droplet to the following endpoint: http://169.254.169.254/v1/maintenance_event
You can test this by logging into the Droplet and sending a cURL request from the command line:
curl http://169.254.169.254/v1/maintenance_event
The endpoint returns the string live_migrate at least 10 minutes before a scheduled migration. If the request returns a blank string, we have not scheduled the Droplet for a migration.
To further minimize live migration disruptions for high workload Droplets, you can use the response from this endpoint to develop automation that automatically shifts workload to other Droplets in your fleet when the Droplet detects a migration. If the Droplet cannot act on a live migration notice in a meaningful manner, such as shifting workloads to other Droplets, it is not necessary to poll the endpoint.