We have removed the daily allowance and rate limits for uploads to Spaces in AMS3
We have completed the AMS3 expansion and reenabled the creation of new Spaces in AMS3. The allowance and rate limits on uploads to Spaces in AMS3 will stay in place to ensure high performance.
Our storage engineering team has finished building and deploying the AMS3 expansion. The majority of the work is complete, and our engineers are now completing pre-release testing.
We expect to finish our final tests soon, at which point we will remove restrictions on Spaces in AMS3. We will continue to update this page with progress.
Due to unforeseen delays, we were not able complete AMS3’s capacity expansion by November as originally planned.
Our revised timeline for completing the region’s new cluster deployment is February 2020, at which point we’ll lift the temporary limits on upload allowance and bandwidth rates and reenable the creation of new Spaces in AMS3.
Starting in September, we are applying an allowance and bandwidth rate limit on uploads to existing Spaces in AMS3:
Uploading up to 150 GB of data in 24 hours is unthrottled.
If uploads to a bucket exceed 150 GB of data in 24 hours, that bucket’s upload bandwidth is gradually throttled until its 24-hour accumulated uploads return to 150 GB or less.
If uploads to a bucket exceed 270 GB of data in 24 hours, that bucket’s upload bandwidth is set to 1MB/s for the next 24 hours.
The upload allowance and rate limiting will remain in place until the datacenter expansion later this year.
Over the last few weeks, some customers have reported intermittent performance issues with Spaces in AMS3. Our engineering team identified two primary causes: the way the cluster manages available system memory and the overall load on the cluster.
We’re taking the following actions:
On 4 June, we deployed a fix that resolves the memory management issues.
Starting at 9 PM UTC on 19 July, we will be performing maintenance on the cluster to alleviate the load issues. We will be temporarily disabling the creation of new Spaces in AMS3 until this maintenance is complete.
During the maintenance, existing Spaces in AMS3 will continue to serve content as normal, and you can still create new Spaces in any other Spaces-compatible datacenter.
Starting at 9 PM UTC on 19 July, we may also selectively throttle some operations from Spaces in AMS3 for a limited number of customer accounts.
This will not affect most accounts, and we will reach out beforehand if we identify your Space as a candidate.
In November of this year, we plan to expand AMS3’s capacity as a long-term solution to handle the cluster’s load.
We will continue to update this page with our progress on the AMS3 maintenance and expansion.