Examples of s3cmd 2.x Usage with DigitalOcean Spaces

Spaces Object Storage is an S3-compatible object storage service that lets you store and serve large amounts of data. Each Space is a bucket for you to store and serve files. The built-in Spaces CDN minimizes page load times and improves performance.

s3cmd is a popular cross-platform command-line tool for managing S3 and S3-compatible object stores. Once you’ve set up s3cmd, you can use it to manage your Spaces buckets and files.

If you’re using an alternative configuration file, you must explicitly provide it at the end of each command by appending -c ~/path/to/config/file.

Create Buckets

Use the command mb, short for “make bucket”, to create a new bucket:

s3cmd mb s3://spacename s3://secondspace

Object stores use a flat namespace and don’t have directories. What look like directories in the path of an object are actually part of the object’s file name. Graphical interfaces mimic the more familiar experience of directories for their users by creating an empty file for each element that doesn’t already exist.

From the command line, there’s no need to create empty files. Instead, type the path where you want the file to be as part of the put command.

List Buckets and Contents

List all existing buckets

s3cmd ls

List all the contents of buckets recursively

For one or more specific buckets:

s3cmd ls s3://spacename s3://secondspace

List all content in all buckets

s3cmd la --recursive

Upload files to a bucket

Use the put command to copy files from your local machine to a bucket. In all of these commands, you must include the trailing slash.

Put one file

When you include the trailing slash, as in the example below, the original file name is appended. If you omit the slash, then the file is copied to the bucket with the new name, path.

s3cmd put file.txt s3://spacename/path/

Put a file under a new name

You can change the name of a file as you put it in a bucket by typing the new name at the end of the path as follows:

s3cmd put file.txt s3://spacename/newname.txt

Put multiple files

s3cmd put file1.txt file2.txt path/to/file3.txt s3://spacename/path/

Put all files in your current directory

Using the * with the put command copies everything in the current working directory, recursively, into your bucket:

s3cmd put * s3://spacename/path/ --recursive

You can set public permissions for all files at once by adding --acl-public, and you can similarly set metadata with --add-header (like --add-header=Cache-Control:max-age=86400):

s3cmd put * s3://yourfolder --acl-public --add-header=Cache-Control:max-age=86400 --recursive

Save Files to your local computer

The command get copies files from a bucket to your local computer.

Get one file

s3cmd get s3://spacename/path/to/file.txt

Get all the files in a directory

To get multiple files, the s3 address must end with a trailing slash, and the command requires the --recursive flag.

s3cmd get s3://spacename/path/ --recursive

Save a file under a new name

Like the put command, the command get command allows you to give the file a different name.

s3cmd get s3://spacename/file.txt newfilename.txt

Set Directory Listings on a Space

s3cmd only provides output when the command you issue actually changes access levels. For example, when you change the ACL from private to public, you’ll see output like s3://spacename/: ACL set to Public. If the ACL is already public, s3cmd returns silently to the command prompt.

Enable directory listings

s3cmd setacl s3://spacename/ --acl-public

Disable directory listings

s3cmd setacl s3://spacename/ --acl-private

Set Permissions on Files

Using the setacl command, files can be made private so that only someone connecting with a valid key pair is able to read the file, or public so that anyone can read the file with either an S3 compatible client or via HTTPS.

s3cmd only provides output when the command you issue changes the access. For example, when you change the ACL from private to public, you’ll see output like s3://spacename/test.txt: ACL set to Public [1 of 1]. If the ACL is already public, s3cmd returns silently to the command prompt.

Make a file public

s3cmd setacl s3://spacename/file.txt --acl-public

Make all the files at a path public recursively

Use the --recursive flag to make multiple files public recursively:

s3cmd setacl s3://spacename/path/to/files/ --acl-public --recursive

Make a file private

s3cmd setacl s3://spacename/file.txt --acl-private

Make all the files at a path private recursively

Use the --recursive flag to make multiple files private recursively:

s3cmd setacl s3://spacename/path/to/files/ --acl-private --recursive

Delete Buckets and Files

The s3cmd commands del and rm are identical and can be used interchangeably.

Destroy a Bucket

Use rb (short for “remove bucket”) to destroy an empty bucket. If you wish to remove the bucket and all its contents, append --recursive. This permanently removes all the files.

s3cmd rb s3://spacename

Delete a file

s3cmd rm s3://spacename/name/of/file

Delete all files in a bucket

Use rm or del with both the --recursive and --force flags to remove all the files in a bucket but not the bucket itself.

s3cmd rm s3://spacename/  --recursive --force

Encrypt Files

Adding the -e or --encrypt flag when you put a file in a bucket with s3cmd encrypts the file to protect it from being read on the server or in transit. If you download the file using s3cmd and the same configuration file, s3cmd automatically uses the password to decrypt it. Other users would need to use gpg -d file.txt to decrypt it and would have to enter the password you supplied.

s3cmd put s3://path/to/file.txt -e

s3cmd allows you to use only one password, so it’s suitable for you and for other users with full administrative access. You can learn more about GPG’s symmetric encryption in The GNU Privacy Handbook.

More Information

For a comprehensive guide to s3cmd usage, see the s3cmd usage guide or access the help file from the command line with s3cmd --help.