How to Connect S3-Compatible Data Sources to Notebooks

Notebooks are a web-based Jupyter IDE with shared persistent storage for long-term development and inter-notebook collaboration, backed by accelerated compute.

When you subscribe to Paperspace’s Pro or Growth plans, you can mount public and private S3-compatible external data sources to your notebook to access data.

DigitalOcean Spaces Object Storage

To mount Spaces Object Storage into Notebooks, use the following steps:

1. In the Paperspace console, click the Data Sources icon in the sidebar, and then click + or the Add a Data Source button.

2. In the Add a Data Source window, select the DigitalOcean Spaces option.

   

3. Enter the following information:

   • Display Name: A name for the data source.

   • Endpoint: The URL for the bucket.

     To find the Spaces endpoint URL, go to the DigitalOcean Control Panel, click your bucket, and click the Settings tab. Copy the value shown in the Origin Endpoint dropdown. For example, https://ps-spaces.nyc3.digitaloceanspaces.com.

   • Access Key ID and Secret Access Key: The access key and secret key for the bucket.

     You must generate the access and secret key and then, add them as project secrets in the Settings tab of your project.

4. Click the Add DigitalOcean Data Source button.

5. Once the Spaces bucket is added, find it in the list of data sources and click the Mount button.

   This creates a bi-directional mount for reading and writing data to the Space.

AWS S3

The dashboard allows you to mount public and private S3 buckets into the notebook to access data that is stored externally for both Pro and Growth plans.

1. Add a new data source and select the Amazon S3 icon.
2. Enter the name of your data source and bucket URL.
3. If the bucket is private add an Access Key ID and Secret Access Key by choosing a Paperspace Secret in the dropdown. To learn about Paperspace secrets, check out the CLI/SDK reference that instructs you on how to create a Paperspace secret.
4. You can mount a data source to your notebook and access it through the data source panel.

S3-Compatible Data Source

To connect to other S3-compatible data sources, you should first add the S3-compatible bucket URL. For example, GCP would look like this: s3://example-bucket-name. Then, open the Advanced Settings and change the default endpoint. For example, you would enter https://storage.googleapis.com for GCP.

Making an S3 Bucket Publicly Accessible

Warning

Anyone on the internet has access your files. Do not enable this if you have sensitive information in your S3 bucket.

To make an AWS S3 bucket publicly accessible without credentials, update two parameters in your bucket settings under the Permissions tab:

1. Uncheck “Block all public access”.

2. Edit ACL to allow Everyone (public access) List Objects and Read Bucket ACL.