Data Sources > Google Cloud Storage

Google Cloud Storage

This section covers how you can set up a live integration with your Google Cloud Storage buckets and create Rockset collections from them. Once you create a GCS backed collection, Rockset continuously ingests objects from your target buckets/paths and updates your collection automatically as new objects get added to it.

Setting up

Prior to ingesting from Google Cloud Storage buckets, you must set up and configure a GCP Service Account Integration.

Creating a Google Cloud Storage Sourced Collection

Once you have set up an integration, you can go on to create an Google Cloud Storage sourced collection. When you are creating a collection, you can choose which paths you want to include in your collection by adding multiple sources with distinct path names.

In the Rockset Console, you can create a collection from Workspace > Collections > Create Collection > Add Source > Google Cloud Storage.

Create GCS Collection

Using the CLI, you can run the following:

$ rock create collection my-gcs-collection \
    gs://my-bucket/my-path-1 \
    --integration=my-gcp-integration

Collection "my-gcs-collection" was created successfully.

Note that any of the above operations can also be performed using Rockset Client libraries or REST APIs.