This section covers how you can set up a live integration with your Google Cloud Storage buckets and create Rockset collections from them. Once you create a GCS backed collection, Rockset continuously ingests objects from your target buckets/paths and updates your collection automatically as new objects get added to it.
Prior to ingesting from Google Cloud Storage buckets, you must set up and configure a GCP Service Account Integration.
Once you have set up an integration, you can go on to create an Google Cloud Storage sourced collection. When you are creating a collection, you can choose which paths you want to include in your collection by adding multiple sources with distinct path names.
In the Rockset Console, you can create a collection from Workspace > Collections > Create Collection > Add Source > Google Cloud Storage.
Using the CLI, you can run the following:
$ rock create collection my-gcs-collection \ gs://my-bucket/my-path-1 \ --integration=my-gcp-integration Collection "my-gcs-collection" was created successfully.
Note that any of the above operations can also be performed using Rockset Client libraries or REST APIs.