This page covers how to use a MongoDB Atlas collection as a data source in Rockset. This includes:
For the following steps, you must have access to a MongoDB Atlas account and be able to manage Custom Roles and Database Users within it. If you do not have access, please invite your MongoDB Atlas administrator to Rockset.
The steps below show how to set up a MongoDB Atlas integration using MongoDB SCRAM Authentication mechanism. An integration can provide access to one or more MongoDB collections across different databases in the same MongoDB Atlas cluster. You can use an integration to create Rockset collections that continuously sync data from your MongoDB collections.
collStatsand also enter the names of databases as well as collections for each of these actions or roles. You can update access to databases and collections in Mongo UI at any time without changes required in Rockset integration. The same integration can be used to create more Rockset collections based on permissions.
find- Required for initial collection scan when reading data.
changeStream- Required for retrieving records from MongoDB Atlas Change Streams.
collStats- Required for metadata about MongoDB Atlas collections.
You’ll need to create a MongoDB Atlas user to grant Rockset permissions to access your MongoDB resources.
You’ll need to provide connection string for your MongoDB Atlas cluster for Rockset to connect to it.
mongodb+srv://<username>:<password>@cluster0.mongodb.net/test. You don’t need to replace username and password tags in the connection string.
To ensure connectivity with Atlas, you must whitelist the inbound network access from your application environment to MongoDB Atlas by whitelisting public IP addresses. For more details, refer to MongoDB Atlas documentation on Whitelist Entries. This is the most secure and recommended way to allow Rockset to access your MongoDB cluster. Although, if you choose to skip adding Rockset whitelist entries, make sure you select “Allow Access From Anywhere” which enables access to the cluster from anywhere.
Once you create a collection backed by MongoDB Atlas, Rockset scans the MongoDB collections to continuously ingest and then subsequently uses the MongoDB Change Stream to update collections as new records are added to the MongoDB collection.
In the Rockset Console, you can create a collection from the integration details page of an existing MongoDB integration using “Create collection from Integration” or from Collections > Create Collection page. It is recommended that you add one MongoDB collection into a Rockset collection.
If your MongoDB collection is a capped collection, MongoDB change streams don’t receive deletes for old documents and hence Rockset collection can go out of sync. For this we recommend setting retention on Rockset collection at time of creation.
When a MongoDB Atlas backed collection is created, indexing in Rockset occurs in two stages:
Once a MongoDB backed collection is set up, it will be a replica of the MongoDB collection, up-to-date to within a few seconds.