Data Sources > MongoDB Atlas

MongoDB Atlas

This page covers how to use a MongoDB Atlas collection as a data source in Rockset. This includes:

  • Creating a MongoDB integration to securely connect collections in your MongoDB account with Rockset.
  • Create a collection which continuously syncs your data from a MongoDB collection into Rockset in real-time.

For the following steps, you must have access to a MongoDB Atlas account and be able to manage Custom Roles and Database Users within it. If you do not have access, please invite your MongoDB Atlas administrator to Rockset.

Create a MongoDB Atlas Integration

The steps below show how to set up a MongoDB Atlas integration using MongoDB SCRAM Authentication mechanism. An integration can provide access to one or more MongoDB collections across different databases in the same MongoDB Atlas cluster. You can use an integration to create Rockset collections that continuously sync data from your MongoDB collections.

Step 1: Configure MongoDB Atlas Custom Role

  1. Navigate to “Database Access” (from left navigation) in your MongoDB Atlas account for the cluster you want to connect to Rockset.
  2. Create a new custom role by navigating to “Custom Roles” and clicking "Add New Custom Role". If you already have a role set up for Rockset, you may update that existing role. For more details, refer to MongoDB Atlas documentation on Custom Roles.

MongoDB Custom Roles

  1. Set up read-only access to your MongoDB collection. Add the following Actions or roles: find, changeStream, collStats and also enter the names of databases as well as collections for each of these actions or roles. You can update access to databases and collections in Mongo UI at any time without changes required in Rockset integration. The same integration can be used to create more Rockset collections based on permissions.

MongoDB Actions or Roles

  1. Save the newly create or updated custom role and give it a descriptive name. You will attach this custom role to a new or existing Atlas user.

Why these permissions?

  • find - Required for initial collection scan when reading data.
  • changeStream - Required for retrieving records from MongoDB Atlas Change Streams.
  • collStats - Required for metadata about MongoDB Atlas collections.

Step 2: Configure MongoDB Atlas User

You’ll need to create a MongoDB Atlas user to grant Rockset permissions to access your MongoDB resources.

  1. Navigate to “Database Access” (from left navigation) in MongoDB Atlas UI.
  2. Set up a new user by navigating to “Database Users” and clicking "Add New Database User". Note: If you already have a user for Rockset set up, you may re-use it or update the custom role directly. For more details, refer to MongoDB Atlas documentation on Database Users.

MongoDB Database Users

  1. Using SCRAM Password authentication enter a username and password for the database user and select the custom role created in Step 1 under "Database User Privileges".

MongoDB Add New Database User

  1. Finish by clicking “Add User” and record both username and password in the Rockset Console within a new MongoDB integration. Note that if you change the password later, you will need to drop and recreate the integration in Rockset.

Step 3: Access Connection String

You’ll need to provide connection string for your MongoDB Atlas cluster for Rockset to connect to it.

  1. Navigate to the cluster (from left navigation) you want to connect to Rockset and click on "Connect".

MongoDB connect cluster

  1. Select “Connect your application” for connection method.

MongoDB connect application

  1. Copy the “Connection String” and record it in the Rockset Console for the integration. Connection String looks like mongodb+srv://<username>:<password>@cluster0.mongodb.net/test. You don’t need to replace username and password tags in the connection string.

MongoDB connection string

Step 4: Whitelist Rockset IPs

To ensure connectivity with Atlas, you must whitelist the inbound network access from your application environment to MongoDB Atlas by whitelisting public IP addresses. For more details, refer to MongoDB Atlas documentation on Whitelist Entries. This is the most secure and recommended way to allow Rockset to access your MongoDB cluster. Although, if you choose to skip adding Rockset whitelist entries, make sure you select “Allow Access From Anywhere” which enables access to the cluster from anywhere.

  1. Navigate to “Network Access” (from left navigation) in MongoDB Atlas UI.
  2. Click on “Add IP Address” and create three whitelist entries for the following public IP addresses of Rockset service:
    • 50.112.53.111/32
    • 35.160.119.116/32
    • 54.214.107.166/32

MongoDB whitelist entry

Create a Collection

Once you create a collection backed by MongoDB Atlas, Rockset scans the MongoDB collections to continuously ingest and then subsequently uses the MongoDB Change Stream to update collections as new records are added to the MongoDB collection.

In the Rockset Console, you can create a collection from the integration details page of an existing MongoDB integration using “Create collection from Integration” or from Collections > Create Collection page. It is recommended that you add one MongoDB collection into a Rockset collection.

If your MongoDB collection is a capped collection, MongoDB change streams don’t receive deletes for old documents and hence Rockset collection can go out of sync. For this we recommend setting retention on Rockset collection at time of creation.

Create Collection

How it works

When a MongoDB Atlas backed collection is created, indexing in Rockset occurs in two stages:

  1. A one-time full scan of the MongoDB collection in which all records are indexed and stored in the Rockset collection.
  2. Following that, continuous monitoring and sync of changes from the MongoDB collection (inserts, deletes and updates) to the Rockset collection in real-time using MongoDB Change Streams.

Once a MongoDB backed collection is set up, it will be a replica of the MongoDB collection, up-to-date to within a few seconds.

Join us on Slack!
Building on Rockset? Come chat with us!