Before productionalizing applications powered by Rockset, it's important to consider a range of crucial factors. We've created a comprehensive checklist that Rockset users can go through to ensure their setup and architecture aligns with Rockset's best practices for production workloads.

General

• Separate development and production environments by using distinct Workspaces or different Rockset accounts (examples of how to do this here).
• Utilize Terraform (and/or Rockset API) for deployment.

Integrations

• Review data source best practices for used Integrations.
• Review and/or clean unused integrations.

Data Management

• Use Workspaces to segregate data access and keep logical separation between different use cases.
  • Avoid using the commons workspace.
• Use Collection Aliases in queries and Query Lambdas, and avoid referencing Collections directly.
• Add clustering to collections, if relevant.
• Add data retention, if relevant.
• Review Ingest Transformations and remove unnecessary fields.
• Use special fields like _id and _op to ensure proper handling of data changes, if relevant.
• Review and/or clean unused workspaces, collections, Views, and aliases.

Compute Resources

• Use Compute-Compute Separation to have isolated, dedicated Virtual Instances for each use case.
• Use continuous mounts on Query VIs, depending on use case.
• Set-up auto-suspend for Query VIs.
• Set-up auto-scaling for the Ingest VI.
• Review Virtual instances classes and sizes to target price/performance for each use case.
• Use dedicated streaming ingest for latency sensitive workloads.
• Use Micro-Batching for workloads that are not sensitive to ingest latency.
• Set-up your Default Query Virtual Instance to execute all queries and query lambdas that don't target a specific VI.
• Review advanced VI configurations, such as concurrency limits (Concurrent Query Execution Limit - CQEL).

Query and API Access

• Perform performance testing to get baseline performance for single queries and multiple concurrent queries.
• Use asynchronous option for long running/batch queries.
• Perform load testing on your system before going to production.
• Learn common query troubleshooting techniques.
• Ensure application logic (if applicable) has client timeout set and retry mechanisms in place with exponential back-off and maximum number of retries (review these configurations in the Execute SQL query endpoint and Execute Query Lambda endpoint of the Rockset API).
• Review and/or clean unused Query Lambdas.

Monitoring

• Enable Metrics Endpoint and integrate with Datadog, Grafana, or similar visualization tools.
• Set-up a Query Log collection, preferably in a separate workspace like logs.
• Set-up an Ingest Log collection, preferably in a separate workspace like logs.

Security and Access Control

• Set-up SSO for Rockset.
  • Enforce SSO-only.
• Review and/or clean RBAC roles and user assignment to ensure least privilege access and data segregation.
• Use customer managed encryption keys, if relevant.
• Review and/or clean user list.
• Review and/or clean API keys.
• Set-up a rotation mechanism for API keys.

Networking

• Configure IP allowlisting.
• Configure and enable PrivateLink.

High Availability/Disaster Recovery

• Use single or multi-region deployment to achieve higher levels of availability, if applicable.
• Use collection snapshots to keep a copy of data, if relevant.
• Build an emergency query re-direct to a new VI in case the current VI becomes overwhelmed. Test by building an internal failover and timing how long it takes to switch 100% of queries to a new VI.

Support

• Know how to reach Rockset Support (via chat support and/or support tickets).
• Enable Rockset Support Access by navigating to the Security tab of the Rockset Console.
• Subscribe to the Rockset status page.
• If you have a support contract, run a mock fire drill. File a SEV1 support ticket and time how quickly support acknowledges the ticket.

Billing and Cost Management

• Learn Rockset's billing structure.
• Learn cost management best practices:
  • Review and optimize collection sizes by utilizing storage compression, implementing data retention, removing unnecessary fields, and/or removing fields from the search index.
  • Use Rollup collections to reduce collection size, if applicable.
  • Set auto-suspend for Query VIs to save cost when not used.
  • Set auto-scaling for the Ingest VI to save cost under lower usage.

Appendix

Rockset Asset Topology

Mental model on how different assets/components are organized within a Rockset organization:

Rockset Asset Hierarchy

Hierarchical representation of assets/components within Rockset:

Rockset Environment Separation

Separate environments (dev, test, production, etc.) by having multiple Rockset organizations (for example — one for dev/test, one for production) and using workspaces within them to separate use cases. This ensures complete separation of resources and is the recommended way of separating environments.

Separate environments by having a single Rockset organization with workspaces for each environment and use case (for example — dev/test for use case 1, prod for use case 1, dev/test for use case 2, prod for use case 2).