Before productionalizing applications powered by Rockset, it's important to consider a range of crucial factors. We've created a comprehensive checklist that Rockset users can go through to ensure their setup and architecture aligns with Rockset's best practices for production workloads.

General

• Separate development and production environments by using distinct workspaces or different Rockset accounts (examples of how to do this here).
• Utilize Terraform (and/or Rockset API) for deployment.

Integrations

• Review data source best practices for used integrations.
• Review and/or clean unused integrations.

Data management

• Use workspaces to segregate data access and keep logical separation between different use cases.
  • Avoid using the commons workspace.
• Use aliases in queries and query lambdas, and avoid referencing collections directly.
• Add clustering to collections, if relevant.
• Add data retention, if relevant.
• Review ingest transformations and remove unnecessary fields.
• Use special fields like _id and _op to ensure proper handling of data changes, if relevant.
• Review and/or clean unused workspaces, collections, views, and aliases.

Compute resources

• Use compute-compute separation to have isolated, dedicated VIs for each use case.
• Use continuous mounts on secondary VIs, depending on use case.
• Set-up auto-suspend for secondary VIs.
• Set-up auto-scaling for main/ingest VI.
• Review Virtual instances sizes to target price/performance for each use case.
• Use dedicated streaming ingest for latency sensitive workloads.

Query and API access

• Perform performance testing to get baseline performance for single queries and multiple concurrent queries.
• Use asynchronous option for long running / batch queries.
• Learn common query troubleshooting techniques.
• Ensure application logic (if applicable) has client timeout set and retry mechanisms in place with back-off and maximum number of retries (review these configurations in the Execute SQL query endpoint and Execute Query Lambda endpoint).
• Review and/or clean unused query lambdas.

Monitoring

• Enable Metrics endpoint and integrate with Datadog, Grafana, or similar visualization tools.
• Set-up a Query Log collection, preferably in a separate workspace like logs.

Security and access control

• Set-up SSO for Rockset.
  • Enforce SSO-only.
• Review and/or clean RBAC roles and user assignment to ensure least privilege access and data segregation.
• Use customer managed encryption keys, if relevant.
• Review and/or clean user list.
• Review and/or clean API keys.
• Set-up a rotation mechanism for API keys.

Networking

• Configure IP allowlisting.
• Configure and enable PrivateLink.

High availability / Disaster recovery

• Use single or multi-region deployment to achieve higher levels of availability, if applicable.
• Use collection snapshots to keep a copy of data, if relevant.
• Build an emergency query re-direct to a new VI in case the current VI becomes overwhelmed. Test by building an internal failover and timing how long it takes to switch 100% of queries to a new VI.

Support

• Know how to reach Rockset support (via chat support and/or support tickets).
• Enable Rockset support access by going to Settings > Security.
• Subscribe to Rockset status page: https://status.rockset.com/
• If you have a support contract, run a mock fire drill. File a SEV1 support ticket and time how quickly support acknowledges the ticket.

Billing and cost management

• Learn Rockset's billing structure.
• Learn cost management best practices:
  • Review and optimize collection sizes by utilizing storage compression, implementing data retention, removing unnecessary fields, and/or removing fields from the search index.
  • Use roll-up collections to reduce collection size, if applicable.
  • Set auto-suspend for secondary VI to save cost when not used.
  • Set auto-scaling for main/ingest VI to save cost under lower usage.

Appendix

Rockset Asset Topology

Mental model on how different assets / components are organized within a Rockset organization:

Rockset asset hierarchy

Hierarchical representation of assets / components within Rockset:

Rockset environment separation

Separate environments (dev, test, production, etc.) by having multiple Rockset organizations (for example — one for dev/test, one for production) and using workspaces within them to separate use cases. This ensures complete separation of resources and is the recommended way of separating environments.

Separate environments by having a single Rockset organization with workspaces for each environment and use case (for example — dev/test for use case 1, prod for use case 1, dev/test for use case 2, prod for use case 2).