Before productionalizing applications powered by Rockset, it's important to consider a range of crucial factors. We've created a comprehensive checklist that Rockset users can go through to ensure their setup and architecture aligns with Rockset's best practices for production workloads.
- Separate development and production environments by using distinct workspaces or different Rockset accounts (examples of how to do this here).
- Utilize Terraform (and/or Rockset API) for deployment.
- Review data source best practices for used integrations.
- Review and/or clean unused integrations.
- Use workspaces to segregate data access and keep logical separation between different use cases.
- Avoid using the
- Avoid using the
- Use aliases in queries and query lambdas, and avoid referencing collections directly.
- Add clustering to collections, if relevant.
- Add data retention, if relevant.
- Review ingest transformations and remove unnecessary fields.
- Use special fields like
_opto ensure proper handling of data changes, if relevant.
- Review and/or clean unused workspaces, collections, views, and aliases.
- Use compute-compute separation to have isolated, dedicated VIs for each use case.
- Use continuous mounts on secondary VIs, depending on use case.
- Set-up auto-suspend for secondary VIs.
- Set-up auto-scaling for main/ingest VI.
- Review Virtual instances sizes to target price/performance for each use case.
- Use dedicated streaming ingest for latency sensitive workloads.
- Perform performance testing to get baseline performance for single queries and multiple concurrent queries.
- Use asynchronous option for long running / batch queries.
- Learn common query troubleshooting techniques.
- Ensure application logic (if applicable) has client timeout set and retry mechanisms in place with back-off and maximum number of retries (review these configurations in the
Execute SQL queryendpoint and
Execute Query Lambdaendpoint).
- Review and/or clean unused query lambdas.
- Enable Metrics endpoint and integrate with Datadog, Grafana, or similar visualization tools.
- Set-up a Query Log collection, preferably in a separate workspace like
- Set-up SSO for Rockset.
- Enforce SSO-only.
- Review and/or clean RBAC roles and user assignment to ensure least privilege access and data segregation.
- Use customer managed encryption keys, if relevant.
- Review and/or clean user list.
- Review and/or clean API keys.
- Set-up a rotation mechanism for API keys.
- Use single or multi-region deployment to achieve higher levels of availability, if applicable.
- Use collection snapshots to keep a copy of data, if relevant.
- Build an emergency query re-direct to a new VI in case the current VI becomes overwhelmed. Test by building an internal failover and timing how long it takes to switch 100% of queries to a new VI.
- Know how to reach Rockset support (via chat support and/or support tickets).
- Enable Rockset support access by going to Settings > Security.
- Subscribe to Rockset status page: https://status.rockset.com/
- If you have a support contract, run a mock fire drill. File a SEV1 support ticket and time how quickly support acknowledges the ticket.
- Learn Rockset's billing structure.
- Learn cost management best practices:
- Review and optimize collection sizes by utilizing storage compression, implementing data retention, removing unnecessary fields, and/or removing fields from the search index.
- Use roll-up collections to reduce collection size, if applicable.
- Set auto-suspend for secondary VI to save cost when not used.
- Set auto-scaling for main/ingest VI to save cost under lower usage.
Mental model on how different assets / components are organized within a Rockset organization:
Hierarchical representation of assets / components within Rockset:
Separate environments (dev, test, production, etc.) by having multiple Rockset organizations (for example — one for dev/test, one for production) and using workspaces within them to separate use cases. This ensures complete separation of resources and is the recommended way of separating environments.
Separate environments by having a single Rockset organization with workspaces for each environment and use case (for example — dev/test for use case 1, prod for use case 1, dev/test for use case 2, prod for use case 2).
Updated 1 day ago