Virtual Instances

A Virtual Instance is a set of compute resources used to process streaming ingest and queries.

Every Rockset organization starts with one Virtual Instance per active region. This becomes the default Virtual Instance for that region. Unless otherwise explicitly specified, all streaming ingest and queries are routed to this Virtual Instance.

Virtual Instances are compute resources and are separate from data storage. Virtual Instances can have mounted collections. Mounting a collection allows the Virtual Instance to serve queries that access that collection. The same collection can be mounted on any number of Virtual Instances.

You can manage your account's Virtual Instances in the Virtual Instances tab of the Rockset Console.

Virtual Instances can be hosted in one or more regions. For a list of current supported regions refer to the list here. Looking for a different region? Contact us at [email protected].

Choosing a Virtual Instance

Virtual Instances come in two types - Dedicated and Shared.

Dedicated Virtual Instances are isolated and meant for production use cases. Rockset offers the following Dedicated Virtual Instance sizes:

Shared Virtual Instances share a global resource pool with other accounts and are a cost-effective way to develop and prototype.

Note: Ingest represents the peak streaming ingest limit for insert-only workloads with record sizes of 1 KiB. Source and collection configurations such as average document size, ingest transformations, and clustering may lead to different observed throughputs.

Typically, the compute you'll require will depend on four things:

• data size: the size of the data you're querying
• query complexity: the complexity of your query
• query load: the number of concurrent queries
• query latency: the query latency requirement of your application
• streaming ingest load: the number and size of documents being processed, transformed, and
  indexed from the source

There are different limits according to the types and sizes of Virtual Instances, which are listed here. You can choose a Virtual Instance size that meets your needs.

Multiple Virtual Instances

A Virtual Instance is a set of compute resources used to process streaming ingest and queries.

Every Rockset organization starts with one Virtual Instance per active region. This becomes the default Virtual Instance for that region. Unless otherwise explicitly specified, all streaming ingest and queries are routed to this Virtual Instance.

Virtual Instances come in two types - Shared and Dedicated. Shared Virtual Instances share a global resource pool with other accounts and are a cost-effective way to develop and prototype. Dedicated Virtual Instances are isolated and meant for production use cases.

Virtual Instances are compute resources and are separate from data storage. To be able to query a collection using a Virtual Instance, the collection needs to be mounted to a Virtual Instance. Mounting a collection allows the Virtual Instance to serve queries that access that collection. The same collection can be mounted on any number of Virtual Instances.

Why Virtual Instances?

Virtual Instances allow you to isolate and independently scale compute for streaming ingest, low latency queries, and multiple applications also referred to as compute-compute separation. The workload served by one Virtual Instance does not affect the workload running on any other Virtual Instance.

Streaming Ingest Isolation & Scaling

Virtual Instances allow you to isolate and independently scale compute for ingest and queries. To eliminate the problem of compute contention, we recommend using one Virtual Instance for handling all the streaming ingest workload exclusively (with no query requests routed to that Virtual Instance, sometimes referred to as an “Ingest VI”), and additional VIs for each query use case.

Note: Today only the default Virtual Instance may be used for streaming ingest. We recommend dedicating this Virtual Instances to streaming ingest and using other Virtual Instances for queries.

Isolation & Scaling for Multiple Real-time Applications

Suppose you're using Rockset to power both a feature in your customer-facing application and an internal dashboard. A spike in internal usage shouldn't degrade the performance of your customer-facing application, and you should be able to isolate and scale compute for each application independently (upsizing customer-facing compute for an anticipated event, for example). We recommend using a dedicated Virtual Instance for each application.

Concurrency Scaling

Applications with fluctuations in demand can provide a consistent and predictable performance by scaling out the query requests to multiple virtual instances. Each of the virtual instances can mount the same set of collections and serve the same queries, allowing for better load balancing of increased loads during peak times.

Isolation for Analysis/Development

Virtual Instances allow you to isolate compute for one-off data analysis sessions (say running a set of complex compute-intensive aggregations for a monthly or quarterly report). You can run a compute-intensive query on one Virtual Instance while your application queries are safely served from another.

👍

We suggest using a second dedicated Virtual Instance for development queries to avoid affecting your production workloads.

Multi-Tenant Isolation

Virtual Instances allow you to provide your own tenants (whether they be internal teams or customers) compute isolation and track usage and spend accordingly. You can mount collections across these Virtual Instances to share data as needed while using custom roles to limit access.

Providing a dedicated Virtual Instance for your largest tenants enables a better experience by avoiding resource contention from your smaller tenants and vice versa.

Using Virtual Instances

Streaming Ingest

Today the default Virtual Instance is set to process streaming ingest for all collections. You cannot configure this behavior otherwise at this time.

Dedicated Streaming Ingest

Dedicated Streaming Ingest is an optional feature which provisions dedicated tailer infrastructure and applies configurations to deliver the best performance for your latency sensitive ingest operations. To learn more about this feature see here.

Collection Mounts: Making Data Available for Queries

A collection mount represents a relationship between a collection (storage) and a Virtual Instance (compute). Queries routed to a particular Virtual Instance can only access a given collection if that collection is mounted to the Virtual Instance. Collection mounts do not incur any additional storage costs, but they can incur additional CPU overhead on the Virtual Instance to which they are mounted.

Mounting and unmounting a collection should only take a few seconds, but sometimes it can take longer (up to one minute). You can manage collection mounts through the 'Virtual Instances Details' page Rockset Console or directly through the API.

Routing Queries to Virtual Instances

In the Console Query Editor, you can select the Virtual Instance you're sending queries to:

You can also specify the Virtual Instance for queries through the API.

You can find the virtualInstanceId you need to use in the Virtual Instance details page with a copy button right next to it:

Copy the above ID use it with the query endpoint, as follows:

https://api.[region].rockset.com/v1/orgs/self/virtualinstances/[virtualInstanceId]/queries

Alternatively use the request body for executing Query Lambdas and include the virtualInstanceId field.

If no Virtual Instance is specified, the query will be routed to the default Virtual Instance.

Virtual Instance Configuration

Virtual Instances have additional configurations for advanced usage. The default values should be appropriate in most circumstances.

• Auto-Suspend: If the Virtual Instance is not queried for a specified time, the Virtual Instance will automatically suspend and you will no longer be charged for running the Virtual Instance. By default, Virtual Instances will auto-suspend after 1 hour.
• Mount Type: Rockset supports two different mount types that determine if the mounted collection will receive any updates from the underlying collection.
  • Live mounted collections will stay up-to-date with the underlying collection in real time. They will incur a small CPU overhead required to process streaming updates to your underlying collections. This is the default value, and live mounts should be used in most situations. Live mounts are temporarily not supported for Small Virtual Instances. You can still use a Small default Virtual Instance to save on cost. Contact us for further assistance with architecting your Rockset org for an optimal price-performance.
  • Static mounted collections will not receive updates from the underlying collection. They can be used for queries where you don't expect the collection data to change. Static mounts will not incur the same CPU overhead, since they do not process any updates.
• Remount on Resume: If this setting is enabled, suspending a Virtual Instance will not unmount all collections. Instead, the collections will be suspended and remounted when the Virtual Instance is resumed. By default, remount on resume is enabled. You may want to disable this setting if you do not want to incur the CPU overhead of mounting collections that are unused.

These settings can be configured upon Virtual Instance creation under "Additional Configuration" in the console or through the API.

Virtual Instance Auto-Scaling

Note: Virtual Instance Auto-Scaling is currently in Beta.

CPU-based vertical Auto-Scaling is a technique that automatically adjusts Virtual Instance size according to current workload demands. It is the most cost-efficient way to run Rockset while maximizing performance. With this feature enabled, the Virtual Instance size will auto-scale up during heavier workloads to deliver better performance and auto-scale down when idle to avoid over provisioning resources and reduce costs.

How Rockset Auto-Scales Virtual Instance Size

Rockset analyzes Virtual Instance CPU utilization to assess whether to auto-scale a Virtual Instance size up or down. CPU utilization is observed for auto-scaling using an iterative decay approach, allowing for historical analysis with amplified emphasis on recent measurements when making auto-scaling decisions.

• Auto-Scale Up occurs when the CPU utilization decay value exceeds 75%.
• Auto-Scale Down occurs when the CPU utilization decay value is below 25%.
• After a Virtual Instance switch has completed or upon enabling auto-scaling, distinct cooldown periods of 3 minutes and 1 hour are observed for subsequent auto-scale up and down actions.

Configuring an Auto-Scaling Policy

Auto-Scaling can be configured in the Rockset Console or through the Rockset API.

To create an Auto-Scaling policy on a Virtual Instance, go to the Virtual Instance tab of the Rockset Console and navigate to the Edit Virtual Instance menu for the Virtual Instance you want to configure. Define the minimum and maximum sizes you will allow the policy to scale the Virtual Instance.

Note: Virtual Instance Auto-Scaling is currently only available for Default Virtual Instances.

Monitoring Auto-Scaling Events

When an Auto-Scaling occurs:

• Rockset logs Auto-Scaling events in the Activity Logs in the Virtual Instance details page.
• Rockset sends an email with details about the Auto-Scaling action.

Monitor how Rockset makes Auto-Scaling decisions based on CPU utilization with Virtual Instance metrics monitoring.

Manually Resizing VI with Auto-Scaling Enabled

If you have Auto-Scaling enabled but want to manually resize your VI, there are two possible outcomes:

1. Auto-Scaling will remain enabled if you resize your VI within your Auto-Scaling range. For example, if your Auto-Scaling has a minimum size of Small and maximum size of Large and you resize your VI from Large to Medium, Auto-Scaling will remain in place.
2. Auto-Scaling will be disabled if you resize your VI outside of your Auto-Scaling range. For example, if your Auto-Scaling minimum size is Small and maximum size is Large and you resize from Large to XL, Auto-Scaling will be disabled.

Managing Virtual Instances

Suspending and Resuming

Virtual Instances can be suspended and resumed. Suspending a VI will unmount all collections. Collections will not be automatically remounted when the Virtual Instance is resumed.

You can configure an Auto-Suspend Policy for a Virtual Instance. The Virtual Instance will suspend itself if it does not receive new queries during this duration.

Note: The default Virtual Instance cannot be suspended since it needs to process incoming data at all times.

Monitoring

You can find metrics specific to each Virtual Instance in the Console. The main Metrics tab has a 'Virtual Instance' selector, and each Virtual Instance Details page will also include some metrics.

The Metrics Endpoint will include Virtual Instance as a label in all compute-related metrics by default - allowing you to monitor each Virtual Instance independently as needed.

Access Management

You can scope privileges to specific Virtual Instances to restrict access to them. The following privileges can be given:

• Create: allows creating new Virtual Instances
• Query: allows sending queries to a particular VI
• Update: allows upsizing and downsizing as well as mounting/unmounting collections for a particular VI
• Suspend/Resume: allows suspending and resuming a particular VI
• Delete: allows deletion

You can find the full Role-Based Access Control reference here.

FAQs

Is the Shared Virtual Instance suitable for production workloads?

The Shared Virtual Instance is not suitable for production workloads. Accounts in this Virtual Instance share a global resource pool and isolation is not guaranteed. Your data is strictly kept isolated from other accounts but you share CPU and memory resources with other accounts. Sharing resources with other accounts allows you to utilize a lot of resources at a cost-effective price and is a great place to start using Rockset. A noisy neighbor can affect the latency and throughput of your queries. Bursty workloads from accounts may impact your query performance and availability.

The Shared Virtual Instance is usually useful for development and prototyping.

Will moving to a larger Virtual Instance make my queries faster?

Generally, adding compute by switching to a larger Virtual Instance will make your queries faster. Typically, doubling CPU allocation (e.g., moving from Medium (8 CPUs) to Large (16 CPUs) will approximately double your query performance (e.g., reduce query latency from 100ms to 50ms). Note that eventually your queries will not be bound by compute and instead reach the limitations of the underlying infrastructure — at which point adding additional compute will cease to affect latency.

If I expect my query load to increase significantly, do I need to move to a larger Virtual Instance?

Generally, yes. For example, if you're running 50 queries-per-sec and add an additional 50 queries-per-second (for 100 queries-per-second total), each query will get a proportionally smaller compute allocation, and query latency will increase accordingly (generally proportional to the increase in QPS). In this example, you might notice a query latency increase of approximately 2x. If you want your query latencies to remain the same even when you double your QPS, you should migrate to a Virtual Instance that is 2x larger in size.

Alternatively, you can choose to create additional Virtual Instances, mount the required collections, and spread the query request load among those Virtual Instances. You can suspend and resume those Virtual Instances based on when the load is higher or lower.

We always recommend stress-testing production workloads before launching your application on Rockset.

Does a Virtual Instance have a fixed amount of compute resources?

Yes, a Virtual Instance has a fixed amount of compute resources. You can change the size of a Virtual Instance through this API.

Is there a limit to the number of additional Virtual Instances I can create in a single Rockset org?

You can have as many additional (Query) VIs as you need. Each Query VI adds a minor CPU overhead to the primary VI (responsible for stream ingest processing) and that overhead is dependent on the write rate of your collections. The lower the write rate on your collections, the lesser the overhead on the primary VI. This means that the size of the primary VI and your write rate determines the maximum number of query VIs. Because Rockset is cloud-native, you can always increase the size of the primary VI if you want to increase the number of query VIs

Does resizing a Virtual Instance incur downtime?

You can resize your Virtual Instance at any time in the Virtual Instances tab of the Rockset Console. There is no downtime associated with this action. Once the resize is completed, Rockset will automatically begin executing queries on your new Virtual Instance.

Can you mount the same collection on multiple Virtual Instances?

Yes, the same collection can be mounted on more than one Virtual Instances. Writes to that collection are processed by the default Virtual Instance. The other Virtual Instances are used to serve queries on that collection.

Can you mount multiple collections on one Virtual Instances?

Yes, you can mount multiple collections on the same Virtual Instance.

If you run a INSERT INTO SELECT query on a Virtual Instance, which compute resources does it use?

The SELECT part of the query uses the compute resources from the Virtual Instance that is specified as part of the query request. This compute is used to generate the result set as specified by the SELECT query. Then the result set has to be inserted into the specified target collection. The compute resources from the default Virtual Instance is used to write the result set into the target collection.