This section walks through a few high-level errors you may encounter with Rockset. This section helps you diagnose and fix problems with ingest errors.
When you’re ingesting data from a source, you may encounter the following errors:
- Parse Errors
- Field Malformations During Ingest Transformation
- Document Size is too Large
- Authentication Errors
- Source errors
- SQL Errors
Parse Errors occur when data is malformed. When you send a document to Rockset, you sometimes have the option to specify its type:
You might see a Parse Error if there’s a discrepancy between your file format and the expected file format (e.g. if you specify a CSV file and send a JSON document instead). This error is also returned for an improperly formatted CSV or TSV file.
To fix Parse Errors:
- Verify that your code or the console is ingesting the correct document.
- If you are ingesting a CSV or TSV file, make sure the separator, encoding, header, quote character, and
escape character are all correct.
An incorrectly-written ingest transformation results in a
_meta.bad error message in the ingested document document. Common reasons for this error include:
- Mapping a non-string field to
- Mapping a non-timestamp field to
To fix field malformations for an ingest transformation error:
- Fix the rollup and ingest transformation query (i.e., map string fields to
_idand timestamp fields to
If you’re unsure of how to update the query, please reach out to the community for help.
You can ingest records or documents with sizes up to 40 MiB. Field names are capped at 10 KiB.
If you encounter an error indicating "Size of document exceeds the maximum allowed size of 41943040 bytes", you can drop fields at ingestion time via an ingest transformation.
Note that Write API has separate request limits enforced before parsing.
Authentication errors occur when a user is not authorized to access a resource. Authentication errors look similar to the following:
AZ_BLOB_STORAGE bucket "jsrocksetblob" does not have list objects permissions set. Authentication failed for AWS cross-account role integration with Role ARN
To fix authentication errors:
- Check our docs page for data sources and ensure that the proper permissions have been set.
- Ensure you select the correct source.
- Once the permissions have been updated, recreate your collection with the relevant integration.
If you have questions, please reach out to the Rockset Community.
Source Errors occur when Rockset has trouble accessing the data source. Some common source-related errors include:
The following error can occur when someone deletes a table:
DynamoDB resource not found for table `table_name` in region us-east-1
The following error can occur when someone deletes a Kafka topic:
The Kafka topic is not known to the broker: "nucleus.custom_activities.backup.partner.609040244446599c8b47cb39.membership_counts_o3k"
The following error can occur when a MongoDB resume token expires:
Resume token has expired
MongoDB stream events are emitted to the oplog on the server. There is one oplog per MongoDB cluster. Each oplog has a size limit, so the last event is truncated when the limit is exceeded. If Rockset is falling behind while tailing the oplog (due to a cap on ingest rate, based on VI size) then it can lose the position in the oplog and cannot tail the stream further, in which case the Rockset Collection is not updated.
To fix Source Errors:
- Ensure the tables, collections, and streams are created.
- If you’re getting a token that has an expired-type error, try increasing the VI to increase the ingest rate.
Updated 1 day ago