Special Fields

This page describes fields treated specially in Rockset documents.

Special fields can be queried like any other field.

SELECT

    _id,

    _meta,

    _event_time

FROM

    mycollection

LIMIT

    5;

+---------------------------------------------------------------------------------+----------------+----------------------------------------+

| _meta                                                                           | _event_time    | _id                                    |

|---------------------------------------------------------------------------------+----------------+----------------------------------------|

| {'s3': {'bucket': 'mycollection', 'offset': 0, 'path': '47304-1709397620935'}}  | 1535068823524  | '5d99f42c-44ec-2024-a032-ac14d1cbf44d' |

| {'s3': {'bucket': 'mycollection', 'offset': 0, 'path': '46242-1709397472272'}}  | 1535068823426  | 'e5d4c136-8d56-ac39-38f5-2f795ae38007' |

| {'s3': {'bucket': 'mycollection', 'offset': 0, 'path': '46242-1503238554088'}}  | 1535068823326  | '71df712e-04bc-7c9b-0114-97b19741b215' |

| {'s3': {'bucket': 'mycollection', 'offset': 0, 'path': '47302-1151051235692'}}  | 1535068822852  | '517412d2-1a23-1b49-fc3b-441c630ed4ff' |

| {'s3': {'bucket': 'mycollection', 'offset': 0, 'path': '45189-1709397393868'}}  | 1535068822527  | 'd16bf924-f758-21db-c832-4a25323b2938' |

+---------------------------------------------------------------------------------+----------------+----------------------------------------+

#_id

Every document in a Rockset collection is uniquely identified by its _id field.

• If the source of a document does not already have an _id field, Rockset populates it with an automatically generated uuid.
• If the source of a document has _id specified, or a field mapping outputs an _id field, its value is preserved. A newly ingested document will overwrite any existing document with the same _id value.

#_meta

Metadata regarding each document is stored in a _meta field of object type.

If the source of a document specifies a _meta field, Rockset will ignore the field. Currently, _meta holds information about the source from which the document was inserted into the collection (e.g. the bucket name and path in case of S3). If Rockset is unable to parse the source of a document, it will create a document without any of the source's fields and will have _meta with a nested field named bad.

#_event_time

Rockset associates a timestamp with each document in a field named _event_time, recorded as microseconds since the Unix epoch. Queries on this field are significantly faster than similar queries on regularly-indexed fields.

A user can select a field of the source of a document to be copied into the _event_time field at collection creation (refer to the event data guide for more information). This is done by specifying:

• the name of the selected field, which should be of type int or timestamp
• the format with which to interpret the field, can be one of:
  • milliseconds_since_epoch
  • seconds_since_epoch
• default time zone, specified as one of the IANA time zone values

If an _event_time configuration is not specified, Rockset will first check for a pre-existing _event_time field (must be type int or timestamp) and, if nonexistent, populates it with the time of document insertion.