-
Classes Class Description org.apache.pinot.spi.data.TimeFieldSpec UseDateTimeFieldSpecinstead. This should only be used in 1) tests 2) wherever required for backward compatible handling of schemas with TimeFieldSpec https://github.com/apache/pinot/issues/2756org.apache.pinot.spi.data.TimeGranularitySpec Use DateTimeFieldSpec instead. TheTimeGranularitySpecclass contains all specs related to time field.-
DataType: data type of the time column (e.g. INT, LONG).-
TimeType: time unit of the time column (e.g. MINUTES, HOURS).-
TimeUnitSize: size of the time buckets (e.g. 10 MINUTES, 2 HOURS). By default this is set to 1.-
TimeFormat: Can be either EPOCH (default) or SIMPLE_DATE_FORMAT:pattern e.g SIMPLE_DATE_FORMAT:yyyyMMdd-
Name: name of the time column.E.g.
If the time column is in millisecondsSinceEpoch, constructor can be invoked as:
TimeGranularitySpec(LONG, MILLISECONDS, timeColumnName)If the time column is in tenMinutesSinceEpoch, constructor can be invoked as:
TimeGranularitySpec(LONG, 10, MINUTES, timeColumnName)If the time column is in Simple Date Format:
new TimeGranularitySpec(DataType.STRING, 1, TimeUnit.HOURS, TimeFormat.SIMPLE_DATE_FORMAT.toString() +":yyyyMMdd", "hour");org.apache.pinot.spi.utils.CommonConstants.Segment.Offline org.apache.pinot.spi.utils.TimeConverter This conversion should be done via transform functions set on the schema field spec TimeConverter to convert value to/from milliseconds since epoch based on the givenTimeGranularitySpec.
-
Enums Enum Description org.apache.pinot.spi.data.TimeGranularitySpec.TimeFormat
-
Fields Field Description org.apache.pinot.spi.data.FieldSpec._transformFunction org.apache.pinot.spi.stream.StreamConfigProperties.DEPRECATED_SEGMENT_FLUSH_DESIRED_SIZE because the property key is confusing (desired size is not indicative of segment size). UseStreamConfigProperties.SEGMENT_FLUSH_THRESHOLD_SEGMENT_SIZEThe desired size of a completed realtime segment. This config is used only if REALTIME_SEGMENT_FLUSH_SIZE is set to 0. Default value of REALTIME_SEGMENT_FLUSH_SIZE is "200M". Values are parsed using DataSize class. The value for this configuration should be chosen based on the amount of memory available on consuming machines, the number of completed segments that are expected to be resident on the machine and the amount of memory used by consuming machines. In other words: numPartitionsInMachine * (consumingPartitionMemory + numPartitionsRetained * REALTIME_DESIRED_SEGMENT_SIZE) must be less than or equal to the total memory available to store pinot data. Note that consumingPartitionMemory will vary depending on the rows that are consumed. Not included here is any heap memory used (currently inverted index uses heap memory for consuming partitions).org.apache.pinot.spi.stream.StreamConfigProperties.DEPRECATED_SEGMENT_FLUSH_THRESHOLD_ROWS because the property key is confusing (says size but is actually rows). UseStreamConfigProperties.SEGMENT_FLUSH_THRESHOLD_ROWSRow count flush threshold for realtime segments. This behaves in a similar way for HLC and LLC. For HLC, since there is only one consumer per server, this size is used as the size of the consumption buffer and determines after how many rows we flush to disk. For example, if this threshold is set to two million rows, then a high level consumer would have a buffer size of two million. For LLC, this size is divided across all the segments assigned to a given server and is set on a per segment basis. Assuming a low level consumer server is assigned four stream partitions to consume from and a flush size of two million, then each consuming segment would have a flush size of five hundred thousand rows, for a total of two million rows in memory. Keep in mind that this NOT a hard threshold, as other tables can also be assigned to this server, and that in certain conditions (eg. if the number of servers, replicas of partitions changes) where partition to server assignment changes, it's possible to end up with more (or less) than this number of rows in memory. If this value is set to 0, then the consumers adjust the number of rows consumed by a partition such that the size of the completed segment is the desired size (see REALTIME_DESIRED_SEGMENT_SIZE), unless REALTIME_SEGMENT_FLUSH_TIME is reached first)org.apache.pinot.spi.utils.CommonConstants.Broker.Request.QueryOptionKey.GROUP_BY_MODE org.apache.pinot.spi.utils.CommonConstants.Broker.Request.QueryOptionKey.PRESERVE_TYPE org.apache.pinot.spi.utils.CommonConstants.Broker.Request.QueryOptionKey.RESPONSE_FORMAT org.apache.pinot.spi.utils.CommonConstants.Helix.DEPRECATED_ENABLE_CASE_INSENSITIVE_KEY org.apache.pinot.spi.utils.CommonConstants.Helix.Instance.INSTANCE_ID_KEY org.apache.pinot.spi.utils.CommonConstants.Minion.DEPRECATED_CONFIG_OF_METRICS_PREFIX_KEY org.apache.pinot.spi.utils.CommonConstants.Minion.DEPRECATED_PREFIX_OF_CONFIG_OF_PINOT_CRYPTER org.apache.pinot.spi.utils.CommonConstants.Minion.DEPRECATED_PREFIX_OF_CONFIG_OF_PINOT_FS_FACTORY org.apache.pinot.spi.utils.CommonConstants.Minion.DEPRECATED_PREFIX_OF_CONFIG_OF_SEGMENT_FETCHER_FACTORY org.apache.pinot.spi.utils.CommonConstants.Minion.DEPRECATED_PREFIX_OF_CONFIG_OF_SEGMENT_UPLOADER org.apache.pinot.spi.utils.CommonConstants.Segment.Realtime.DOWNLOAD_URL org.apache.pinot.spi.utils.CommonConstants.Server.SegmentCompletionProtocol.CONFIG_OF_CONTROLLER_HTTPS_ENABLED org.apache.pinot.spi.utils.CommonConstants.Server.SegmentCompletionProtocol.CONFIG_OF_CONTROLLER_HTTPS_PORT