Package org.apache.pinot.segment.local.utils

Class TableConfigUtils

  • public final class TableConfigUtils
extends Object
    Utils related to table config operations FIXME: Merge this TableConfigUtils with the TableConfigUtils from pinot-common when merging of modules is done

    • Method Detail

      • validate

        public static void validate​(TableConfig tableConfig,
                            @Nullable
                            Schema schema,
                            @Nullable
                            String typesToSkip,
                            boolean disableGroovy)
        Performs table config validations. Includes validations for the following: 1. Validation config 2. IngestionConfig 3. TierConfigs 4. Indexing config 5. Field Config List TODO: Add more validations for each section (e.g. validate conditions are met for aggregateMetrics)

      • validateTableName

        public static void validateTableName​(TableConfig tableConfig,
                                     boolean allowTableNameWithDatabase)
        Validates the table name with the following rules:
        • If there is a flag allowing database name in it, table name can have one dot in it.
        • Otherwise, there is no dot allowed in table name.

      • validateIngestionConfig

        public static void validateIngestionConfig​(TableConfig tableConfig,
                                           @Nullable
                                           Schema schema)

      • validateIngestionConfig

        public static void validateIngestionConfig​(TableConfig tableConfig,
                                           @Nullable
                                           Schema schema,
                                           boolean disableGroovy)
        Validates the following: 1. validity of filter function 2. checks for duplicate transform configs 3. checks for null column name or transform function in transform config 4. validity of transform function string 5. checks for source fields used in destination columns 6. ingestion type for dimension tables

      • validateIngestionAggregation

        public static void validateIngestionAggregation​(AggregationFunctionType functionType)

      • ensureMinReplicas

        public static void ensureMinReplicas​(TableConfig tableConfig,
                                     int defaultTableMinReplicas)
        TODO: After deprecating "replicasPerPartition", we can change this function's behavior to always overwrite config to "replication" only. Ensure that the table config has the minimum number of replicas set as per cluster configs. If is doesn't, set the required amount of replication in the table config

      • ensureStorageQuotaConstraints

        public static void ensureStorageQuotaConstraints​(TableConfig tableConfig,
                                                 String maxAllowedSize)
        Ensure the table config has storage quota set as per cluster configs. If it doesn't, set the quota config into the table config

      • verifyHybridTableConfigs

        public static void verifyHybridTableConfigs​(String rawTableName,
                                            TableConfig offlineTableConfig,
                                            TableConfig realtimeTableConfig)
        Consistency checks across the offline and realtime counterparts of a hybrid table

      • needsEmptySegmentPruner

        public static boolean needsEmptySegmentPruner​(TableConfig tableConfig)
        needsEmptySegmentPruner checks if EmptySegmentPruner is needed for a TableConfig.
        Parameters:
        tableConfig - Input table config.

      • createTableConfigFromOldFormat

        public static TableConfig createTableConfigFromOldFormat​(TableConfig tableConfig,
                                                         Schema schema)
        Helper method to extract TableConfig in updated syntax from current TableConfig.
        • Moves all index configs to FieldConfig.indexes
        • Clean up index related configs from IndexingConfig and FieldConfig.IndexTypes