Package org.apache.pinot.segment.local.utils

Class TableConfigUtils

java.lang.Object

org.apache.pinot.segment.local.utils.TableConfigUtils

public final class TableConfigUtils
extends Object

Utils related to table config operations FIXME: Merge this TableConfigUtils with the TableConfigUtils from pinot-common when merging of modules is done

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TableConfigUtils.ValidationType

Field Summary

Fields

Modifier and Type	Field	Description
static EnumSet<AggregationFunctionType>	AVAILABLE_CORE_VALUE_AGGREGATORS

Method Summary

Modifier and Type	Method	Description
static TableConfig	createTableConfigFromOldFormat(TableConfig tableConfig, Schema schema)	Helper method to extract TableConfig in updated syntax from current TableConfig.
static void	ensureMinReplicas(TableConfig tableConfig, int defaultTableMinReplicas)	Ensure that the table config has the minimum number of replicas set as per cluster configs.
static void	ensureStorageQuotaConstraints(TableConfig tableConfig, String maxAllowedSize)	Ensure the table config has storage quota set as per cluster configs.
static boolean	needsEmptySegmentPruner(TableConfig tableConfig)	needsEmptySegmentPruner checks if EmptySegmentPruner is needed for a TableConfig.
static void	validate(TableConfig tableConfig, Schema schema)
static void	validate(TableConfig tableConfig, Schema schema, String typesToSkip, boolean disableGroovy)	Performs table config validations.
static void	validateIngestionAggregation(AggregationFunctionType functionType)
static void	validateIngestionConfig(TableConfig tableConfig, Schema schema)
static void	validateIngestionConfig(TableConfig tableConfig, Schema schema, boolean disableGroovy)	Validates the following: 1.
static void	validateTableName(TableConfig tableConfig)	Validates the table name with the following rules: Table name can have at most one dot in it.
static void	verifyHybridTableConfigs(String rawTableName, TableConfig offlineTableConfig, TableConfig realtimeTableConfig)	Consistency checks across the offline and realtime counterparts of a hybrid table

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

AVAILABLE_CORE_VALUE_AGGREGATORS

public static final EnumSet<AggregationFunctionType> AVAILABLE_CORE_VALUE_AGGREGATORS

Method Detail

validate

public static void validate(TableConfig tableConfig,
                            @Nullable
                            Schema schema)

See Also:

validate(TableConfig, Schema, String, boolean)

validate

public static void validate(TableConfig tableConfig,
                            @Nullable
                            Schema schema,
                            @Nullable
                            String typesToSkip,
                            boolean disableGroovy)

Performs table config validations. Includes validations for the following: 1. Validation config 2. IngestionConfig 3. TierConfigs 4. Indexing config 5. Field Config List TODO: Add more validations for each section (e.g. validate conditions are met for aggregateMetrics)

validateTableName

public static void validateTableName(TableConfig tableConfig)

Validates the table name with the following rules:

• Table name can have at most one dot in it.
• Table name does not have whitespace.

validateIngestionConfig

public static void validateIngestionConfig(TableConfig tableConfig,
                                           @Nullable
                                           Schema schema)

validateIngestionConfig

public static void validateIngestionConfig(TableConfig tableConfig,
                                           @Nullable
                                           Schema schema,
                                           boolean disableGroovy)

Validates the following: 1. validity of filter function 2. checks for duplicate transform configs 3. checks for null column name or transform function in transform config 4. validity of transform function string 5. checks for source fields used in destination columns 6. ingestion type for dimension tables

validateIngestionAggregation

public static void validateIngestionAggregation(AggregationFunctionType functionType)

ensureMinReplicas

public static void ensureMinReplicas(TableConfig tableConfig,
                                     int defaultTableMinReplicas)

Ensure that the table config has the minimum number of replicas set as per cluster configs.

ensureStorageQuotaConstraints

public static void ensureStorageQuotaConstraints(TableConfig tableConfig,
                                                 String maxAllowedSize)

Ensure the table config has storage quota set as per cluster configs. If it doesn't, set the quota config into the table config

verifyHybridTableConfigs

public static void verifyHybridTableConfigs(String rawTableName,
                                            TableConfig offlineTableConfig,
                                            TableConfig realtimeTableConfig)

Consistency checks across the offline and realtime counterparts of a hybrid table

needsEmptySegmentPruner

public static boolean needsEmptySegmentPruner(TableConfig tableConfig)

needsEmptySegmentPruner checks if EmptySegmentPruner is needed for a TableConfig.

Parameters:

tableConfig - Input table config.

createTableConfigFromOldFormat

public static TableConfig createTableConfigFromOldFormat(TableConfig tableConfig,
                                                         Schema schema)

Helper method to extract TableConfig in updated syntax from current TableConfig.

• Moves all index configs to FieldConfig.indexes
• Clean up index related configs from IndexingConfig and FieldConfig.IndexTypes