Package org.apache.pinot.segment.local.segment.index.loader

Class IndexLoadingConfig

  • public class IndexLoadingConfig
extends Object
    Table level index loading config.

    • Constructor Detail

      • IndexLoadingConfig

        public IndexLoadingConfig​(InstanceDataManagerConfig instanceDataManagerConfig,
                          TableConfig tableConfig,
                          @Nullable
                          Schema schema)
        NOTE: This step might modify the passed in table config and schema.

      • IndexLoadingConfig

        public IndexLoadingConfig​(TableConfig tableConfig,
                          @Nullable
                          Schema schema)

      • IndexLoadingConfig

        public IndexLoadingConfig()

    • Method Detail

      • refreshIndexConfigs

        public void refreshIndexConfigs()

      • getReadMode

        public ReadMode getReadMode()

      • setReadMode

        public void setReadMode​(ReadMode readMode)
        For tests only.

      • getSortedColumns

        public List<String> getSortedColumns()

      • setSortedColumn

        public void setSortedColumn​(String sortedColumn)
        For tests only.

      • getInvertedIndexColumns

        public Set<String> getInvertedIndexColumns()

      • getRangeIndexColumns

        public Set<String> getRangeIndexColumns()

      • addRangeIndexColumn

        public void addRangeIndexColumn​(String... columns)

      • getRangeIndexVersion

        public int getRangeIndexVersion()

      • getFSTIndexType

        public FSTType getFSTIndexType()

      • getTextIndexColumns

        public Set<String> getTextIndexColumns()
        Used in two places: (1) In PhysicalColumnIndexContainer to create the index loading info for immutable segments (2) In RealtimeSegmentDataManager to create the RealtimeSegmentConfig. RealtimeSegmentConfig is used to specify the text index column info for newly to-be-created Mutable Segments
        Returns:
        a set containing names of text index columns

      • getFSTIndexColumns

        public Set<String> getFSTIndexColumns()

      • setInvertedIndexColumns

        public void setInvertedIndexColumns​(Set<String> invertedIndexColumns)
        For tests only.

      • addInvertedIndexColumns

        public void addInvertedIndexColumns​(String... invertedIndexColumns)

      • addInvertedIndexColumns

        public void addInvertedIndexColumns​(Collection<String> invertedIndexColumns)

      • removeInvertedIndexColumns

        public void removeInvertedIndexColumns​(String... invertedIndexColumns)

      • removeInvertedIndexColumns

        public void removeInvertedIndexColumns​(Collection<String> invertedIndexColumns)

      • setNoDictionaryColumns

        public void setNoDictionaryColumns​(Set<String> noDictionaryColumns)
        For tests only. Used by segmentPreProcessorTest to set raw columns.

      • removeNoDictionaryColumns

        public void removeNoDictionaryColumns​(String... noDictionaryColumns)

      • removeNoDictionaryColumns

        public void removeNoDictionaryColumns​(Collection<String> noDictionaryColumns)

      • addNoDictionaryColumns

        public void addNoDictionaryColumns​(String... noDictionaryColumns)

      • addNoDictionaryColumns

        public void addNoDictionaryColumns​(Collection<String> noDictionaryColumns)

      • setCompressionConfigs

        public void setCompressionConfigs​(Map<String,​FieldConfig.CompressionCodec> compressionConfigs)
        For tests only. Used by segmentPreProcessorTest to set compression configs.

      • setRangeIndexColumns

        public void setRangeIndexColumns​(Set<String> rangeIndexColumns)
        For tests only.

      • addRangeIndexColumns

        public void addRangeIndexColumns​(String... rangeIndexColumns)

      • removeRangeIndexColumns

        public void removeRangeIndexColumns​(String... rangeIndexColumns)

      • setTextIndexColumns

        public void setTextIndexColumns​(Set<String> textIndexColumns)
        Used directly from text search unit test code since the test code doesn't really have a table config and is directly testing the query execution code of text search using data from generated segments and then loading those segments.

      • addTextIndexColumns

        public void addTextIndexColumns​(String... textIndexColumns)

      • removeTextIndexColumns

        public void removeTextIndexColumns​(String... textIndexColumns)

      • setFSTIndexColumns

        public void setFSTIndexColumns​(Set<String> fstIndexColumns)

      • addFSTIndexColumns

        public void addFSTIndexColumns​(String... fstIndexColumns)

      • removeFSTIndexColumns

        public void removeFSTIndexColumns​(String... fstIndexColumns)

      • setFSTIndexType

        public void setFSTIndexType​(FSTType fstType)

      • setJsonIndexColumns

        public void setJsonIndexColumns​(Set<String> jsonIndexColumns)

      • setOnHeapDictionaryColumns

        public void setOnHeapDictionaryColumns​(Set<String> onHeapDictionaryColumns)

      • setForwardIndexDisabledColumns

        public void setForwardIndexDisabledColumns​(Set<String> forwardIndexDisabledColumns)
        For tests only.

      • addForwardIndexDisabledColumns

        public void addForwardIndexDisabledColumns​(String... forwardIndexDisabledColumns)

      • removeForwardIndexDisabledColumns

        public void removeForwardIndexDisabledColumns​(String... forwardIndexDisabledColumns)

      • getNoDictionaryColumns

        public Set<String> getNoDictionaryColumns()

      • getCompressionConfigs

        public Map<String,​FieldConfig.CompressionCodec> getCompressionConfigs()
        Populates a map containing column name as key and compression type as value. This map will only contain the compressionType overrides, and it does not correspond to the default value of compressionType (derived using SegmentColumnarIndexCreator.getColumnCompressionType()) used for a column. Note that only RAW forward index columns will be populated in this map.
        Returns:
        a map containing column name as key and compressionType as value.

      • getNoDictionaryConfig

        public Map<String,​String> getNoDictionaryConfig()

      • getVarLengthDictionaryColumns

        public Set<String> getVarLengthDictionaryColumns()

      • getOnHeapDictionaryColumns

        public Set<String> getOnHeapDictionaryColumns()

      • getForwardIndexDisabledColumns

        public Set<String> getForwardIndexDisabledColumns()

      • isEnableDynamicStarTreeCreation

        public boolean isEnableDynamicStarTreeCreation()

      • isEnableDefaultStarTree

        public boolean isEnableDefaultStarTree()

      • getSegmentVersion

        @Nullable
public SegmentVersion getSegmentVersion()

      • setSegmentVersion

        public void setSegmentVersion​(SegmentVersion segmentVersion)
        For tests only.

      • isRealtimeOffHeapAllocation

        public boolean isRealtimeOffHeapAllocation()

      • isDirectRealtimeOffHeapAllocation

        public boolean isDirectRealtimeOffHeapAllocation()

      • getSegmentStoreURI

        public String getSegmentStoreURI()

      • setColumnMinMaxValueGeneratorMode

        public void setColumnMinMaxValueGeneratorMode​(ColumnMinMaxValueGeneratorMode columnMinMaxValueGeneratorMode)
        For tests only.

      • getRealtimeAvgMultiValueCount

        public int getRealtimeAvgMultiValueCount()

      • getSchema

        @Nullable
public Schema getSchema()

      • setTableConfig

        public void setTableConfig​(TableConfig tableConfig)

      • getSegmentDirectoryLoader

        public String getSegmentDirectoryLoader()

      • getInstanceId

        public String getInstanceId()

      • setTableDataDir

        public void setTableDataDir​(String tableDataDir)

      • isErrorOnColumnBuildFailure

        public boolean isErrorOnColumnBuildFailure()

      • setErrorOnColumnBuildFailure

        public void setErrorOnColumnBuildFailure​(boolean errorOnColumnBuildFailure)

      • getTableDataDir

        public String getTableDataDir()

      • setSegmentTier

        public void setSegmentTier​(String segmentTier)

      • getSegmentTier

        public String getSegmentTier()

      • getAllKnownColumns

        public Set<String> getAllKnownColumns()
        Returns a subset of the columns on the table. When getSchema() is defined, the subset is equal the columns on the schema. In other cases, this method tries its bests to get the columns from other attributes like getTableConfig(), which may also not be defined or may not be complete.

      • addKnownColumns

        public void addKnownColumns​(Set<String> columns)