Package org.apache.pinot.segment.local.segment.index.loader

Class SegmentPreProcessor

java.lang.Object

org.apache.pinot.segment.local.segment.index.loader.SegmentPreProcessor

All Implemented Interfaces:

AutoCloseable

public class SegmentPreProcessor
extends Object
implements AutoCloseable

Use mmap to load the segment and perform all pre-processing steps. (This can be slow)

Pre-processing steps include:

• Use InvertedIndexHandler to create inverted indices
• Use DefaultColumnHandler to update auto-generated default columns
• Use ColumnMinMaxValueGenerator to add min/max value to column metadata

Constructor Summary

Constructors

Constructor	Description
SegmentPreProcessor(SegmentDirectory segmentDirectory, IndexLoadingConfig indexLoadingConfig, Schema schema)

Method Summary

Modifier and Type	Method	Description
void	close()
boolean	needProcess()	This method checks if there is any discrepancy between the segment and current table config and schema.
void	process()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SegmentPreProcessor

public SegmentPreProcessor(SegmentDirectory segmentDirectory,
                           IndexLoadingConfig indexLoadingConfig,
                           @Nullable
                           Schema schema)

Method Detail

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

process

public void process()
             throws Exception

Throws:

Exception

needProcess

public boolean needProcess()
                    throws Exception

This method checks if there is any discrepancy between the segment and current table config and schema. If so, it returns true indicating the segment needs to be reprocessed. Right now, the default columns, all types of indices and column min/max values are checked against what's set in table config and schema.

Throws:

Exception