Package org.apache.pinot.segment.local.segment.index.loader

Class InvertedIndexAndDictionaryBasedForwardIndexCreator

java.lang.Object

org.apache.pinot.segment.local.segment.index.loader.InvertedIndexAndDictionaryBasedForwardIndexCreator

All Implemented Interfaces:

AutoCloseable

public class InvertedIndexAndDictionaryBasedForwardIndexCreator
extends Object
implements AutoCloseable

Helper classed used by the SegmentPreProcessor to generate the forward index from inverted index and dictionary when the forward index is enabled for columns where it was previously disabled. This is also invoked by the IndexHandler code in scenarios where the forward index needs to be temporarily created to generate other indexes for the given column. In such cases the forward index will be cleaned up after the IndexHandler code completes. For multi-value columns the following invariants cannot be maintained: - Ordering of elements within a given multi-value row. This will always be a limitation. TODO: Currently for multi-value columns generating the forward index can lead to a data loss as frequency information is not available for repeats within a given row. This needs to be addressed by tracking the frequency data as part of an on-disk structure when forward index is disabled for a column. TODO (index-spi): Rename this class, as it is not an implementation of IndexCreator.

Constructor Summary

Constructors

Constructor	Description
InvertedIndexAndDictionaryBasedForwardIndexCreator(String columnName, SegmentDirectory segmentDirectory, boolean dictionaryEnabled, ForwardIndexConfig fwdConf, SegmentDirectory.Writer segmentWriter, boolean isTemporaryForwardIndex)

Method Summary

Modifier and Type	Method	Description
void	close()
void	regenerateForwardIndex()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InvertedIndexAndDictionaryBasedForwardIndexCreator

public InvertedIndexAndDictionaryBasedForwardIndexCreator(String columnName,
                                                          SegmentDirectory segmentDirectory,
                                                          boolean dictionaryEnabled,
                                                          ForwardIndexConfig fwdConf,
                                                          SegmentDirectory.Writer segmentWriter,
                                                          boolean isTemporaryForwardIndex)
                                                   throws IOException

Throws:

IOException

Method Detail

regenerateForwardIndex

public void regenerateForwardIndex()
                            throws IOException

Throws:

IOException

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception