Package org.apache.druid.segment
Interface IndexMerger
-
- All Known Implementing Classes:
IndexMergerV9
public interface IndexMerger
-
-
Field Summary
Fields Modifier and Type Field Description static intINVALID_ROWstatic Loggerlogstatic SerializerUtilsSERIALIZER_UTILSstatic intUNLIMITED_MAX_COLUMNS_TO_MERGE
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description static List<String>getLexicographicMergedDimensions(List<IndexableAdapter> indexes)static List<String>getLongestSharedDimOrder(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)static List<String>getMergedDimensions(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)static List<String>getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes, DimensionsSpec dimensionsSpec)static booleanisDimensionOrderingValid(List<IndexableAdapter> indexes, Iterable<String> orderingCandidate)static TransformableRowIteratormakeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator, ColumnValueSelector[] convertedDimensionSelectors, ColumnValueSelector[] convertedMarkedDimensionSelectors)Filemerge(List<IndexableAdapter> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, DimensionsSpec dimensionsSpec, IndexSpec indexSpec, int maxColumnsToMerge)Only used as a convenience method in tests.static <T extends Comparable<? super T>>
ArrayList<T>mergeIndexed(List<Iterable<T>> indexedLists)default FilemergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge)Merge a collection ofQueryableIndex.FilemergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, DimensionsSpec dimensionsSpec, File outDir, IndexSpec indexSpec, IndexSpec indexSpecForIntermediatePersists, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge)Merge a collection ofQueryableIndex.default Filepersist(IncrementalIndex index, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)Equivalent topersist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory)without a progress indicator and with interval set toIncrementalIndex.getInterval().Filepersist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)Persist an IncrementalIndex to disk in such a way that it can be loaded back up as aQueryableIndex.default Filepersist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)Equivalent topersist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory)without a progress indicator.static List<IndexableAdapter>toIndexableAdapters(List<QueryableIndex> indexes)static TransformableRowIteratortoMergedIndexRowIterator(TransformableRowIterator sourceRowIterator, int indexNumber, List<DimensionMergerV9> mergers)This method appliesDimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector)to all dimension column selectors of the given sourceRowIterator, using the given index number.
-
-
-
Field Detail
-
log
static final Logger log
-
SERIALIZER_UTILS
static final SerializerUtils SERIALIZER_UTILS
-
INVALID_ROW
static final int INVALID_ROW
- See Also:
- Constant Field Values
-
UNLIMITED_MAX_COLUMNS_TO_MERGE
static final int UNLIMITED_MAX_COLUMNS_TO_MERGE
- See Also:
- Constant Field Values
-
-
Method Detail
-
getMergedDimensionsFromQueryableIndexes
static List<String> getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes, @Nullable DimensionsSpec dimensionsSpec)
-
toIndexableAdapters
static List<IndexableAdapter> toIndexableAdapters(List<QueryableIndex> indexes)
-
getMergedDimensions
static List<String> getMergedDimensions(List<IndexableAdapter> indexes, @Nullable DimensionsSpec dimensionsSpec)
-
getLongestSharedDimOrder
@Nullable static List<String> getLongestSharedDimOrder(List<IndexableAdapter> indexes, @Nullable DimensionsSpec dimensionsSpec)
-
isDimensionOrderingValid
static boolean isDimensionOrderingValid(List<IndexableAdapter> indexes, Iterable<String> orderingCandidate)
-
getLexicographicMergedDimensions
static List<String> getLexicographicMergedDimensions(List<IndexableAdapter> indexes)
-
mergeIndexed
static <T extends Comparable<? super T>> ArrayList<T> mergeIndexed(List<Iterable<T>> indexedLists)
-
persist
default File persist(IncrementalIndex index, File outDir, IndexSpec indexSpec, @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) throws IOException
Equivalent topersist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory)without a progress indicator and with interval set toIncrementalIndex.getInterval().- Throws:
IOException
-
persist
default File persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) throws IOException
Equivalent topersist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory)without a progress indicator.- Throws:
IOException
-
persist
File persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, ProgressIndicator progress, @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) throws IOException
Persist an IncrementalIndex to disk in such a way that it can be loaded back up as aQueryableIndex. This is *not* thread-safe and havoc will ensue if this is called and writes are still occurring on the IncrementalIndex object.- Parameters:
index- the IncrementalIndex to persistdataInterval- the Interval that the data represents. Typically, this is the same as the interval from the correspondingSegmentId.outDir- the directory to persist the data toindexSpec- storage and compression optionsprogress- an object that will receive progress updatessegmentWriteOutMediumFactory- controls allocation of temporary data structures- Returns:
- the index output directory
- Throws:
IOException- if an IO error occurs persisting the index
-
mergeQueryableIndex
default File mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, IndexSpec indexSpec, @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge) throws IOException
Merge a collection ofQueryableIndex. Only used as a convenience method in tests. In production code, use the full versionmergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int).- Throws:
IOException
-
mergeQueryableIndex
File mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, @Nullable DimensionsSpec dimensionsSpec, File outDir, IndexSpec indexSpec, IndexSpec indexSpecForIntermediatePersists, ProgressIndicator progress, @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge) throws IOException
Merge a collection ofQueryableIndex.- Throws:
IOException
-
merge
File merge(List<IndexableAdapter> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, DimensionsSpec dimensionsSpec, IndexSpec indexSpec, int maxColumnsToMerge) throws IOException
Only used as a convenience method in tests. In production code, to merge multipleQueryableIndex, usemergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int). To merge multipleIncrementalIndex, call one of thepersist(org.apache.druid.segment.incremental.IncrementalIndex, java.io.File, org.apache.druid.segment.IndexSpec, org.apache.druid.segment.writeout.SegmentWriteOutMediumFactory)methods and then merge the resultingQueryableIndex.- Throws:
IOException
-
toMergedIndexRowIterator
static TransformableRowIterator toMergedIndexRowIterator(TransformableRowIterator sourceRowIterator, int indexNumber, List<DimensionMergerV9> mergers)
This method appliesDimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector)to all dimension column selectors of the given sourceRowIterator, using the given index number.
-
makeRowIteratorWithConvertedDimensionColumns
static TransformableRowIterator makeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator, ColumnValueSelector[] convertedDimensionSelectors, ColumnValueSelector[] convertedMarkedDimensionSelectors)
-
-