org.apache.druid.segment

Interface IndexMerger

All Known Implementing Classes:

IndexMergerV9

public interface IndexMerger

Field Summary

Fields

Modifier and Type	Field and Description
static int	INVALID_ROW
static Logger	log
static SerializerUtils	SERIALIZER_UTILS
static int	UNLIMITED_MAX_COLUMNS_TO_MERGE

Method Summary

Modifier and Type	Method and Description
static List<String>	getLexicographicMergedDimensions(List<IndexableAdapter> indexes)
static List<String>	getLongestSharedDimOrder(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)
static List<String>	getMergedDimensions(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)
static List<String>	getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes, DimensionsSpec dimensionsSpec)
static boolean	isDimensionOrderingValid(List<IndexableAdapter> indexes, Iterable<String> orderingCandidate)
static TransformableRowIterator	makeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator, ColumnValueSelector[] convertedDimensionSelectors, ColumnValueSelector[] convertedMarkedDimensionSelectors)
File	merge(List<IndexableAdapter> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, DimensionsSpec dimensionsSpec, IndexSpec indexSpec, int maxColumnsToMerge) Only used as a convenience method in tests.
static <T extends Comparable<? super T>> ArrayList<T>	mergeIndexed(List<Iterable<T>> indexedLists)
File	mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, DimensionsSpec dimensionsSpec, File outDir, IndexSpec indexSpec, IndexSpec indexSpecForIntermediatePersists, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge) Merge a collection of QueryableIndex.
default File	mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge) Merge a collection of QueryableIndex.
default File	persist(IncrementalIndex index, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator and with interval set to IncrementalIndex.getInterval().
File	persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) Persist an IncrementalIndex to disk in such a way that it can be loaded back up as a QueryableIndex.
default File	persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory) Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator.
static List<IndexableAdapter>	toIndexableAdapters(List<QueryableIndex> indexes)
static TransformableRowIterator	toMergedIndexRowIterator(TransformableRowIterator sourceRowIterator, int indexNumber, List<DimensionMergerV9> mergers) This method applies DimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector) to all dimension column selectors of the given sourceRowIterator, using the given index number.

Field Detail

log

static final Logger log

SERIALIZER_UTILS

static final SerializerUtils SERIALIZER_UTILS

INVALID_ROW

static final int INVALID_ROW

See Also:

Constant Field Values

UNLIMITED_MAX_COLUMNS_TO_MERGE

static final int UNLIMITED_MAX_COLUMNS_TO_MERGE

See Also:

Constant Field Values

Method Detail

getMergedDimensionsFromQueryableIndexes

static List<String> getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes,
                                                            @Nullable
                                                            DimensionsSpec dimensionsSpec)

toIndexableAdapters

static List<IndexableAdapter> toIndexableAdapters(List<QueryableIndex> indexes)

getMergedDimensions

static List<String> getMergedDimensions(List<IndexableAdapter> indexes,
                                        @Nullable
                                        DimensionsSpec dimensionsSpec)

getLongestSharedDimOrder

@Nullable
static List<String> getLongestSharedDimOrder(List<IndexableAdapter> indexes,
                                                       @Nullable
                                                       DimensionsSpec dimensionsSpec)

isDimensionOrderingValid

static boolean isDimensionOrderingValid(List<IndexableAdapter> indexes,
                                        Iterable<String> orderingCandidate)

getLexicographicMergedDimensions

static List<String> getLexicographicMergedDimensions(List<IndexableAdapter> indexes)

mergeIndexed

static <T extends Comparable<? super T>> ArrayList<T> mergeIndexed(List<Iterable<T>> indexedLists)

persist

default File persist(IncrementalIndex index,
                     File outDir,
                     IndexSpec indexSpec,
                     @Nullable
                     SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
              throws IOException

Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator and with interval set to IncrementalIndex.getInterval().

Throws:

IOException

persist

default File persist(IncrementalIndex index,
                     org.joda.time.Interval dataInterval,
                     File outDir,
                     IndexSpec indexSpec,
                     @Nullable
                     SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
              throws IOException

Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator.

Throws:

IOException

persist

File persist(IncrementalIndex index,
             org.joda.time.Interval dataInterval,
             File outDir,
             IndexSpec indexSpec,
             ProgressIndicator progress,
             @Nullable
             SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
      throws IOException

Persist an IncrementalIndex to disk in such a way that it can be loaded back up as a QueryableIndex. This is *not* thread-safe and havoc will ensue if this is called and writes are still occurring on the IncrementalIndex object.

Parameters:

index - the IncrementalIndex to persist

dataInterval - the Interval that the data represents. Typically, this is the same as the interval from the corresponding SegmentId.

outDir - the directory to persist the data to

indexSpec - storage and compression options

progress - an object that will receive progress updates

segmentWriteOutMediumFactory - controls allocation of temporary data structures

Returns:

the index output directory

Throws:

IOException - if an IO error occurs persisting the index

mergeQueryableIndex

default File mergeQueryableIndex(List<QueryableIndex> indexes,
                                 boolean rollup,
                                 AggregatorFactory[] metricAggs,
                                 File outDir,
                                 IndexSpec indexSpec,
                                 @Nullable
                                 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory,
                                 int maxColumnsToMerge)
                          throws IOException

Merge a collection of QueryableIndex. Only used as a convenience method in tests. In production code, use the full version mergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int).

Throws:

IOException

mergeQueryableIndex

File mergeQueryableIndex(List<QueryableIndex> indexes,
                         boolean rollup,
                         AggregatorFactory[] metricAggs,
                         @Nullable
                         DimensionsSpec dimensionsSpec,
                         File outDir,
                         IndexSpec indexSpec,
                         IndexSpec indexSpecForIntermediatePersists,
                         ProgressIndicator progress,
                         @Nullable
                         SegmentWriteOutMediumFactory segmentWriteOutMediumFactory,
                         int maxColumnsToMerge)
                  throws IOException

Merge a collection of QueryableIndex.

Throws:

IOException

merge

File merge(List<IndexableAdapter> indexes,
           boolean rollup,
           AggregatorFactory[] metricAggs,
           File outDir,
           DimensionsSpec dimensionsSpec,
           IndexSpec indexSpec,
           int maxColumnsToMerge)
    throws IOException

Only used as a convenience method in tests. In production code, to merge multiple QueryableIndex, use mergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int). To merge multiple IncrementalIndex, call one of the persist(org.apache.druid.segment.incremental.IncrementalIndex, java.io.File, org.apache.druid.segment.IndexSpec, org.apache.druid.segment.writeout.SegmentWriteOutMediumFactory) methods and then merge the resulting QueryableIndex.

Throws:

IOException

toMergedIndexRowIterator

static TransformableRowIterator toMergedIndexRowIterator(TransformableRowIterator sourceRowIterator,
                                                         int indexNumber,
                                                         List<DimensionMergerV9> mergers)

This method applies DimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector) to all dimension column selectors of the given sourceRowIterator, using the given index number.

makeRowIteratorWithConvertedDimensionColumns

static TransformableRowIterator makeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator,
                                                                             ColumnValueSelector[] convertedDimensionSelectors,
                                                                             ColumnValueSelector[] convertedMarkedDimensionSelectors)