org.apache.druid.frame.processor

Class SuperSorterProgressTracker

java.lang.Object

org.apache.druid.frame.processor.SuperSorterProgressTracker

public class SuperSorterProgressTracker
extends Object

A thread-safe class that keeps track of the progress of an n-way, multilevel merge sort. The logic for this class is coupled to the implementation of SuperSorter, which it is designed to track. It keeps a track of the following things: 1. Total levels to merge (Cannnot be modified once set) 2. Total mergers to merge in each level 3. Total merged batches in each level There are a few states in which this progress tracker can be in (depending on the SuperSorter's progres): 1. Unknown everything | totalMergingLevels = -1, mergersForUltimateLevel = UNKNOWN_TOTAL 2. Known total levels, but unknown output partitions | totalMergingLevels = set, mergersForUltimateLevelSet = UNKNOWN_TOTAL 3. Unknown total levels, but known output partitions | totalMergingLevels = -1, mergersForUltimateLevelSet != UNKNOWN_TOTAL 4. Known everything | totalMergingLevels = set, mergersForUltimateLevelSet != UNKNOWN_TOTAL Progress can only be quantified if the total levels are known snapshot function does the housekeeping to hide this operational complexity away from classes other than SuperSorter and SuperSorterProgressTracker

Constructor Summary

Constructors

Constructor and Description
SuperSorterProgressTracker()

Method Summary

Modifier and Type	Method and Description
void	addMergedBatchesForLevel(int level, long additionalMergedBatches) This method is designed to be called during the course of the sorting.
void	markTriviallyComplete() If the SuperSorter is trivially done without doing any work (for eg - empty input), the tracker can be marked as trivially complete.
void	setTotalMergersForLevel(int level, long totalMergers) Sets the total mergers for a level.
void	setTotalMergersForUltimateLevel(long totalMergersForUltimateLevel) Sets the number of mergers in the ultimate level (number of mergers = number of output partitions).
void	setTotalMergingLevels(int totalMergingLevels) Set total merging levels for the SuperSorter it is tracking.
SuperSorterProgressSnapshot	snapshot()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SuperSorterProgressTracker

public SuperSorterProgressTracker()

Method Detail

setTotalMergingLevels

public void setTotalMergingLevels(int totalMergingLevels)

Set total merging levels for the SuperSorter it is tracking. Can be set only once

setTotalMergersForLevel

public void setTotalMergersForLevel(int level,
                                    long totalMergers)

Sets the total mergers for a level. Can be set only once, except for the ultimate level (if total levels are known) because they get overridden by totalMergersForUltimateLevel

setTotalMergersForUltimateLevel

public void setTotalMergersForUltimateLevel(long totalMergersForUltimateLevel)

Sets the number of mergers in the ultimate level (number of mergers = number of output partitions). Can only be set once

addMergedBatchesForLevel

public void addMergedBatchesForLevel(int level,
                                     long additionalMergedBatches)

This method is designed to be called during the course of the sorting. The batches once merged for a particular level can be marked as such through this.

markTriviallyComplete

public void markTriviallyComplete()

If the SuperSorter is trivially done without doing any work (for eg - empty input), the tracker can be marked as trivially complete. Once a tracker is marked as complete, the snapshots will always report back the progress digest as 1. Any modification to the state of the tracker (eg: calling setTotalMergersForLevel()) would proceed as regular, but

snapshot

public SuperSorterProgressSnapshot snapshot()

Returns:

Aggregates the information present in the tracker object and returns a SuperSorterProgressSnapshot representing the same information