org.apache.druid.server

Class SegmentManager

java.lang.Object

org.apache.druid.server.SegmentManager

public class SegmentManager
extends Object

This class is responsible for managing data sources and their states like timeline, total segment size, and number of segments. All public methods of this class must be thread-safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SegmentManager.DataSourceState Represent the state of a data source including the timeline, total segment size, and number of segments.

Constructor Summary

Constructors

Constructor and Description
SegmentManager(SegmentLoader segmentLoader)

Method Summary

Modifier and Type	Method and Description
void	dropSegment(DataSegment segment)
Map<String,Long>	getAverageRowCountForDatasource()
Map<String,Long>	getDataSourceCounts() Returns a map of dataSource to the number of segments managed by this segmentManager.
Set<String>	getDataSourceNames()
Map<String,Long>	getDataSourceSizes() Returns a map of dataSource to the total byte size of segments managed by this segmentManager.
Optional<Stream<ReferenceCountingIndexedTable>>	getIndexedTables(DataSourceAnalysis analysis) Returns the collection of IndexedTable for the entire timeline (since join conditions do not currently consider the queries intervals), if the timeline exists for each of its segments that are joinable.
Map<String,SegmentRowCountDistribution>	getRowCountDistribution()
Optional<VersionedIntervalTimeline<String,ReferenceCountingSegment>>	getTimeline(DataSourceAnalysis analysis) Returns the timeline for a datasource, if it exists.
boolean	hasIndexedTables(String dataSourceName)
boolean	loadSegment(DataSegment segment, boolean lazy, SegmentLazyLoadFailCallback loadFailed)
boolean	loadSegment(DataSegment segment, boolean lazy, SegmentLazyLoadFailCallback loadFailed, ExecutorService loadSegmentIntoPageCacheExec) Load a single segment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SegmentManager

@Inject
public SegmentManager(SegmentLoader segmentLoader)

Method Detail

getDataSourceSizes

public Map<String,Long> getDataSourceSizes()

Returns a map of dataSource to the total byte size of segments managed by this segmentManager. This method should be used carefully because the returned map might be different from the actual data source states.

Returns:

a map of dataSources and their total byte sizes

getAverageRowCountForDatasource

public Map<String,Long> getAverageRowCountForDatasource()

getRowCountDistribution

public Map<String,SegmentRowCountDistribution> getRowCountDistribution()

getDataSourceNames

public Set<String> getDataSourceNames()

getDataSourceCounts

public Map<String,Long> getDataSourceCounts()

Returns a map of dataSource to the number of segments managed by this segmentManager. This method should be carefully because the returned map might be different from the actual data source states.

Returns:

a map of dataSources and number of segments

getTimeline

public Optional<VersionedIntervalTimeline<String,ReferenceCountingSegment>> getTimeline(DataSourceAnalysis analysis)

Returns the timeline for a datasource, if it exists. The analysis object passed in must represent a scan-based datasource of a single table.

Parameters:

analysis - data source analysis information

Returns:

timeline, if it exists

Throws:

IllegalStateException - if 'analysis' does not represent a scan-based datasource of a single table

getIndexedTables

public Optional<Stream<ReferenceCountingIndexedTable>> getIndexedTables(DataSourceAnalysis analysis)

Returns the collection of IndexedTable for the entire timeline (since join conditions do not currently consider the queries intervals), if the timeline exists for each of its segments that are joinable.

hasIndexedTables

public boolean hasIndexedTables(String dataSourceName)

loadSegment

public boolean loadSegment(DataSegment segment,
                           boolean lazy,
                           SegmentLazyLoadFailCallback loadFailed)
                    throws SegmentLoadingException

Throws:

SegmentLoadingException

loadSegment

public boolean loadSegment(DataSegment segment,
                           boolean lazy,
                           SegmentLazyLoadFailCallback loadFailed,
                           ExecutorService loadSegmentIntoPageCacheExec)
                    throws SegmentLoadingException

Load a single segment.

Parameters:

segment - segment to load

lazy - whether to lazy load columns metadata

loadFailed - callBack to execute when segment lazy load failed

loadSegmentIntoPageCacheExec - If null is specified, the default thread pool in segment loader to load segments into page cache on download will be used. You can specify a dedicated thread pool of larger capacity when this function is called during historical process bootstrap to speed up initial loading.

Returns:

true if the segment was newly loaded, false if it was already loaded

Throws:

SegmentLoadingException - if the segment cannot be loaded

dropSegment

public void dropSegment(DataSegment segment)