org.apache.druid.metadata

Interface SegmentsMetadataManager

All Known Implementing Classes:

SqlSegmentsMetadataManager

public interface SegmentsMetadataManager

The difference between this class and org.apache.druid.sql.calcite.schema.MetadataSegmentView is that this class resides in Coordinator's memory, while org.apache.druid.sql.calcite.schema.MetadataSegmentView resides in Broker's memory.

Method Summary

Modifier and Type	Method and Description
Collection<ImmutableDruidDataSource>	getImmutableDataSourcesWithAllUsedSegments() Returns a set of ImmutableDruidDataSource objects containing information about all used segments.
ImmutableDruidDataSource	getImmutableDataSourceWithUsedSegments(String dataSource) If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object.
Set<SegmentId>	getOvershadowedSegments() Returns a set of overshadowed segment ids.
DataSourcesSnapshot	getSnapshotOfDataSourcesWithAllUsedSegments() Returns a snapshot of DruidDataSources and overshadowed segments
List<org.joda.time.Interval>	getUnusedSegmentIntervals(String dataSource, org.joda.time.DateTime maxEndTime, int limit) Returns top N unused segment intervals with the end time no later than the specified maxEndTime when ordered by segment start time, end time.
boolean	isPollingDatabasePeriodically()
com.google.common.base.Optional<Iterable<DataSegment>>	iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval(String datasource, org.joda.time.Interval interval, boolean requiresLatest) Returns an iterable to go over all used and non-overshadowed segments of given data sources over given interval.
Iterable<DataSegment>	iterateAllUsedSegments() Returns an iterable to go over all segments in all data sources.
int	markAsUnusedAllSegmentsInDataSource(String dataSource) Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused).
int	markAsUnusedSegmentsInInterval(String dataSource, org.joda.time.Interval interval)
int	markAsUsedAllNonOvershadowedSegmentsInDataSource(String dataSource) Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used).
int	markAsUsedNonOvershadowedSegments(String dataSource, Set<String> segmentIds)
int	markAsUsedNonOvershadowedSegmentsInInterval(String dataSource, org.joda.time.Interval interval)
boolean	markSegmentAsUnused(String segmentId) Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as unused), false otherwise.
boolean	markSegmentAsUsed(String segmentId) Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise.
int	markSegmentsAsUnused(String dataSource, Set<String> segmentIds)
void	poll()
Set<String>	retrieveAllDataSourceNames() Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not.
void	startPollingDatabasePeriodically()
void	stopPollingDatabasePeriodically()

Method Detail

startPollingDatabasePeriodically

void startPollingDatabasePeriodically()

stopPollingDatabasePeriodically

void stopPollingDatabasePeriodically()

isPollingDatabasePeriodically

boolean isPollingDatabasePeriodically()

markAsUsedAllNonOvershadowedSegmentsInDataSource

int markAsUsedAllNonOvershadowedSegmentsInDataSource(String dataSource)

Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used). If the call results in a database error, an exception is relayed to the caller.

markAsUsedNonOvershadowedSegmentsInInterval

int markAsUsedNonOvershadowedSegmentsInInterval(String dataSource,
                                                org.joda.time.Interval interval)

markAsUsedNonOvershadowedSegments

int markAsUsedNonOvershadowedSegments(String dataSource,
                                      Set<String> segmentIds)
                               throws UnknownSegmentIdsException

Throws:

UnknownSegmentIdsException

markSegmentAsUsed

boolean markSegmentAsUsed(String segmentId)

Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise. If the call results in a database error, an exception is relayed to the caller.

markAsUnusedAllSegmentsInDataSource

int markAsUnusedAllSegmentsInDataSource(String dataSource)

Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused). If the call results in a database error, an exception is relayed to the caller.

markAsUnusedSegmentsInInterval

int markAsUnusedSegmentsInInterval(String dataSource,
                                   org.joda.time.Interval interval)

markSegmentsAsUnused

int markSegmentsAsUnused(String dataSource,
                         Set<String> segmentIds)

markSegmentAsUnused

boolean markSegmentAsUnused(String segmentId)

Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as unused), false otherwise. If the call results in a database error, an exception is relayed to the caller.

getImmutableDataSourceWithUsedSegments

@Nullable
ImmutableDruidDataSource getImmutableDataSourceWithUsedSegments(String dataSource)

If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object. If there are no used segments belonging to the given data source this method returns null.

getImmutableDataSourcesWithAllUsedSegments

Collection<ImmutableDruidDataSource> getImmutableDataSourcesWithAllUsedSegments()

Returns a set of ImmutableDruidDataSource objects containing information about all used segments. ImmutableDruidDataSource objects in the returned collection are unique. If there are no used segments, this method returns an empty collection.

getOvershadowedSegments

Set<SegmentId> getOvershadowedSegments()

Returns a set of overshadowed segment ids.

getSnapshotOfDataSourcesWithAllUsedSegments

DataSourcesSnapshot getSnapshotOfDataSourcesWithAllUsedSegments()

Returns a snapshot of DruidDataSources and overshadowed segments

iterateAllUsedSegments

Iterable<DataSegment> iterateAllUsedSegments()

Returns an iterable to go over all segments in all data sources. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.

iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval

com.google.common.base.Optional<Iterable<DataSegment>> iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval(String datasource,
                                                                                                                  org.joda.time.Interval interval,
                                                                                                                  boolean requiresLatest)

Returns an iterable to go over all used and non-overshadowed segments of given data sources over given interval. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times. If is true then a force metadatastore poll will be triggered. This can cause a longer response time but will ensure that the latest segment information (at the time this method is called) is returned. If is false then segment information from stale snapshot of up to the last periodic poll period SqlSegmentsMetadataManager.periodicPollDelay will be used.

retrieveAllDataSourceNames

Set<String> retrieveAllDataSourceNames()

Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not. If there are no segments in the database, returns an empty set. Performance warning: this method makes a query into the database. This method might return a different set of data source names than may be observed via getImmutableDataSourcesWithAllUsedSegments() method. This method will include a data source name even if there are no used segments belonging to it, while getImmutableDataSourcesWithAllUsedSegments() won't return such a data source.

getUnusedSegmentIntervals

List<org.joda.time.Interval> getUnusedSegmentIntervals(String dataSource,
                                                       org.joda.time.DateTime maxEndTime,
                                                       int limit)

Returns top N unused segment intervals with the end time no later than the specified maxEndTime when ordered by segment start time, end time.

poll

void poll()