Package org.apache.druid.metadata

Interface SegmentsMetadataManager

  • All Known Implementing Classes:
    SqlSegmentsMetadataManager
    public interface SegmentsMetadataManager
    The difference between this class and org.apache.druid.sql.calcite.schema.MetadataSegmentView is that this class resides in Coordinator's memory, while org.apache.druid.sql.calcite.schema.MetadataSegmentView resides in Broker's memory.

    • Method Detail

      • startPollingDatabasePeriodically

        void startPollingDatabasePeriodically()

      • stopPollingDatabasePeriodically

        void stopPollingDatabasePeriodically()

      • isPollingDatabasePeriodically

        boolean isPollingDatabasePeriodically()

      • markAsUsedAllNonOvershadowedSegmentsInDataSource

        int markAsUsedAllNonOvershadowedSegmentsInDataSource​(String dataSource)
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used). If the call results in a database error, an exception is relayed to the caller.

      • markAsUsedNonOvershadowedSegmentsInInterval

        int markAsUsedNonOvershadowedSegmentsInInterval​(String dataSource,
                                                org.joda.time.Interval interval)

      • markSegmentAsUsed

        boolean markSegmentAsUsed​(String segmentId)
        Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise. If the call results in a database error, an exception is relayed to the caller.

      • markAsUnusedAllSegmentsInDataSource

        int markAsUnusedAllSegmentsInDataSource​(String dataSource)
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused). If the call results in a database error, an exception is relayed to the caller.

      • markAsUnusedSegmentsInInterval

        int markAsUnusedSegmentsInInterval​(String dataSource,
                                   org.joda.time.Interval interval)

      • markSegmentsAsUnused

        int markSegmentsAsUnused​(Set<org.apache.druid.timeline.SegmentId> segmentIds)

      • markSegmentAsUnused

        boolean markSegmentAsUnused​(org.apache.druid.timeline.SegmentId segmentId)
        Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as unused), false otherwise. If the call results in a database error, an exception is relayed to the caller.

      • getImmutableDataSourceWithUsedSegments

        @Nullable
ImmutableDruidDataSource getImmutableDataSourceWithUsedSegments​(String dataSource)
        If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object. If there are no used segments belonging to the given data source this method returns null.

      • getSnapshotOfDataSourcesWithAllUsedSegments

        DataSourcesSnapshot getSnapshotOfDataSourcesWithAllUsedSegments()
        Returns a snapshot of DruidDataSources and overshadowed segments

      • iterateAllUsedSegments

        Iterable<org.apache.druid.timeline.DataSegment> iterateAllUsedSegments()
        Returns an iterable to go over all segments in all data sources. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.

      • iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval

        com.google.common.base.Optional<Iterable<org.apache.druid.timeline.DataSegment>> iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval​(String datasource,
                                                                                                                                            org.joda.time.Interval interval,
                                                                                                                                            boolean requiresLatest)
        Returns an iterable to go over all used and non-overshadowed segments of given data sources over given interval. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times. If {@param requiresLatest} is true then a force metadatastore poll will be triggered. This can cause a longer response time but will ensure that the latest segment information (at the time this method is called) is returned. If {@param requiresLatest} is false then segment information from stale snapshot of up to the last periodic poll period SqlSegmentsMetadataManager.periodicPollDelay will be used.

      • iterateAllUnusedSegmentsForDatasource

        Iterable<DataSegmentPlus> iterateAllUnusedSegmentsForDatasource​(String datasource,
                                                                @Nullable
                                                                org.joda.time.Interval interval,
                                                                @Nullable
                                                                Integer limit,
                                                                @Nullable
                                                                String lastSegmentId,
                                                                @Nullable
                                                                SortOrder sortOrder)
        Returns an iterable to go over un-used segments and their associated metadata for a given datasource over an optional interval. The order in which segments are iterated is from earliest start-time, with ties being broken with earliest end-time first. Note: the iteration may not be as trivially cheap as for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.
        Parameters:
        datasource - the name of the datasource.
        interval - an optional interval to search over. If none is specified, Intervals.ETERNITY
        limit - an optional maximum number of results to return. If none is specified, the results are not limited.
        lastSegmentId - an optional last segment id from which to search for results. All segments returned are > this segment lexigraphically if sortOrder is null or SortOrder.ASC, or < this segment lexigraphically if sortOrder is SortOrder.DESC. If none is specified, no such filter is used.
        sortOrder - an optional order with which to return the matching segments by id, start time, end time. If none is specified, the order of the results is not guarenteed.

      • retrieveAllDataSourceNames

        Set<String> retrieveAllDataSourceNames()
        Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not. If there are no segments in the database, returns an empty set. Performance warning: this method makes a query into the database. This method might return a different set of data source names than may be observed via getImmutableDataSourcesWithAllUsedSegments() method. This method will include a data source name even if there are no used segments belonging to it, while getImmutableDataSourcesWithAllUsedSegments() won't return such a data source.

      • getUnusedSegmentIntervals

        List<org.joda.time.Interval> getUnusedSegmentIntervals​(String dataSource,
                                                       org.joda.time.DateTime minStartTime,
                                                       org.joda.time.DateTime maxEndTime,
                                                       int limit,
                                                       org.joda.time.DateTime maxUsedStatusLastUpdatedTime)
        Returns a list of up to limit unused segment intervals for the specified datasource. Segments are filtered based on the following criteria:
      • The start time of the segment must be no earlier than the specified minStartTime (if not null).
      • The end time of the segment must be no later than the specified maxEndTime.
      • The used_status_last_updated time of the segment must be no later than maxUsedStatusLastUpdatedTime. Segments that have no used_status_last_updated time (due to an upgrade from legacy Druid) will have maxUsedStatusLastUpdatedTime ignored.

        • The list of intervals is ordered by segment start time and then by end time.