Package org.apache.druid.metadata

Class SqlSegmentsMetadataManager

    • Constructor Detail

      • SqlSegmentsMetadataManager

        @Inject
public SqlSegmentsMetadataManager​(com.fasterxml.jackson.databind.ObjectMapper jsonMapper,
                                  com.google.common.base.Supplier<SegmentsMetadataManagerConfig> config,
                                  com.google.common.base.Supplier<org.apache.druid.metadata.MetadataStorageTablesConfig> dbTables,
                                  SQLMetadataConnector connector)

    • Method Detail

      • start

        public void start()
        Don't confuse this method with startPollingDatabasePeriodically(). This is a lifecycle starting method to be executed just once for an instance of SqlSegmentsMetadataManager.

      • stop

        public void stop()
        Don't confuse this method with stopPollingDatabasePeriodically(). This is a lifecycle stopping method to be executed just once for an instance of SqlSegmentsMetadataManager.

      • markSegmentAsUsed

        public boolean markSegmentAsUsed​(String segmentId)
        Description copied from interface: SegmentsMetadataManager
        Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise. If the call results in a database error, an exception is relayed to the caller.
        Specified by:
        markSegmentAsUsed in interface SegmentsMetadataManager

      • markAsUsedAllNonOvershadowedSegmentsInDataSource

        public int markAsUsedAllNonOvershadowedSegmentsInDataSource​(String dataSource)
        Description copied from interface: SegmentsMetadataManager
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used). If the call results in a database error, an exception is relayed to the caller.
        Specified by:
        markAsUsedAllNonOvershadowedSegmentsInDataSource in interface SegmentsMetadataManager

      • markAsUnusedAllSegmentsInDataSource

        public int markAsUnusedAllSegmentsInDataSource​(String dataSource)
        Description copied from interface: SegmentsMetadataManager
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused). If the call results in a database error, an exception is relayed to the caller.
        Specified by:
        markAsUnusedAllSegmentsInDataSource in interface SegmentsMetadataManager

      • markSegmentAsUnused

        public boolean markSegmentAsUnused​(org.apache.druid.timeline.SegmentId segmentId)
        This method does not update dataSourcesSnapshot, see the comments in doPoll() about snapshot update. The update of the segment's state will be reflected after the next SqlSegmentsMetadataManager.DatabasePoll.
        Specified by:
        markSegmentAsUnused in interface SegmentsMetadataManager

      • iterateAllUsedSegments

        public Iterable<org.apache.druid.timeline.DataSegment> iterateAllUsedSegments()
        Description copied from interface: SegmentsMetadataManager
        Returns an iterable to go over all segments in all data sources. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.
        Specified by:
        iterateAllUsedSegments in interface SegmentsMetadataManager

      • iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval

        public com.google.common.base.Optional<Iterable<org.apache.druid.timeline.DataSegment>> iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval​(String datasource,
                                                                                                                                                   org.joda.time.Interval interval,
                                                                                                                                                   boolean requiresLatest)
        Description copied from interface: SegmentsMetadataManager
        Returns an iterable to go over all used and non-overshadowed segments of given data sources over given interval. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times. If {@param requiresLatest} is true then a force metadatastore poll will be triggered. This can cause a longer response time but will ensure that the latest segment information (at the time this method is called) is returned. If {@param requiresLatest} is false then segment information from stale snapshot of up to the last periodic poll period periodicPollDelay will be used.
        Specified by:
        iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval in interface SegmentsMetadataManager

      • iterateAllUnusedSegmentsForDatasource

        public Iterable<DataSegmentPlus> iterateAllUnusedSegmentsForDatasource​(String datasource,
                                                                       @Nullable
                                                                       org.joda.time.Interval interval,
                                                                       @Nullable
                                                                       Integer limit,
                                                                       @Nullable
                                                                       String lastSegmentId,
                                                                       @Nullable
                                                                       SortOrder sortOrder)
        Retrieves segments and their associated metadata for a given datasource that are marked unused and that are *fully contained by* an optionally specified interval. If the interval specified is null, this method will retrieve all unused segments. This call does not return any information about realtime segments.
        Specified by:
        iterateAllUnusedSegmentsForDatasource in interface SegmentsMetadataManager
        Parameters:
        datasource - The name of the datasource
        interval - an optional interval to search over.
        limit - an optional maximum number of results to return. If none is specified, the results are not limited.
        lastSegmentId - an optional last segment id from which to search for results. All segments returned are > this segment lexigraphically if sortOrder is null or SortOrder.ASC, or < this segment lexigraphically if sortOrder is SortOrder.DESC. If none is specified, no such filter is used.
        sortOrder - an optional order with which to return the matching segments by id, start time, end time. If none is specified, the order of the results is not guarenteed. Returns an iterable.

      • getUnusedSegmentIntervals

        public List<org.joda.time.Interval> getUnusedSegmentIntervals​(String dataSource,
                                                              @Nullable
                                                              org.joda.time.DateTime minStartTime,
                                                              org.joda.time.DateTime maxEndTime,
                                                              int limit,
                                                              org.joda.time.DateTime maxUsedStatusLastUpdatedTime)
        Description copied from interface: SegmentsMetadataManager
        Returns a list of up to limit unused segment intervals for the specified datasource. Segments are filtered based on the following criteria:
      • The start time of the segment must be no earlier than the specified minStartTime (if not null).
      • The end time of the segment must be no later than the specified maxEndTime.
      • The used_status_last_updated time of the segment must be no later than maxUsedStatusLastUpdatedTime. Segments that have no used_status_last_updated time (due to an upgrade from legacy Druid) will have maxUsedStatusLastUpdatedTime ignored.

        • The list of intervals is ordered by segment start time and then by end time.

Specified by:
getUnusedSegmentIntervals in interface SegmentsMetadataManager