Package org.apache.druid.metadata

Class SqlSegmentsMetadataQuery

    • Method Detail

      • forHandle

        public static SqlSegmentsMetadataQuery forHandle​(org.skife.jdbi.v2.Handle handle,
                                                 SQLMetadataConnector connector,
                                                 org.apache.druid.metadata.MetadataStorageTablesConfig dbTables,
                                                 com.fasterxml.jackson.databind.ObjectMapper jsonMapper)
        Create a query object. This instance is scoped to a single handle and is meant to be short-lived. It is okay to use it for more than one query, though.

      • retrieveUsedSegments

        public org.apache.druid.java.util.common.parsers.CloseableIterator<org.apache.druid.timeline.DataSegment> retrieveUsedSegments​(String dataSource,
                                                                                                                               Collection<org.joda.time.Interval> intervals)
        Retrieves segments for a given datasource that are marked used (i.e. published) in the metadata store, and that *overlap* any interval in a particular collection of intervals. If the collection of intervals is empty, this method will retrieve all used segments. You cannot assume that segments returned by this call are actually active. Because there is some delay between new segment publishing and the marking-unused of older segments, it is possible that some segments returned by this call are overshadowed by other segments. To check for this, use SegmentTimeline.forSegments(Iterable). This call does not return any information about realtime segments. Returns a closeable iterator. You should close it when you are done.

      • retrieveUnusedSegments

        public org.apache.druid.java.util.common.parsers.CloseableIterator<org.apache.druid.timeline.DataSegment> retrieveUnusedSegments​(String dataSource,
                                                                                                                                 Collection<org.joda.time.Interval> intervals,
                                                                                                                                 @Nullable
                                                                                                                                 Integer limit,
                                                                                                                                 @Nullable
                                                                                                                                 String lastSegmentId,
                                                                                                                                 @Nullable
                                                                                                                                 SortOrder sortOrder,
                                                                                                                                 @Nullable
                                                                                                                                 org.joda.time.DateTime maxUsedStatusLastUpdatedTime)
        Retrieves segments for a given datasource that are marked unused and that are *fully contained by* any interval in a particular collection of intervals. If the collection of intervals is empty, this method will retrieve all unused segments. This call does not return any information about realtime segments.
        Parameters:
        dataSource - The name of the datasource
        intervals - The intervals to search over
        limit - The limit of segments to return
        lastSegmentId - the last segment id from which to search for results. All segments returned are > this segment lexigraphically if sortOrder is null or ASC, or < this segment lexigraphically if sortOrder is DESC.
        sortOrder - Specifies the order with which to return the matching segments by start time, end time. A null value indicates that order does not matter.
        maxUsedStatusLastUpdatedTime - The maximum used_status_last_updated time. Any unused segment in intervals with used_status_last_updated no later than this time will be included in the iterator. Segments without used_status_last_updated time (due to an upgrade from legacy Druid) will have maxUsedStatusLastUpdatedTime ignored Returns a closeable iterator. You should close it when you are done.

      • retrieveUnusedSegmentsPlus

        public org.apache.druid.java.util.common.parsers.CloseableIterator<DataSegmentPlus> retrieveUnusedSegmentsPlus​(String dataSource,
                                                                                                               Collection<org.joda.time.Interval> intervals,
                                                                                                               @Nullable
                                                                                                               Integer limit,
                                                                                                               @Nullable
                                                                                                               String lastSegmentId,
                                                                                                               @Nullable
                                                                                                               SortOrder sortOrder,
                                                                                                               @Nullable
                                                                                                               org.joda.time.DateTime maxUsedStatusLastUpdatedTime)
        Similar to retrieveUnusedSegments(java.lang.String, java.util.Collection<org.joda.time.Interval>, java.lang.Integer, java.lang.String, org.apache.druid.metadata.SortOrder, org.joda.time.DateTime), but also retrieves associated metadata for the segments for a given datasource that are marked unused and that are *fully contained by* any interval in a particular collection of intervals. If the collection of intervals is empty, this method will retrieve all unused segments. This call does not return any information about realtime segments.
        Parameters:
        dataSource - The name of the datasource
        intervals - The intervals to search over
        limit - The limit of segments to return
        lastSegmentId - the last segment id from which to search for results. All segments returned are > this segment lexigraphically if sortOrder is null or ASC, or < this segment lexigraphically if sortOrder is DESC.
        sortOrder - Specifies the order with which to return the matching segments by start time, end time. A null value indicates that order does not matter.
        maxUsedStatusLastUpdatedTime - The maximum used_status_last_updated time. Any unused segment in intervals with used_status_last_updated no later than this time will be included in the iterator. Segments without used_status_last_updated time (due to an upgrade from legacy Druid) will have maxUsedStatusLastUpdatedTime ignored Returns a closeable iterator. You should close it when you are done.

      • markSegments

        public int markSegments​(Collection<org.apache.druid.timeline.SegmentId> segmentIds,
                        boolean used)
        Marks the provided segments as either used or unused. For better performance, please try to 1) ensure that the caller passes only used segments to this method when marking them as unused. 2) Similarly, please try to call this method only on unused segments when marking segments as used with this method. Returns the number of segments actually modified.

      • markSegmentsUnused

        public int markSegmentsUnused​(String dataSource,
                              org.joda.time.Interval interval)
        Marks all used segments that are *fully contained by* a particular interval as unused. Returns the number of segments actually modified.

      • retrieveUsedSegmentForId

        public org.apache.druid.timeline.DataSegment retrieveUsedSegmentForId​(String id)
        Retrieve the used segment for a given id if it exists in the metadata store and null otherwise

      • retrieveSegmentForId

        public org.apache.druid.timeline.DataSegment retrieveSegmentForId​(String id)
        Retrieve the segment for a given id if it exists in the metadata store and null otherwise

      • appendConditionForIntervalsAndMatchMode

        public static void appendConditionForIntervalsAndMatchMode​(StringBuilder sb,
                                                           Collection<org.joda.time.Interval> intervals,
                                                           org.apache.druid.metadata.SqlSegmentsMetadataQuery.IntervalMode matchMode,
                                                           SQLMetadataConnector connector)
        Append the condition for the interval and match mode to the given string builder with a partial query
        Parameters:
        sb - - StringBuilder containing the paritial query with SELECT clause and WHERE condition for used, datasource
        intervals - - intervals to fetch the segments for
        matchMode - - Interval match mode - overlaps or contains
        connector - - SQL connector

      • bindQueryIntervals

        public static void bindQueryIntervals​(org.skife.jdbi.v2.Query<Map<String,​Object>> query,
                                      Collection<org.joda.time.Interval> intervals)
        Given a Query object bind the input intervals to it
        Parameters:
        query - Query to fetch segments
        intervals - Intervals to fetch segments for