Package org.apache.druid.segment.loading

Interface SegmentLoader

  • All Known Implementing Classes:
    SegmentLocalCacheLoader
    public interface SegmentLoader
    Loading segments from deep storage to local storage. Internally, this class can delegate the download to SegmentCacheManager. Implementations must be thread-safe.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void cleanup​(org.apache.druid.timeline.DataSegment segment)
      cleanup any state used by this segment
      org.apache.druid.segment.ReferenceCountingSegment getSegment​(org.apache.druid.timeline.DataSegment segment, boolean lazy, org.apache.druid.segment.SegmentLazyLoadFailCallback loadFailed)
      Returns a ReferenceCountingSegment that will be added by the SegmentManager to the VersionedIntervalTimeline.
      void loadSegmentIntoPageCache​(org.apache.druid.timeline.DataSegment segment, ExecutorService exec)
      Asyncly load segment into page cache.

    • Method Detail

      • getSegment

        org.apache.druid.segment.ReferenceCountingSegment getSegment​(org.apache.druid.timeline.DataSegment segment,
                                                             boolean lazy,
                                                             org.apache.druid.segment.SegmentLazyLoadFailCallback loadFailed)
                                                      throws org.apache.druid.segment.loading.SegmentLoadingException
        Returns a ReferenceCountingSegment that will be added by the SegmentManager to the VersionedIntervalTimeline. This method can be called multiple times by the SegmentManager and implementation can either return same ReferenceCountingSegment or a different ReferenceCountingSegment. Caller should not assume any particular behavior. Returning a ReferenceCountingSegment will let custom implementations keep track of reference count for segments that the custom implementations are creating. That way, custom implementations can know when the segment is in use or not.
        Parameters:
        segment - - Segment to load
        lazy - - Whether column metadata de-serialization is to be deferred to access time. Setting this flag to true can speed up segment loading
        loadFailed - - Callback to invoke if lazy loading fails during column access.
        Throws:
        org.apache.druid.segment.loading.SegmentLoadingException - - If there is an error in loading the segment

      • cleanup

        void cleanup​(org.apache.druid.timeline.DataSegment segment)
        cleanup any state used by this segment

      • loadSegmentIntoPageCache

        void loadSegmentIntoPageCache​(org.apache.druid.timeline.DataSegment segment,
                              ExecutorService exec)
        Asyncly load segment into page cache. Equivalent to `cat segment_files > /dev/null` to force loading the segment index files into page cache so that later when the segment is queried, they are already in page cache and only a minor page fault needs to be triggered instead of a major page fault to make the query latency more consistent.
        Parameters:
        segment - The segment to load its index files into page cache
        exec - The thread pool to use