org.apache.druid.segment.loading

Interface SegmentLoader

All Known Implementing Classes:

SegmentLocalCacheLoader

public interface SegmentLoader

Loading segments from deep storage to local storage. Internally, this class can delegate the download to SegmentCacheManager. Implementations must be thread-safe.

Method Summary

Modifier and Type	Method and Description
void	cleanup(DataSegment segment) cleanup any state used by this segment
ReferenceCountingSegment	getSegment(DataSegment segment, boolean lazy, SegmentLazyLoadFailCallback loadFailed) Returns a ReferenceCountingSegment that will be added by the SegmentManager to the VersionedIntervalTimeline.
void	loadSegmentIntoPageCache(DataSegment segment, ExecutorService exec) Asyncly load segment into page cache.

Method Detail

getSegment

ReferenceCountingSegment getSegment(DataSegment segment,
                                    boolean lazy,
                                    SegmentLazyLoadFailCallback loadFailed)
                             throws SegmentLoadingException

Returns a ReferenceCountingSegment that will be added by the SegmentManager to the VersionedIntervalTimeline. This method can be called multiple times by the SegmentManager and implementation can either return same ReferenceCountingSegment or a different ReferenceCountingSegment. Caller should not assume any particular behavior. Returning a ReferenceCountingSegment will let custom implementations keep track of reference count for segments that the custom implementations are creating. That way, custom implementations can know when the segment is in use or not.

Parameters:

segment - - Segment to load

lazy - - Whether column metadata de-serialization is to be deferred to access time. Setting this flag to true can speed up segment loading

loadFailed - - Callback to invoke if lazy loading fails during column access.

Throws:

SegmentLoadingException - - If there is an error in loading the segment

cleanup

void cleanup(DataSegment segment)

cleanup any state used by this segment

loadSegmentIntoPageCache

void loadSegmentIntoPageCache(DataSegment segment,
                              ExecutorService exec)

Asyncly load segment into page cache. Equivalent to `cat segment_files > /dev/null` to force loading the segment index files into page cache so that later when the segment is queried, they are already in page cache and only a minor page fault needs to be triggered instead of a major page fault to make the query latency more consistent.

Parameters:

segment - The segment to load its index files into page cache

exec - The thread pool to use