org.apache.druid.segment.loading

Class SegmentLocalCacheManager

java.lang.Object

org.apache.druid.segment.loading.SegmentLocalCacheManager

All Implemented Interfaces:

SegmentCacheManager

public class SegmentLocalCacheManager
extends Object
implements SegmentCacheManager

Constructor Summary

Constructors

Constructor and Description
SegmentLocalCacheManager(List<StorageLocation> locations, SegmentLoaderConfig config, StorageLocationSelectorStrategy strategy, com.fasterxml.jackson.databind.ObjectMapper mapper)
SegmentLocalCacheManager(SegmentLoaderConfig config, com.fasterxml.jackson.databind.ObjectMapper mapper) creates instance with default storage location selector strategy This ctor is mainly for test cases, including test cases in other modules

Method Summary

Modifier and Type	Method and Description
void	cleanup(DataSegment segment) Cleanup the cache space used by the segment.
List<StorageLocation>	getLocations()
File	getSegmentFiles(DataSegment segment) Make sure segments files in loc is intact, otherwise function like loadSegments will failed because of segment files is damaged.
ConcurrentHashMap<DataSegment,org.apache.druid.segment.loading.SegmentLocalCacheManager.ReferenceCountingLock>	getSegmentLocks()
boolean	isSegmentCached(DataSegment segment) Checks whether a segment is already cached.
void	loadSegmentIntoPageCache(DataSegment segment, ExecutorService exec) Asyncly load segment into page cache.
boolean	release(DataSegment segment) Reverts the effects of SegmentCacheManager.reserve(DataSegment) (DataSegment)} by releasing the location reserved for this segment.
boolean	reserve(DataSegment segment) Tries to reserve the space for a segment on any location.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SegmentLocalCacheManager

@Inject
public SegmentLocalCacheManager(List<StorageLocation> locations,
                                        SegmentLoaderConfig config,
                                        @Nonnull
                                        StorageLocationSelectorStrategy strategy,
                                        com.fasterxml.jackson.databind.ObjectMapper mapper)

SegmentLocalCacheManager

public SegmentLocalCacheManager(SegmentLoaderConfig config,
                                com.fasterxml.jackson.databind.ObjectMapper mapper)

creates instance with default storage location selector strategy This ctor is mainly for test cases, including test cases in other modules

Method Detail

isSegmentCached

public boolean isSegmentCached(DataSegment segment)

Description copied from interface: SegmentCacheManager

Checks whether a segment is already cached. It can return false even if SegmentCacheManager.reserve(DataSegment) has been successful for a segment but is not downloaded yet.

Specified by:

isSegmentCached in interface SegmentCacheManager

getSegmentFiles

public File getSegmentFiles(DataSegment segment)
                     throws SegmentLoadingException

Make sure segments files in loc is intact, otherwise function like loadSegments will failed because of segment files is damaged.

Specified by:

getSegmentFiles in interface SegmentCacheManager

Parameters:

segment -

Returns:

Throws:

SegmentLoadingException

reserve

public boolean reserve(DataSegment segment)

Description copied from interface: SegmentCacheManager

Tries to reserve the space for a segment on any location. When the space has been reserved, SegmentCacheManager.getSegmentFiles(DataSegment) should download the segment on the reserved location or fail otherwise. This function is useful for custom extensions. Extensions can try to reserve the space first and if not successful, make some space by cleaning up other segments, etc. There is also improved concurrency for extensions with this function. Since reserve is a cheaper operation to invoke till the space has been reserved. Hence it can be put inside a lock if required by the extensions. getSegment can't be put inside a lock since it is a time-consuming operation, on account of downloading the files.

Specified by:

reserve in interface SegmentCacheManager

Parameters:

segment - - Segment to reserve

Returns:

True if enough space found to store the segment, false otherwise

release

public boolean release(DataSegment segment)

Description copied from interface: SegmentCacheManager

Reverts the effects of SegmentCacheManager.reserve(DataSegment) (DataSegment)} by releasing the location reserved for this segment. Callers, that explicitly reserve the space via SegmentCacheManager.reserve(DataSegment), should use this method to release the space. Implementation can throw error if the space is being released but there is data present. Callers are supposed to ensure that any data is removed via SegmentCacheManager.cleanup(DataSegment)

Specified by:

release in interface SegmentCacheManager

Parameters:

segment - - Segment to release the location for.

Returns:

- True if any location was reserved and released, false otherwise.

cleanup

public void cleanup(DataSegment segment)

Description copied from interface: SegmentCacheManager

Cleanup the cache space used by the segment. It will not release the space if the space has been explicitly reserved via SegmentCacheManager.reserve(DataSegment)

Specified by:

cleanup in interface SegmentCacheManager

loadSegmentIntoPageCache

public void loadSegmentIntoPageCache(DataSegment segment,
                                     ExecutorService exec)

Description copied from interface: SegmentCacheManager

Asyncly load segment into page cache. Equivalent to `cat segment_files > /dev/null` to force loading the segment index files into page cache so that later when the segment is queried, they are already in page cache and only a minor page fault needs to be triggered instead of a major page fault to make the query latency more consistent.

Specified by:

loadSegmentIntoPageCache in interface SegmentCacheManager

Parameters:

segment - The segment to load its index files into page cache

exec - The thread pool to use

getSegmentLocks

public ConcurrentHashMap<DataSegment,org.apache.druid.segment.loading.SegmentLocalCacheManager.ReferenceCountingLock> getSegmentLocks()

getLocations

public List<StorageLocation> getLocations()