Package org.apache.druid.segment.loading

Interface SegmentCacheManager

  • All Known Implementing Classes:
    SegmentLocalCacheManager
    public interface SegmentCacheManager
    A class to fetch segment files to local disk and manage the local cache. Implementations must be thread-safe.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void cleanup​(org.apache.druid.timeline.DataSegment segment)
      Cleanup the cache space used by the segment.
      File getSegmentFiles​(org.apache.druid.timeline.DataSegment segment)
      This method fetches the files for the given segment if the segment is not downloaded already.
      boolean isSegmentCached​(org.apache.druid.timeline.DataSegment segment)
      Checks whether a segment is already cached.
      void loadSegmentIntoPageCache​(org.apache.druid.timeline.DataSegment segment, ExecutorService exec)
      Asyncly load segment into page cache.
      boolean release​(org.apache.druid.timeline.DataSegment segment)
      Reverts the effects of reserve(DataSegment) (DataSegment)} by releasing the location reserved for this segment.
      boolean reserve​(org.apache.druid.timeline.DataSegment segment)
      Tries to reserve the space for a segment on any location.

    • Method Detail

      • isSegmentCached

        boolean isSegmentCached​(org.apache.druid.timeline.DataSegment segment)
        Checks whether a segment is already cached. It can return false even if reserve(DataSegment) has been successful for a segment but is not downloaded yet.

      • getSegmentFiles

        File getSegmentFiles​(org.apache.druid.timeline.DataSegment segment)
              throws org.apache.druid.segment.loading.SegmentLoadingException
        This method fetches the files for the given segment if the segment is not downloaded already. It is not required to reserve(DataSegment) before calling this method. If caller has not reserved the space explicitly via reserve(DataSegment), the implementation should reserve space on caller's behalf. If the space has been explicitly reserved already - implementation should use only the reserved space to store segment files. - implementation should not release the location in case of download erros and leave it to the caller.
        Throws:
        org.apache.druid.segment.loading.SegmentLoadingException - if there is an error in downloading files

      • reserve

        boolean reserve​(org.apache.druid.timeline.DataSegment segment)
        Tries to reserve the space for a segment on any location. When the space has been reserved, getSegmentFiles(DataSegment) should download the segment on the reserved location or fail otherwise. This function is useful for custom extensions. Extensions can try to reserve the space first and if not successful, make some space by cleaning up other segments, etc. There is also improved concurrency for extensions with this function. Since reserve is a cheaper operation to invoke till the space has been reserved. Hence it can be put inside a lock if required by the extensions. getSegment can't be put inside a lock since it is a time-consuming operation, on account of downloading the files.
        Parameters:
        segment - - Segment to reserve
        Returns:
        True if enough space found to store the segment, false otherwise

      • release

        boolean release​(org.apache.druid.timeline.DataSegment segment)
        Reverts the effects of reserve(DataSegment) (DataSegment)} by releasing the location reserved for this segment. Callers, that explicitly reserve the space via reserve(DataSegment), should use this method to release the space. Implementation can throw error if the space is being released but there is data present. Callers are supposed to ensure that any data is removed via cleanup(DataSegment)
        Parameters:
        segment - - Segment to release the location for.
        Returns:
        - True if any location was reserved and released, false otherwise.

      • cleanup

        void cleanup​(org.apache.druid.timeline.DataSegment segment)
        Cleanup the cache space used by the segment. It will not release the space if the space has been explicitly reserved via reserve(DataSegment)

      • loadSegmentIntoPageCache

        void loadSegmentIntoPageCache​(org.apache.druid.timeline.DataSegment segment,
                              ExecutorService exec)
        Asyncly load segment into page cache. Equivalent to `cat segment_files > /dev/null` to force loading the segment index files into page cache so that later when the segment is queried, they are already in page cache and only a minor page fault needs to be triggered instead of a major page fault to make the query latency more consistent.
        Parameters:
        segment - The segment to load its index files into page cache
        exec - The thread pool to use