Package org.apache.pinot.segment.local.io.readerwriter

Class RealtimeIndexOffHeapMemoryManager

  • All Implemented Interfaces:
    Closeable, AutoCloseable, PinotDataBufferMemoryManager
    Direct Known Subclasses:
    DirectMemoryManager, MmapMemoryManager
    @NotThreadSafe
public abstract class RealtimeIndexOffHeapMemoryManager
extends Object
implements PinotDataBufferMemoryManager
    RealtimeIndexOffHeapMemoryManager is an abstract class that implements base functionality to allocate and release memory that is acquired during realtime segment consumption. Realtime consuming segments use memory for dictionary, forward index, and inverted indices. For off-heap allocation of memory, we instantiate one RealtimeIndexOffHeapMemoryManager for each segment. Closing the RealtimeOffHeapMemoryManager also releases all the resources allocated. This class is NOT thread safe. Only one thread should access this class.

    • Constructor Detail

      • RealtimeIndexOffHeapMemoryManager

        protected RealtimeIndexOffHeapMemoryManager​(ServerMetrics serverMetrics,
                                            String segmentName)

    • Method Detail

      • allocate

        public PinotDataBuffer allocate​(long size,
                                String allocationContext)
        Allocate memory for use by a column. Sub-classes may implement this method according using different allocation policies. This method can be called multiple times for each column within the segment. Each invocation is guaranteed to return a new block of memory.
        Specified by:
        allocate in interface PinotDataBufferMemoryManager
        Parameters:
        size - size of memory
        allocationContext - Name of the column for which memory is being allocated
        Returns:
        PinotDataBuffer

      • doClose

        protected abstract void doClose()
                         throws IOException
        Method to be implemented by inheriting concrete classes
        Throws:
        IOException

      • allocateInternal

        protected abstract PinotDataBuffer allocateInternal​(long size,
                                                    String columnName)

      • close

        public void close()
           throws IOException
        Close out this memory manager and release all memory and resources. This method must be called when all the memory allocated by this class is no longer in use. The application may choose to call (or not call) PinotDataBuffer.close(), but this.close() MUST be called to release all resources allocated.
        Specified by:
        close in interface AutoCloseable
        Specified by:
        close in interface Closeable
        Throws:
        IOException