@InterfaceAudience.Private public class MemStoreLABImpl extends Object implements MemStoreLAB
The MemStoreLAB is basically a bump-the-pointer allocator that allocates big (2MB) byte[] chunks from and then doles it out to threads that request slices into the array.
The purpose of this class is to combat heap fragmentation in the regionserver. By ensuring that all Cells in a given memstore refer only to large chunks of contiguous memory, we ensure that large blocks get freed up when the memstore is flushed.
Without the MSLAB, the byte array allocated during insertion end up interleaved throughout the heap, and the old generation gets progressively more fragmented until a stop-the-world compacting collection occurs.
TODO: we should probably benchmark whether word-aligning the allocations would provide a
performance improvement - probably would speed up the Bytes.toLong/Bytes.toInt calls in KeyValue,
but some of those are cached anyway. The chunks created by this MemStoreLAB can get pooled at
ChunkCreator. When the Chunk comes from pool, it can be either an on heap or an off heap
backed chunk. The chunks, which this MemStoreLAB creates on its own (when no chunk available from
pool), those will be always on heap backed.
NOTE:if user requested to work with MSLABs (whether on- or off-heap), in
CompactingMemStore ctor, the CompactingMemStore.indexType could only be
CompactingMemStore.IndexType.CHUNK_MAP,that is to say the immutable segments using MSLABs are going to use
CellChunkMap as their index.
CHUNK_POOL_INITIALSIZE_KEY, CHUNK_POOL_MAXSIZE_KEY, CHUNK_SIZE_DEFAULT, CHUNK_SIZE_KEY, INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT, INDEX_CHUNK_SIZE_PERCENTAGE_KEY, MAX_ALLOC_DEFAULT, MAX_ALLOC_KEY, MSLAB_CLASS_NAME, POOL_INITIAL_SIZE_DEFAULT, POOL_MAX_SIZE_DEFAULT, USEMSLAB_DEFAULT, USEMSLAB_KEY| Constructor and Description |
|---|
MemStoreLABImpl() |
MemStoreLABImpl(org.apache.hadoop.conf.Configuration conf) |
| Modifier and Type | Method and Description |
|---|---|
void |
close()
Close this instance since it won't be used any more, try to put the chunks back to pool
|
Cell |
copyCellInto(Cell cell)
Allocates slice in this LAB and copy the passed Cell into this area.
|
void |
decScannerCount()
Called when closing a scanner on the data of this MemStoreLAB
|
Cell |
forceCopyOfBigCellInto(Cell cell)
When a cell's size is too big (bigger than maxAlloc), copyCellInto does not allocate it on
MSLAB.
|
Chunk |
getNewExternalChunk(ChunkCreator.ChunkType chunkType) |
Chunk |
getNewExternalChunk(int size) |
void |
incScannerCount()
Called when opening a scanner on the data of this MemStoreLAB
|
boolean |
isOffHeap() |
boolean |
isOnHeap() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitisEnabled, newInstancepublic MemStoreLABImpl()
public MemStoreLABImpl(org.apache.hadoop.conf.Configuration conf)
public Cell copyCellInto(Cell cell)
MemStoreLABcopyCellInto in interface MemStoreLABpublic Cell forceCopyOfBigCellInto(Cell cell)
forceCopyOfBigCellInto in interface MemStoreLABpublic void close()
close in interface MemStoreLABpublic void incScannerCount()
incScannerCount in interface MemStoreLABpublic void decScannerCount()
decScannerCount in interface MemStoreLABpublic Chunk getNewExternalChunk(ChunkCreator.ChunkType chunkType)
getNewExternalChunk in interface MemStoreLABpublic Chunk getNewExternalChunk(int size)
getNewExternalChunk in interface MemStoreLABpublic boolean isOnHeap()
isOnHeap in interface MemStoreLABpublic boolean isOffHeap()
isOffHeap in interface MemStoreLABCopyright © 2007–2020 The Apache Software Foundation. All rights reserved.