CompactingMemStore (Apache HBase - Server 2.0.0-alpha4 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.AbstractMemStore
- - org.apache.hadoop.hbase.regionserver.CompactingMemStore

All Implemented Interfaces:

MemStore
```
@InterfaceAudience.Private
public class CompactingMemStore
extends AbstractMemStore
```
A memstore implementation which supports in-memory compaction. A compaction pipeline is added between the active set and the snapshot data structures; it consists of a list of kv-sets that are subject to compaction. Like the snapshot, all pipeline components are read-only; updates only affect the active set. To ensure this property we take advantage of the existing blocking mechanism -- the active set is pushed to the pipeline while holding the region's updatesLock in exclusive mode. Periodically, a compaction is applied in the background to all pipeline components resulting in a single read-only component. The ``old'' components are discarded when no scanner is reading them.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`CompactingMemStore.IndexType` Types of indexes (part of immutable segments) to be used after flattening, compaction, or merge are applied.

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`COMPACTING_MEMSTORE_INDEX_DEFAULT`
`static String`	`COMPACTING_MEMSTORE_INDEX_KEY`
`static String`	`COMPACTING_MEMSTORE_TYPE_DEFAULT`
`static String`	`COMPACTING_MEMSTORE_TYPE_KEY`
`static long`	`DEEP_OVERHEAD`
`static String`	`IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY`

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore
active, FIXED_OVERHEAD, snapshot, snapshotId

Constructor Summary

Constructors
Constructor and Description
`CompactingMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c, HStore store, RegionServicesForStores regionServices, MemoryCompactionPolicy compactionPolicy)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`checkActiveSize()` Check whether anything need to be done based on the current active set size.
`protected List<KeyValueScanner>`	`createList(int capacity)`
`protected MemStoreCompactor`	`createMemStoreCompactor(MemoryCompactionPolicy compactionPolicy)`
`void`	`debug()`
`void`	`flattenOneSegment(long requesterVersion)`
`String`	`getFamilyName()`
`MemStoreSize`	`getFlushableSize()` On flush, how much memory we will clear.
`VersionedSegmentsList`	`getImmutableSegments()`
`CompactingMemStore.IndexType`	`getIndexType()`
`List<KeyValueScanner>`	`getScanners(long readPt)`
`protected List<Segment>`	`getSegments()`
`long`	`getSmallestReadPoint()`
`HStore`	`getStore()`
`boolean`	`hasImmutableSegments()`
`protected long`	`heapSize()`
`boolean`	`isCompositeSnapshot()`
`boolean`	`isSloppy()`
`protected long`	`keySize()`
`long`	`preFlushSeqIDEstimation()` This method is called before the flush is executed.
`protected void`	`pushActiveToPipeline(MutableSegment active)`
`void`	`setCompositeSnapshot(boolean useCompositeSnapshot)`
`void`	`setIndexType()`
`protected boolean`	`shouldFlushInMemory()`
`MemStoreSize`	`size()`
`MemStoreSnapshot`	`snapshot()` Push the current active memstore segment into the pipeline and create a snapshot of the tail of current compaction pipeline Snapshot must be cleared by call to `AbstractMemStore.clearSnapshot(long)`.
`void`	`startReplayingFromWAL()` This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
`void`	`stopReplayingFromWAL()` This message intends to inform the MemStore that the replaying edits from WAL are done
`boolean`	`swapCompactedSegments(VersionedSegmentsList versionedList, ImmutableSegment result, boolean merge)`
`void`	`updateLowestUnflushedSequenceIdInWAL(boolean onlyIfGreater)` Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore
add, add, addToScanners, addToScanners, clearSnapshot, dump, getComparator, getConfiguration, getLowest, getNextRow, getSnapshotSize, resetActive, timeOfOldestEdit, toString, upsert

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - COMPACTING_MEMSTORE_TYPE_KEY
```
public static final String COMPACTING_MEMSTORE_TYPE_KEY
```
    See Also:
    
    Constant Field Values
  - COMPACTING_MEMSTORE_TYPE_DEFAULT
```
public static final String COMPACTING_MEMSTORE_TYPE_DEFAULT
```
  - COMPACTING_MEMSTORE_INDEX_KEY
```
public static final String COMPACTING_MEMSTORE_INDEX_KEY
```
    See Also:
    
    Constant Field Values
  - COMPACTING_MEMSTORE_INDEX_DEFAULT
```
public static final String COMPACTING_MEMSTORE_INDEX_DEFAULT
```
  - IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY
```
public static final String IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY
```
    See Also:
    
    Constant Field Values
  - DEEP_OVERHEAD
```
public static final long DEEP_OVERHEAD
```
- Constructor Detail
  - CompactingMemStore
```
public CompactingMemStore(org.apache.hadoop.conf.Configuration conf,
                          CellComparator c,
                          HStore store,
                          RegionServicesForStores regionServices,
                          MemoryCompactionPolicy compactionPolicy)
                   throws IOException
```
    Throws:
    
    IOException
- Method Detail
  - createMemStoreCompactor
```
protected MemStoreCompactor createMemStoreCompactor(MemoryCompactionPolicy compactionPolicy)
```
  - size
```
public MemStoreSize size()
```
    Returns:
    
    Total memory occupied by this MemStore. This won't include any size occupied by the snapshot. We assume the snapshot will get cleared soon. This is not thread safe and the memstore may be changed while computing its size. It is the responsibility of the caller to make sure this doesn't happen.
  - preFlushSeqIDEstimation
```
public long preFlushSeqIDEstimation()
```
    This method is called before the flush is executed.
    
    Returns:
    
    an estimation (lower bound) of the unflushed sequence id in memstore after the flush is executed. if memstore will be cleared returns HConstants.NO_SEQNUM.
  - isSloppy
```
public boolean isSloppy()
```
  - snapshot
```
public MemStoreSnapshot snapshot()
```
    Push the current active memstore segment into the pipeline and create a snapshot of the tail of current compaction pipeline Snapshot must be cleared by call to AbstractMemStore.clearSnapshot(long). AbstractMemStore.clearSnapshot(long).
    
    Returns:
    
    MemStoreSnapshot
  - getFlushableSize
```
public MemStoreSize getFlushableSize()
```
    On flush, how much memory we will clear.
    
    Returns:
    
    size of data that is going to be flushed
  - keySize
```
protected long keySize()
```
    Specified by:
    
    keySize in class AbstractMemStore
    
    Returns:
    
    The total size of cells in this memstore. We will not consider cells in the snapshot
  - heapSize
```
protected long heapSize()
```
    Specified by:
    
    heapSize in class AbstractMemStore
    
    Returns:
    
    The total heap size of cells in this memstore. We will not consider cells in the snapshot
  - updateLowestUnflushedSequenceIdInWAL
```
public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfGreater)
```
    Description copied from class: AbstractMemStore
    
    Updates the wal with the lowest sequence id (oldest entry) that is still in memory
    
    Specified by:
    
    updateLowestUnflushedSequenceIdInWAL in class AbstractMemStore
    
    Parameters:
    
    onlyIfGreater - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id
  - startReplayingFromWAL
```
public void startReplayingFromWAL()
```
    This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
  - stopReplayingFromWAL
```
public void stopReplayingFromWAL()
```
    This message intends to inform the MemStore that the replaying edits from WAL are done
  - getSegments
```
protected List<Segment> getSegments()
```
    Specified by:
    
    getSegments in class AbstractMemStore
    
    Returns:
    
    an ordered list of segments from most recent to oldest in memstore
  - setCompositeSnapshot
```
public void setCompositeSnapshot(boolean useCompositeSnapshot)
```
  - isCompositeSnapshot
```
public boolean isCompositeSnapshot()
```
  - swapCompactedSegments
```
public boolean swapCompactedSegments(VersionedSegmentsList versionedList,
                                     ImmutableSegment result,
                                     boolean merge)
```
  - flattenOneSegment
```
public void flattenOneSegment(long requesterVersion)
```
    Parameters:
    
    requesterVersion - The caller must hold the VersionedList of the pipeline with version taken earlier. This version must be passed as a parameter here. The flattening happens only if versions match.
  - setIndexType
```
public void setIndexType()
```
  - getIndexType
```
public CompactingMemStore.IndexType getIndexType()
```
  - hasImmutableSegments
```
public boolean hasImmutableSegments()
```
  - getImmutableSegments
```
public VersionedSegmentsList getImmutableSegments()
```
  - getSmallestReadPoint
```
public long getSmallestReadPoint()
```
  - getStore
```
public HStore getStore()
```
  - getFamilyName
```
public String getFamilyName()
```
  - getScanners
```
public List<KeyValueScanner> getScanners(long readPt)
                                  throws IOException
```
    Returns:
    
    scanner over the memstore. This might include scanner over the snapshot when one is present.
    
    Throws:
    
    IOException
  - createList
```
protected List<KeyValueScanner> createList(int capacity)
```
  - checkActiveSize
```
protected void checkActiveSize()
```
    Check whether anything need to be done based on the current active set size. The method is invoked upon every addition to the active set. For CompactingMemStore, flush the active set to the read-only memory if it's size is above threshold
    
    Specified by:
    
    checkActiveSize in class AbstractMemStore
  - shouldFlushInMemory
```
protected boolean shouldFlushInMemory()
```
  - pushActiveToPipeline
```
protected void pushActiveToPipeline(MutableSegment active)
```
  - debug
```
public void debug()
```

Class CompactingMemStore

Nested Class Summary

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

Methods inherited from class java.lang.Object

Field Detail

COMPACTING_MEMSTORE_TYPE_KEY

COMPACTING_MEMSTORE_TYPE_DEFAULT

COMPACTING_MEMSTORE_INDEX_KEY

COMPACTING_MEMSTORE_INDEX_DEFAULT

IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY

DEEP_OVERHEAD

Constructor Detail

CompactingMemStore

Method Detail

createMemStoreCompactor

size

preFlushSeqIDEstimation

isSloppy

snapshot

getFlushableSize

keySize

heapSize

updateLowestUnflushedSequenceIdInWAL

startReplayingFromWAL

stopReplayingFromWAL

getSegments

setCompositeSnapshot

isCompositeSnapshot

swapCompactedSegments

flattenOneSegment

setIndexType

getIndexType

hasImmutableSegments

getImmutableSegments

getSmallestReadPoint

getStore

getFamilyName

getScanners

createList

checkActiveSize

shouldFlushInMemory

pushActiveToPipeline

debug