org.apache.hadoop.hbase.mob

Class DefaultMobStoreFlusher

java.lang.Object

org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher

org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher

@InterfaceAudience.Private
public class DefaultMobStoreFlusher
extends DefaultStoreFlusher

An implementation of the StoreFlusher. It extends the DefaultStoreFlusher. If the store is not a mob store, the flusher flushes the MemStore the same with DefaultStoreFlusher, If the store is a mob store, the flusher flushes the MemStore into two places. One is the store files of HBase, the other is the mob files.

1. Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
2. If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
3. If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.conf.Configuration	conf
protected HStore	store

Constructor Summary

Constructors

Constructor and Description
DefaultMobStoreFlusher(org.apache.hadoop.conf.Configuration conf, HStore store)

Method Summary

Modifier and Type	Method and Description
protected InternalScanner	createScanner(List<KeyValueScanner> snapshotScanners, FlushLifeCycleTracker tracker) Creates the scanner for flushing snapshot.
protected StoreFileWriter	createWriter(MemStoreSnapshot snapshot, boolean alwaysIncludesTag, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
protected void	finalizeWriter(StoreFileWriter writer, long cacheFlushSeqNum, MonitoredTask status)
List<org.apache.hadoop.fs.Path>	flushSnapshot(MemStoreSnapshot snapshot, long cacheFlushId, MonitoredTask status, ThroughputController throughputController, FlushLifeCycleTracker tracker, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the snapshot of the MemStore.
protected void	performFlush(InternalScanner scanner, CellSink sink, ThroughputController throughputController) Performs memstore flush, writing data from scanner into sink.
protected void	performMobFlush(MemStoreSnapshot snapshot, long cacheFlushId, InternalScanner scanner, StoreFileWriter writer, MonitoredTask status, ThroughputController throughputController, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Flushes the cells in the mob store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

store

protected HStore store

Constructor Detail

DefaultMobStoreFlusher

public DefaultMobStoreFlusher(org.apache.hadoop.conf.Configuration conf,
                              HStore store)
                       throws IOException

Throws:

IOException

Method Detail

flushSnapshot

public List<org.apache.hadoop.fs.Path> flushSnapshot(MemStoreSnapshot snapshot,
                                                     long cacheFlushId,
                                                     MonitoredTask status,
                                                     ThroughputController throughputController,
                                                     FlushLifeCycleTracker tracker,
                                                     Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                                              throws IOException

Flushes the snapshot of the MemStore. If this store is not a mob store, flush the cells in the snapshot to store files of HBase. If the store is a mob one, the flusher flushes the MemStore into two places. One is the store files of HBase, the other is the mob files.

1. Cells that are not PUT type or have the delete mark will be directly flushed to HBase.
2. If the size of a cell value is larger than a threshold, it'll be flushed to a mob file, another cell with the path of this file will be flushed to HBase.
3. If the size of a cell value is smaller than or equal with a threshold, it'll be flushed to HBase directly.

Overrides:

flushSnapshot in class DefaultStoreFlusher

Parameters:

snapshot - Memstore snapshot.

cacheFlushId - Log cache flush sequence number.

status - Task that represents the flush operation and may be updated with status.

throughputController - A controller to avoid flush too fast

Returns:

List of files written. Can be empty; must not be null.

Throws:

IOException

performMobFlush

protected void performMobFlush(MemStoreSnapshot snapshot,
                               long cacheFlushId,
                               InternalScanner scanner,
                               StoreFileWriter writer,
                               MonitoredTask status,
                               ThroughputController throughputController,
                               Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                        throws IOException

Flushes the cells in the mob store.

In the mob store, the cells with PUT type might have or have no mob tags.
1. If a cell does not have a mob tag, flushing the cell to different files depends on the value length. If the length is larger than a threshold, it's flushed to a mob file and the mob file is flushed to a store file in HBase. Otherwise, directly flush the cell to a store file in HBase.
2. If a cell have a mob tag, its value is a mob file name, directly flush it to a store file in HBase.

Parameters:

snapshot - Memstore snapshot.

cacheFlushId - Log cache flush sequence number.

scanner - The scanner of memstore snapshot.

writer - The store file writer.

status - Task that represents the flush operation and may be updated with status.

throughputController - A controller to avoid flush too fast.

Throws:

IOException

finalizeWriter

protected void finalizeWriter(StoreFileWriter writer,
                              long cacheFlushSeqNum,
                              MonitoredTask status)
                       throws IOException

Throws:

IOException

createWriter

protected final StoreFileWriter createWriter(MemStoreSnapshot snapshot,
                                             boolean alwaysIncludesTag,
                                             Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                                      throws IOException

Throws:

IOException

createScanner

protected final InternalScanner createScanner(List<KeyValueScanner> snapshotScanners,
                                              FlushLifeCycleTracker tracker)
                                       throws IOException

Creates the scanner for flushing snapshot. Also calls coprocessors.

Returns:

The scanner; null if coprocessor is canceling the flush.

Throws:

IOException

performFlush

protected void performFlush(InternalScanner scanner,
                            CellSink sink,
                            ThroughputController throughputController)
                     throws IOException

Performs memstore flush, writing data from scanner into sink.

Parameters:

scanner - Scanner to get data from.

sink - Sink to write data to. Could be StoreFile.Writer.

throughputController - A controller to avoid flush too fast

Throws:

IOException