org.apache.hadoop.hbase.regionserver

Class HMobStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.HStore

org.apache.hadoop.hbase.regionserver.HMobStore

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation

@InterfaceAudience.Private
public class HMobStore
extends HStore

The store implementation to save MOBs (medium objects), it extends the HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek to HBase is done. The store implements how we save MOBs by extending HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.HStore

BLOCK_STORAGE_POLICY_KEY, BLOCKING_STOREFILES_KEY, COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY, conf, DEEP_OVERHEAD, DEFAULT_BLOCK_STORAGE_POLICY, DEFAULT_BLOCKING_STOREFILE_COUNT, DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER, FIXED_OVERHEAD, memstore, MEMSTORE_CLASS_NAME

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store

NO_PRIORITY, PRIORITY_USER

Constructor Summary

Constructors

Constructor and Description
HMobStore(HRegion region, ColumnFamilyDescriptor family, org.apache.hadoop.conf.Configuration confParam, boolean warmup)

Method Summary

Modifier and Type	Method and Description
void	commitFile(org.apache.hadoop.fs.Path sourceFile, org.apache.hadoop.fs.Path targetPath) Commits the mob file.
protected KeyValueScanner	createScanner(Scan scan, ScanInfo scanInfo, NavigableSet<byte[]> targetCols, long readPt) Gets the MobStoreScanner or MobReversedStoreScanner.
protected StoreEngine<?,?,?,?>	createStoreEngine(HStore store, org.apache.hadoop.conf.Configuration conf, CellComparator cellComparator) Creates the mob store engine.
StoreFileWriter	createWriter(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Creates the writer for the mob file in the mob family directory.
StoreFileWriter	createWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction) Creates the writer for the mob file in temp directory.
StoreFileWriter	createWriterInTmp(MobFileName mobFileName, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Creates the writer for the mob file in temp directory.
StoreFileWriter	createWriterInTmp(String date, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) Creates the writer for the mob file in temp directory.
long	getCellsCountCompactedFromMob()
long	getCellsCountCompactedToMob()
long	getCellsSizeCompactedFromMob()
long	getCellsSizeCompactedToMob()
org.apache.hadoop.conf.Configuration	getConfiguration() Gets current config.
List<org.apache.hadoop.fs.Path>	getLocations(TableName tableName)
long	getMobFlushCount()
long	getMobFlushedCellsCount()
long	getMobFlushedCellsSize()
long	getMobScanCellsCount()
long	getMobScanCellsSize()
org.apache.hadoop.fs.Path	getPath() Gets the mob file path.
byte[]	getRefCellTags()
MobCell	resolve(Cell reference, boolean cacheBlocks) Reads the cell from the mob file, and the read point does not count.
MobCell	resolve(Cell reference, boolean cacheBlocks, boolean readEmptyValueOnMobCellMiss) Reads the cell from the mob file with readEmptyValueOnMobCellMiss
MobCell	resolve(Cell reference, boolean cacheBlocks, long readPt, boolean readEmptyValueOnMobCellMiss) Reads the cell from the mob file.
void	updateCellsCountCompactedFromMob(long count)
void	updateCellsCountCompactedToMob(long count)
void	updateCellsSizeCompactedFromMob(long size)
void	updateCellsSizeCompactedToMob(long size)
void	updateMobFlushCount()
void	updateMobFlushedCellsCount(long count)
void	updateMobFlushedCellsSize(long size)
void	updateMobScanCellsCount(long count)
void	updateMobScanCellsSize(long size)

Methods inherited from class org.apache.hadoop.hbase.regionserver.HStore

add, add, addChangedReaderObserver, areWritesEnabled, assertBulkLoadHFileOk, bulkLoadHFile, bulkLoadHFile, cancelRequestedCompaction, canSplit, close, closeAndArchiveCompactedFiles, compact, createCacheConf, createFlushContext, deleteChangedReaderObserver, deregisterChildren, determineTTLFromFamily, doCompaction, finishCompactionRequest, flushCache, getAvgStoreFileAge, getBlockingFileCount, getBloomFilterEligibleRequestsCount, getBloomFilterNegativeResultsCount, getBloomFilterRequestsCount, getCacheConfig, getColumnFamilyDescriptor, getColumnFamilyName, getCompactedCellsCount, getCompactedCellsSize, getCompactedFiles, getCompactedFilesCount, getCompactionCheckMultiplier, getCompactionPressure, getCompactionProgress, getCompactPriority, getComparator, getCoprocessorHost, getCurrentParallelPutCount, getDataBlockEncoder, getFileSystem, getFlushableSize, getFlushedCellsCount, getFlushedCellsSize, getFlushedOutputFileSize, getHFilesSize, getHRegion, getLastCompactSize, getMajorCompactedCellsCount, getMajorCompactedCellsSize, getMaxCompactedStoreFileRefCount, getMaxMemStoreTS, getMaxSequenceId, getMaxStoreFileAge, getMemStoreFlushSize, getMemstoreOnlyRowReadsCount, getMemStoreSize, getMinStoreFileAge, getMixedRowReadsCount, getNumHFiles, getNumReferenceFiles, getOffPeakHours, getReadOnlyConfiguration, getRegionFileSystem, getRegionInfo, getScanInfo, getScanner, getScanners, getScanners, getScanners, getScanners, getSize, getSmallestReadPoint, getSnapshotSize, getSplitPoint, getStoreEngine, getStorefiles, getStoreFilesBeingWritten, getStorefilesCount, getStorefilesRootLevelIndexSize, getStorefilesSize, getStoreFileTtl, getStoreRefCount, getStoreSizeUncompressed, getTableName, getTotalStaticBloomSize, getTotalStaticIndexSize, hasReferences, hasTooManyStoreFiles, heapSize, isPrimaryReplicaStore, isSloppyMemStore, needsCompaction, onConfigurationChange, postSnapshotOperation, preBulkLoadHFile, preFlushSeqIDEstimation, preSnapshotOperation, recreateScanners, refreshStoreFiles, refreshStoreFiles, refreshStoreSizeAndTotalBytes, registerChildren, replayCompactionMarker, requestCompaction, requestCompaction, shouldPerformMajorCompaction, startReplayingFromWAL, stopReplayingFromWAL, throttleCompaction, timeOfOldestEdit, toString, triggerMajorCompaction, tryCommitRecoveredHFile, updateCompactedMetrics, upsert

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HMobStore

public HMobStore(HRegion region,
                 ColumnFamilyDescriptor family,
                 org.apache.hadoop.conf.Configuration confParam,
                 boolean warmup)
          throws IOException

Throws:

IOException

Method Detail

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Gets current config.

createScanner

protected KeyValueScanner createScanner(Scan scan,
                                        ScanInfo scanInfo,
                                        NavigableSet<byte[]> targetCols,
                                        long readPt)
                                 throws IOException

Gets the MobStoreScanner or MobReversedStoreScanner. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Overrides:

createScanner in class HStore

Throws:

IOException

createStoreEngine

protected StoreEngine<?,?,?,?> createStoreEngine(HStore store,
                                                 org.apache.hadoop.conf.Configuration conf,
                                                 CellComparator cellComparator)
                                          throws IOException

Creates the mob store engine.

Overrides:

createStoreEngine in class HStore

Parameters:

store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.

conf - Store configuration.

cellComparator - KVComparator for storeFileManager.

Returns:

StoreEngine to use.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(Date date,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         byte[] startKey,
                                         boolean isCompaction)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

createWriter

public StoreFileWriter createWriter(Date date,
                                    long maxKeyCount,
                                    Compression.Algorithm compression,
                                    byte[] startKey,
                                    boolean isCompaction,
                                    Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                             throws IOException

Creates the writer for the mob file in the mob family directory.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(String date,
                                         org.apache.hadoop.fs.Path basePath,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         byte[] startKey,
                                         boolean isCompaction,
                                         Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The date string, its format is yyyymmmdd.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(MobFileName mobFileName,
                                         org.apache.hadoop.fs.Path basePath,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         boolean isCompaction,
                                         Consumer<org.apache.hadoop.fs.Path> writerCreationTracker)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

mobFileName - The mob file name.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

commitFile

public void commitFile(org.apache.hadoop.fs.Path sourceFile,
                       org.apache.hadoop.fs.Path targetPath)
                throws IOException

Commits the mob file.

Parameters:

sourceFile - The source file.

targetPath - The directory path where the source file is renamed to.

Throws:

IOException

resolve

public MobCell resolve(Cell reference,
                       boolean cacheBlocks)
                throws IOException

Reads the cell from the mob file, and the read point does not count. This is used for DefaultMobStoreCompactor where we can read empty value for the missing cell.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

Returns:

The cell found in the mob file.

Throws:

IOException

resolve

public MobCell resolve(Cell reference,
                       boolean cacheBlocks,
                       boolean readEmptyValueOnMobCellMiss)
                throws IOException

Reads the cell from the mob file with readEmptyValueOnMobCellMiss

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

readEmptyValueOnMobCellMiss - should return empty mob cell if reference can not be resolved.

Returns:

The cell found in the mob file.

Throws:

IOException

resolve

public MobCell resolve(Cell reference,
                       boolean cacheBlocks,
                       long readPt,
                       boolean readEmptyValueOnMobCellMiss)
                throws IOException

Reads the cell from the mob file.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

readPt - the read point.

readEmptyValueOnMobCellMiss - Whether return null value when the mob file is missing or corrupt.

Returns:

The cell found in the mob file.

Throws:

IOException

getLocations

public List<org.apache.hadoop.fs.Path> getLocations(TableName tableName)
                                             throws IOException

Parameters:

tableName - to look up locations for, can not be null

Returns:

a list of location in order of working dir, archive dir. will not be null.

Throws:

IOException

getPath

public org.apache.hadoop.fs.Path getPath()

Gets the mob file path.

Returns:

The mob file path.

updateCellsCountCompactedToMob

public void updateCellsCountCompactedToMob(long count)

getCellsCountCompactedToMob

public long getCellsCountCompactedToMob()

updateCellsCountCompactedFromMob

public void updateCellsCountCompactedFromMob(long count)

getCellsCountCompactedFromMob

public long getCellsCountCompactedFromMob()

updateCellsSizeCompactedToMob

public void updateCellsSizeCompactedToMob(long size)

getCellsSizeCompactedToMob

public long getCellsSizeCompactedToMob()

updateCellsSizeCompactedFromMob

public void updateCellsSizeCompactedFromMob(long size)

getCellsSizeCompactedFromMob

public long getCellsSizeCompactedFromMob()

updateMobFlushCount

public void updateMobFlushCount()

getMobFlushCount

public long getMobFlushCount()

updateMobFlushedCellsCount

public void updateMobFlushedCellsCount(long count)

getMobFlushedCellsCount

public long getMobFlushedCellsCount()

updateMobFlushedCellsSize

public void updateMobFlushedCellsSize(long size)

getMobFlushedCellsSize

public long getMobFlushedCellsSize()

updateMobScanCellsCount

public void updateMobScanCellsCount(long count)

getMobScanCellsCount

public long getMobScanCellsCount()

updateMobScanCellsSize

public void updateMobScanCellsSize(long size)

getMobScanCellsSize

public long getMobScanCellsSize()

getRefCellTags

public byte[] getRefCellTags()