org.apache.hadoop.hbase.io.hfile

Class CombinedBlockCache

java.lang.Object

org.apache.hadoop.hbase.io.hfile.CombinedBlockCache

All Implemented Interfaces:

Iterable<CachedBlock>, HeapSize, BlockCache, ResizableBlockCache

Direct Known Subclasses:

InclusiveCombinedBlockCache

@InterfaceAudience.Private
public class CombinedBlockCache
extends Object
implements ResizableBlockCache, HeapSize

CombinedBlockCache is an abstraction layer that combines FirstLevelBlockCache and BucketCache. The smaller lruCache is used to cache bloom blocks and index blocks. The larger Cache is used to cache data blocks. getBlock(BlockCacheKey, boolean, boolean, boolean) reads first from the smaller l1Cache before looking for the block in the l2Cache. Blocks evicted from l1Cache are put into the bucket cache. Metrics are the combined size and hits and misses of both caches.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CombinedBlockCache.CombinedCacheStats

Field Summary

Fields

Modifier and Type	Field and Description
protected CombinedBlockCache.CombinedCacheStats	combinedCacheStats
protected FirstLevelBlockCache	l1Cache
protected BlockCache	l2Cache

Constructor Summary

Constructors

Constructor and Description
CombinedBlockCache(FirstLevelBlockCache l1Cache, BlockCache l2Cache)

Method Summary

Modifier and Type	Method and Description
Optional<Boolean>	blockFitsIntoTheCache(HFileBlock block) Checks whether there's enough space left in the cache to accommodate the passed block.
void	cacheBlock(BlockCacheKey cacheKey, Cacheable buf) Add block to cache (defaults to not in-memory).
void	cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory) Add block to cache.
void	cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean inMemory, boolean waitWhenCache) Add block to cache.
boolean	evictBlock(BlockCacheKey cacheKey) Evict block from cache.
int	evictBlocksByHfileName(String hfileName) Evicts all blocks for the given HFile.
Cacheable	getBlock(BlockCacheKey cacheKey, boolean caching, boolean repeat, boolean updateCacheMetrics) Fetch block from cache.
BlockCache[]	getBlockCaches() Returns The list of sub blockcaches that make up this one; returns null if no sub caches.
long	getBlockCount() Returns the number of blocks currently cached in the block cache.
Optional<Integer>	getBlockSize(BlockCacheKey key) Returns an Optional containing the size of the block related to the passed key.
long	getCurrentDataSize() Returns the occupied size of data blocks, in bytes.
long	getCurrentSize() Returns the occupied size of the block cache, in bytes.
long	getDataBlockCount() Returns the number of data blocks currently cached in the block cache.
FirstLevelBlockCache	getFirstLevelCache()
long	getFreeSize() Returns the free size of the block cache, in bytes.
Optional<Map<String,Pair<String,Long>>>	getFullyCachedFiles() Returns the list of fully cached files
long	getMaxSize() Returns the Max size of the block cache, in bytes.
Optional<Map<String,Long>>	getRegionCachedInfo() Returns an Optional containing a map of regions and the percentage of how much of it has been cached so far.
int	getRpcRefCount(BlockCacheKey cacheKey)
BlockCache	getSecondLevelCache()
CacheStats	getStats() Get the statistics for this block cache.
long	heapSize()
Optional<Boolean>	isAlreadyCached(BlockCacheKey key) Checks whether the block for the passed key is already cached.
Iterator<CachedBlock>	iterator() Returns Iterator over the blocks in the cache.
void	notifyFileCachingCompleted(org.apache.hadoop.fs.Path fileName, int totalBlockCount, int dataBlockCount, long size) Notifies the cache implementation that the given file has been fully cached (all its blocks made into the cache).
void	setMaxSize(long size) Sets the max heap size that can be used by the BlockCache.
Optional<Boolean>	shouldCacheFile(String fileName) Checks whether blocks for the passed file should be cached or not.
void	shutdown() Shutdown the cache.
long	size() Returns the total size of the block cache, in bytes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.io.hfile.BlockCache

isMetaBlock

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

l1Cache

protected final FirstLevelBlockCache l1Cache

l2Cache

protected final BlockCache l2Cache

combinedCacheStats

protected final CombinedBlockCache.CombinedCacheStats combinedCacheStats

Constructor Detail

CombinedBlockCache

public CombinedBlockCache(FirstLevelBlockCache l1Cache,
                          BlockCache l2Cache)

Method Detail

heapSize

public long heapSize()

Specified by:

heapSize in interface HeapSize

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
                       Cacheable buf,
                       boolean inMemory)

Description copied from interface: BlockCache

Add block to cache.

Specified by:

cacheBlock in interface BlockCache

Parameters:

cacheKey - The block's cache key.

buf - The block contents wrapped in a ByteBuffer.

inMemory - Whether block should be treated as in-memory

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
                       Cacheable buf,
                       boolean inMemory,
                       boolean waitWhenCache)

Description copied from interface: BlockCache

Add block to cache.

Specified by:

cacheBlock in interface BlockCache

Parameters:

cacheKey - The block's cache key.

buf - The block contents wrapped in a ByteBuffer.

inMemory - Whether block should be treated as in-memory

waitWhenCache - Whether to wait for the cache to be flushed mainly when BucketCache is configured.

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
                       Cacheable buf)

Description copied from interface: BlockCache

Add block to cache (defaults to not in-memory).

Specified by:

cacheBlock in interface BlockCache

Parameters:

cacheKey - The block's cache key.

buf - The object to cache.

getBlock

public Cacheable getBlock(BlockCacheKey cacheKey,
                          boolean caching,
                          boolean repeat,
                          boolean updateCacheMetrics)

Description copied from interface: BlockCache

Fetch block from cache.

Specified by:

getBlock in interface BlockCache

Parameters:

cacheKey - Block to fetch.

caching - Whether this request has caching enabled (used for stats)

repeat - Whether this is a repeat lookup for the same block (used to avoid double counting cache misses when doing double-check locking)

updateCacheMetrics - Whether to update cache metrics or not

Returns:

Block or null if block is not in 2 cache.

evictBlock

public boolean evictBlock(BlockCacheKey cacheKey)

Description copied from interface: BlockCache

Evict block from cache.

Specified by:

evictBlock in interface BlockCache

Parameters:

cacheKey - Block to evict

Returns:

true if block existed and was evicted, false if not

evictBlocksByHfileName

public int evictBlocksByHfileName(String hfileName)

Description copied from interface: BlockCache

Evicts all blocks for the given HFile.

Specified by:

evictBlocksByHfileName in interface BlockCache

Returns:

the number of blocks evicted

getStats

public CacheStats getStats()

Description copied from interface: BlockCache

Get the statistics for this block cache.

Specified by:

getStats in interface BlockCache

shutdown

public void shutdown()

Description copied from interface: BlockCache

Shutdown the cache.

Specified by:

shutdown in interface BlockCache

size

public long size()

Description copied from interface: BlockCache

Returns the total size of the block cache, in bytes.

Specified by:

size in interface BlockCache

Returns:

size of cache, in bytes

getMaxSize

public long getMaxSize()

Description copied from interface: BlockCache

Returns the Max size of the block cache, in bytes.

Specified by:

getMaxSize in interface BlockCache

Returns:

size of cache, in bytes

getCurrentDataSize

public long getCurrentDataSize()

Description copied from interface: BlockCache

Returns the occupied size of data blocks, in bytes.

Specified by:

getCurrentDataSize in interface BlockCache

Returns:

occupied space in cache, in bytes

getFreeSize

public long getFreeSize()

Description copied from interface: BlockCache

Returns the free size of the block cache, in bytes.

Specified by:

getFreeSize in interface BlockCache

Returns:

free space in cache, in bytes

getCurrentSize

public long getCurrentSize()

Description copied from interface: BlockCache

Returns the occupied size of the block cache, in bytes.

Specified by:

getCurrentSize in interface BlockCache

Returns:

occupied space in cache, in bytes

getBlockCount

public long getBlockCount()

Description copied from interface: BlockCache

Returns the number of blocks currently cached in the block cache.

Specified by:

getBlockCount in interface BlockCache

Returns:

number of blocks in the cache

getDataBlockCount

public long getDataBlockCount()

Description copied from interface: BlockCache

Returns the number of data blocks currently cached in the block cache.

Specified by:

getDataBlockCount in interface BlockCache

Returns:

number of blocks in the cache

iterator

public Iterator<CachedBlock> iterator()

Description copied from interface: BlockCache

Returns Iterator over the blocks in the cache.

Specified by:

iterator in interface Iterable<CachedBlock>

Specified by:

iterator in interface BlockCache

getBlockCaches

public BlockCache[] getBlockCaches()

Description copied from interface: BlockCache

Returns The list of sub blockcaches that make up this one; returns null if no sub caches.

Specified by:

getBlockCaches in interface BlockCache

getFullyCachedFiles

public Optional<Map<String,Pair<String,Long>>> getFullyCachedFiles()

Returns the list of fully cached files

Specified by:

getFullyCachedFiles in interface BlockCache

Returns:

empty optional if this method is not supported, otherwise the returned optional contains a map of all files that have been fully cached.

getRegionCachedInfo

public Optional<Map<String,Long>> getRegionCachedInfo()

Description copied from interface: BlockCache

Returns an Optional containing a map of regions and the percentage of how much of it has been cached so far.

Specified by:

getRegionCachedInfo in interface BlockCache

Returns:

empty optional if this method is not supported, otherwise the returned optional contains a map of current regions caching percentage.

setMaxSize

public void setMaxSize(long size)

Description copied from interface: ResizableBlockCache

Sets the max heap size that can be used by the BlockCache.

Specified by:

setMaxSize in interface ResizableBlockCache

Parameters:

size - The max heap size.

getRpcRefCount

public int getRpcRefCount(BlockCacheKey cacheKey)

getFirstLevelCache

public FirstLevelBlockCache getFirstLevelCache()

getSecondLevelCache

public BlockCache getSecondLevelCache()

notifyFileCachingCompleted

public void notifyFileCachingCompleted(org.apache.hadoop.fs.Path fileName,
                                       int totalBlockCount,
                                       int dataBlockCount,
                                       long size)

Description copied from interface: BlockCache

Notifies the cache implementation that the given file has been fully cached (all its blocks made into the cache).

Specified by:

notifyFileCachingCompleted in interface BlockCache

Parameters:

fileName - the file that has been completely cached.

totalBlockCount - the total of blocks cached for this file.

dataBlockCount - number of DATA block type cached.

size - the size, in bytes, cached.

blockFitsIntoTheCache

public Optional<Boolean> blockFitsIntoTheCache(HFileBlock block)

Description copied from interface: BlockCache

Checks whether there's enough space left in the cache to accommodate the passed block. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the passed block fits into the remaining cache space available.

Specified by:

blockFitsIntoTheCache in interface BlockCache

Parameters:

block - the block we want to check if fits into the cache.

Returns:

empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block fits into the cache available space.

shouldCacheFile

public Optional<Boolean> shouldCacheFile(String fileName)

Description copied from interface: BlockCache

Checks whether blocks for the passed file should be cached or not. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the passed file should indeed be cached.

Specified by:

shouldCacheFile in interface BlockCache

Parameters:

fileName - to check if it should be cached.

Returns:

empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the file should be cached.

isAlreadyCached

public Optional<Boolean> isAlreadyCached(BlockCacheKey key)

Description copied from interface: BlockCache

Checks whether the block for the passed key is already cached. This method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty. For subclasses implementing this logic, the returned Optional would contain the boolean value reflecting if the block for the passed key is already cached or not.

Specified by:

isAlreadyCached in interface BlockCache

Parameters:

key - for the block we want to check if it's already in the cache.

Returns:

empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block is already cached.

getBlockSize

public Optional<Integer> getBlockSize(BlockCacheKey key)

Description copied from interface: BlockCache

Returns an Optional containing the size of the block related to the passed key. If the block is not in the cache, returned optional will be empty. Also, this method may not be overridden by all implementing classes. In such cases, the returned Optional will be empty.

Specified by:

getBlockSize in interface BlockCache

Parameters:

key - for the block we want to check if it's already in the cache.

Returns:

empty optional if this method is not supported, otherwise the returned optional contains the boolean value informing if the block is already cached.