org.apache.hadoop.hbase.regionserver.throttle

类 StoreHotnessProtector

java.lang.Object

org.apache.hadoop.hbase.regionserver.throttle.StoreHotnessProtector

@InterfaceAudience.Private
public class StoreHotnessProtector
extends Object

StoreHotnessProtector is designed to help limit the concurrency of puts with dense columns, it does best-effort to avoid exhausting all RS's handlers. When a lot of clients write requests with dense (hundreds) columns to a Store at the same time, it will lead to blocking of RS because CSLM degrades when concurrency goes up. It's not a kind of throttling. Throttling is user-oriented, while StoreHotnessProtector is system-oriented, RS-self-protected mechanism.

There are three key parameters:

1. parallelPutToStoreThreadLimitCheckMinColumnCount: If the amount of columns exceed this threshold, the HotProtector will work, 100 by default

2. parallelPutToStoreThreadLimit: The amount of concurrency allowed to write puts to a Store at the same time.

3. parallelPreparePutToStoreThreadLimit: The amount of concurrency allowed to prepare writing puts to a Store at the same time.

Notice that our writing pipeline includes three key process: MVCC acquire, writing MemStore, and WAL. Only limit the concurrency of writing puts to Store(parallelPutToStoreThreadLimit) is not enough since the actual concurrency of puts may still exceed the limit when MVCC contention or slow WAL sync happens. This is why parallelPreparePutToStoreThreadLimit is needed.

This protector is enabled by default and could be turned off by setting hbase.region.store.parallel.put.limit to 0, supporting online configuration change.

字段概要

字段

限定符和类型	字段和说明
static long	FIXED_SIZE
static String	PARALLEL_PREPARE_PUT_STORE_MULTIPLIER
static String	PARALLEL_PUT_STORE_THREADS_LIMIT
static String	PARALLEL_PUT_STORE_THREADS_LIMIT_MIN_COLUMN_COUNT

构造器概要

构造器

构造器和说明
StoreHotnessProtector(Region region, org.apache.hadoop.conf.Configuration conf)

方法概要

限定符和类型	方法和说明
void	finish(Map<byte[],List<Cell>> familyMaps)
void	init(org.apache.hadoop.conf.Configuration conf)
boolean	isEnable()
void	start(Map<byte[],List<Cell>> familyMaps)
String	toString()
void	update(org.apache.hadoop.conf.Configuration conf)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

字段详细资料

PARALLEL_PUT_STORE_THREADS_LIMIT

public static final String PARALLEL_PUT_STORE_THREADS_LIMIT

另请参阅:

常量字段值

PARALLEL_PREPARE_PUT_STORE_MULTIPLIER

public static final String PARALLEL_PREPARE_PUT_STORE_MULTIPLIER

另请参阅:

常量字段值

PARALLEL_PUT_STORE_THREADS_LIMIT_MIN_COLUMN_COUNT

public static final String PARALLEL_PUT_STORE_THREADS_LIMIT_MIN_COLUMN_COUNT

另请参阅:

常量字段值

FIXED_SIZE

public static final long FIXED_SIZE

构造器详细资料

StoreHotnessProtector

public StoreHotnessProtector(Region region,
                             org.apache.hadoop.conf.Configuration conf)

方法详细资料

init

public void init(org.apache.hadoop.conf.Configuration conf)

update

public void update(org.apache.hadoop.conf.Configuration conf)

start

public void start(Map<byte[],List<Cell>> familyMaps)
           throws RegionTooBusyException

抛出:

RegionTooBusyException

finish

public void finish(Map<byte[],List<Cell>> familyMaps)

toString

public String toString()

覆盖:

toString 在类中 Object

isEnable

public boolean isEnable()