org.apache.hadoop.hbase.mapreduce

类 MultithreadedTableMapper<K2,V2>

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,KEYOUT,VALUEOUT>

org.apache.hadoop.hbase.mapreduce.TableMapper<K2,V2>

org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper<K2,V2>

public class MultithreadedTableMapper<K2,V2>
extends TableMapper<K2,V2>

Multithreaded implementation for @link org.apache.hbase.mapreduce.TableMapper

It can be used instead when the Map operation is not CPU bound in order to improve throughput.

Mapper implementations using this MapRunnable must be thread-safe.

The Map-Reduce job has to be configured with the mapper to use via setMapperClass(org.apache.hadoop.mapreduce.Job, java.lang.Class<? extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.hbase.io.ImmutableBytesWritable, org.apache.hadoop.hbase.client.Result, K2, V2>>) and the number of thread the thread-pool can use with the getNumberOfThreads(org.apache.hadoop.mapreduce.JobContext) method. The default value is 10 threads.

嵌套类概要

从类继承的嵌套类/接口 org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

字段概要

字段

限定符和类型	字段和说明
static String	MAPPER_CLASS
static String	NUMBER_OF_THREADS

构造器概要

构造器

构造器和说明
MultithreadedTableMapper()

方法概要

方法

限定符和类型	方法和说明
static <K2,V2> Class<org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,K2,V2>>	getMapperClass(org.apache.hadoop.mapreduce.JobContext job) Get the application's mapper class.
static int	getNumberOfThreads(org.apache.hadoop.mapreduce.JobContext job) The number of threads in the thread pool that will run the map function.
void	run(org.apache.hadoop.mapreduce.Mapper.Context context) Run the application's maps using a thread pool.
static <K2,V2> void	setMapperClass(org.apache.hadoop.mapreduce.Job job, Class<? extends org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,K2,V2>> cls) Set the application's mapper class.
static void	setNumberOfThreads(org.apache.hadoop.mapreduce.Job job, int threads) Set the number of threads in the pool for running maps.

从类继承的方法 org.apache.hadoop.mapreduce.Mapper

cleanup, map, setup

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

NUMBER_OF_THREADS

public static final String NUMBER_OF_THREADS

另请参阅:

常量字段值

MAPPER_CLASS

public static final String MAPPER_CLASS

另请参阅:

常量字段值

构造器详细资料

MultithreadedTableMapper

public MultithreadedTableMapper()

方法详细资料

getNumberOfThreads

public static int getNumberOfThreads(org.apache.hadoop.mapreduce.JobContext job)

The number of threads in the thread pool that will run the map function.

参数:

job - the job

返回:

the number of threads

setNumberOfThreads

public static void setNumberOfThreads(org.apache.hadoop.mapreduce.Job job,
                      int threads)

Set the number of threads in the pool for running maps.

参数:

job - the job to modify

threads - the new number of threads

getMapperClass

public static <K2,V2> Class<org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,K2,V2>> getMapperClass(org.apache.hadoop.mapreduce.JobContext job)

Get the application's mapper class.

类型参数:

K2 - the map's output key type

V2 - the map's output value type

参数:

job - the job

返回:

the mapper class to run

setMapperClass

public static <K2,V2> void setMapperClass(org.apache.hadoop.mapreduce.Job job,
                          Class<? extends org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,K2,V2>> cls)

Set the application's mapper class.

类型参数:

K2 - the map output key type

V2 - the map output value type

参数:

job - the job to modify

cls - the class to use as the mapper

run

public void run(org.apache.hadoop.mapreduce.Mapper.Context context)
         throws IOException,
                InterruptedException

Run the application's maps using a thread pool.

覆盖:

run 在类中 org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,Result,K2,V2>

抛出:

IOException

InterruptedException