org.apache.hadoop.hbase.mapreduce

类 Import

java.lang.Object

org.apache.hadoop.hbase.mapreduce.Import

@InterfaceAudience.Public
@InterfaceStability.Stable
public class Import
extends Object

Import data written by Export.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	Import.Importer Write table content out to files in hdfs.
static class	Import.KeyValueImporter A mapper that just writes out KeyValues.
static class	Import.KeyValueReducer
static class	Import.KeyValueSortImporter
static class	Import.KeyValueWritableComparable
static class	Import.KeyValueWritableComparablePartitioner

字段概要

字段

限定符和类型	字段和说明
static String	BULK_OUTPUT_CONF_KEY
static String	CF_RENAME_PROP
static String	FILTER_ARGS_CONF_KEY
static String	FILTER_CLASS_CONF_KEY
static String	HAS_LARGE_RESULT
static String	TABLE_NAME
static String	WAL_DURABILITY

构造器概要

构造器

构造器和说明
Import()

方法概要

方法

限定符和类型	方法和说明
static void	addFilterAndArguments(org.apache.hadoop.conf.Configuration conf, Class<? extends Filter> clazz, List<String> filterArgs) Add a Filter to be instantiated on import
static void	configureCfRenaming(org.apache.hadoop.conf.Configuration conf, Map<String,String> renameMap) Sets a configuration property with key CF_RENAME_PROP in conf that tells the mapper how to rename column families.
static org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args) Sets up the actual job.
static Cell	filterKv(Filter filter, Cell kv) Attempt to filter out the keyvalue
static void	flushRegionsIfNecessary(org.apache.hadoop.conf.Configuration conf) If the durability is set to Durability.SKIP_WAL and the data is imported to hbase, we need to flush all the regions of the table as the data is held in memory and is also not present in the Write Ahead Log to replay in scenarios of a crash.
static Filter	instantiateFilter(org.apache.hadoop.conf.Configuration conf) Create a Filter to apply to all incoming keys (KeyValues) to optionally not include in the job output
static void	main(String[] args) Main entry point.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

CF_RENAME_PROP

public static final String CF_RENAME_PROP

另请参阅:

常量字段值

BULK_OUTPUT_CONF_KEY

public static final String BULK_OUTPUT_CONF_KEY

另请参阅:

常量字段值

FILTER_CLASS_CONF_KEY

public static final String FILTER_CLASS_CONF_KEY

另请参阅:

常量字段值

FILTER_ARGS_CONF_KEY

public static final String FILTER_ARGS_CONF_KEY

另请参阅:

常量字段值

TABLE_NAME

public static final String TABLE_NAME

另请参阅:

常量字段值

WAL_DURABILITY

public static final String WAL_DURABILITY

另请参阅:

常量字段值

HAS_LARGE_RESULT

public static final String HAS_LARGE_RESULT

另请参阅:

常量字段值

构造器详细资料

Import

public Import()

方法详细资料

instantiateFilter

public static Filter instantiateFilter(org.apache.hadoop.conf.Configuration conf)

Create a Filter to apply to all incoming keys (KeyValues) to optionally not include in the job output

参数:

conf - Configuration from which to load the filter

返回:

the filter to use for the task, or null if no filter to should be used

抛出:

IllegalArgumentException - if the filter is misconfigured

filterKv

public static Cell filterKv(Filter filter,
            Cell kv)
                     throws IOException

Attempt to filter out the keyvalue

参数:

kv - KeyValue on which to apply the filter

返回:

null if the key should not be written, otherwise returns the original KeyValue

抛出:

IOException

configureCfRenaming

public static void configureCfRenaming(org.apache.hadoop.conf.Configuration conf,
                       Map<String,String> renameMap)

Sets a configuration property with key CF_RENAME_PROP in conf that tells the mapper how to rename column families.

Alternately, instead of calling this function, you could set the configuration key CF_RENAME_PROP yourself. The value should look like

srcCf1:destCf1,srcCf2:destCf2,....

. This would have the same effect on the mapper behavior.

参数:

conf - the Configuration in which the CF_RENAME_PROP key will be set

renameMap - a mapping from source CF names to destination CF names

addFilterAndArguments

public static void addFilterAndArguments(org.apache.hadoop.conf.Configuration conf,
                         Class<? extends Filter> clazz,
                         List<String> filterArgs)
                                  throws IOException

Add a Filter to be instantiated on import

参数:

conf - Configuration to update (will be passed to the job)

clazz - Filter subclass to instantiate on the server.

filterArgs - List of arguments to pass to the filter on instantiation

抛出:

IOException

createSubmittableJob

public static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
                                                   String[] args)
                                                            throws IOException

Sets up the actual job.

参数:

conf - The current configuration.

args - The command line parameters.

返回:

The newly created job.

抛出:

IOException - When setting up the job fails.

flushRegionsIfNecessary

public static void flushRegionsIfNecessary(org.apache.hadoop.conf.Configuration conf)
                                    throws IOException,
                                           InterruptedException

If the durability is set to Durability.SKIP_WAL and the data is imported to hbase, we need to flush all the regions of the table as the data is held in memory and is also not present in the Write Ahead Log to replay in scenarios of a crash. This method flushes all the regions of the table in the scenarios of import data to hbase with Durability.SKIP_WAL

抛出:

IOException

InterruptedException

main

public static void main(String[] args)
                 throws Exception

Main entry point.

参数:

args - The command line parameters.

抛出:

Exception - When running the job fails.