| Package | Description |
|---|---|
| org.apache.hadoop.hive.llap | |
| org.apache.hadoop.hive.ql.exec |
Hive QL execution tasks, operators, functions and other handlers.
|
| org.apache.hadoop.hive.ql.exec.mr | |
| org.apache.hadoop.hive.ql.exec.tez | |
| org.apache.hadoop.hive.ql.exec.vector | |
| org.apache.hadoop.hive.ql.io | |
| org.apache.hadoop.hive.ql.io.merge | |
| org.apache.hadoop.hive.ql.io.orc |
The Optimized Row Columnar (ORC) File Format.
|
| org.apache.hadoop.hive.ql.io.rcfile.truncate | |
| org.apache.hadoop.hive.ql.optimizer | |
| org.apache.hadoop.hive.ql.optimizer.physical | |
| org.apache.hadoop.hive.ql.parse | |
| org.apache.hadoop.hive.ql.plan |
| Modifier and Type | Method and Description |
|---|---|
static MapWork |
LlapHiveUtils.findMapWork(org.apache.hadoop.mapred.JobConf job)
Returns MapWork based what is serialized in the JobConf instance provided.
|
| Modifier and Type | Method and Description |
|---|---|
static MapWork |
Utilities.getMapWork(org.apache.hadoop.conf.Configuration conf) |
| Modifier and Type | Method and Description |
|---|---|
Collection<MapWork> |
Task.getMapWork() |
| Modifier and Type | Method and Description |
|---|---|
static void |
Utilities.cacheMapWork(org.apache.hadoop.conf.Configuration conf,
MapWork work,
org.apache.hadoop.fs.Path hiveScratchDir) |
static void |
Utilities.createTmpDirs(org.apache.hadoop.conf.Configuration conf,
MapWork mWork)
Hive uses tmp directories to capture the output of each FileSinkOperator.
|
static int |
Utilities.estimateNumberOfReducers(HiveConf conf,
org.apache.hadoop.fs.ContentSummary inputSummary,
MapWork work,
boolean finalMapRed)
Estimate the number of reducers needed for this job, based on job input,
and configuration parameters.
|
static double |
Utilities.getHighestSamplePercentage(MapWork work)
Returns the highest sample percentage of any alias in the given MapWork
|
static List<org.apache.hadoop.fs.Path> |
Utilities.getInputPaths(org.apache.hadoop.mapred.JobConf job,
MapWork work,
org.apache.hadoop.fs.Path hiveScratchDir,
Context ctx,
boolean skipDummy)
Computes a list of all input paths needed to compute the given MapWork.
|
static List<org.apache.hadoop.fs.Path> |
Utilities.getInputPathsTez(org.apache.hadoop.mapred.JobConf job,
MapWork work)
On Tez we're not creating dummy files when getting/setting input paths.
|
static org.apache.hadoop.fs.ContentSummary |
Utilities.getInputSummary(Context ctx,
MapWork work,
org.apache.hadoop.fs.PathFilter filter)
Calculate the total size of input files.
|
static boolean |
Utilities.getIsVectorized(org.apache.hadoop.conf.Configuration conf,
MapWork mapWork) |
Operator<? extends OperatorDesc> |
Task.getReducer(MapWork work) |
static long |
Utilities.getTotalInputFileSize(org.apache.hadoop.fs.ContentSummary inputSummary,
MapWork work,
double highestSamplePercentage)
Computes the total input file size.
|
static long |
Utilities.getTotalInputNumFiles(org.apache.hadoop.fs.ContentSummary inputSummary,
MapWork work,
double highestSamplePercentage)
Computes the total number of input files.
|
static void |
Utilities.setInputAttributes(org.apache.hadoop.conf.Configuration conf,
MapWork mWork)
Set hive input format, and input format file if necessary.
|
static void |
Utilities.setMapWork(org.apache.hadoop.conf.Configuration conf,
MapWork work) |
static org.apache.hadoop.fs.Path |
Utilities.setMapWork(org.apache.hadoop.conf.Configuration conf,
MapWork w,
org.apache.hadoop.fs.Path hiveScratchDir,
boolean useCache) |
| Modifier and Type | Method and Description |
|---|---|
Collection<MapWork> |
ExecDriver.getMapWork() |
| Modifier and Type | Method and Description |
|---|---|
Operator<? extends OperatorDesc> |
MapRedTask.getReducer(MapWork mapWork) |
static void |
ExecDriver.propagateSplitSettings(org.apache.hadoop.mapred.JobConf job,
MapWork work) |
| Modifier and Type | Method and Description |
|---|---|
Collection<MapWork> |
TezTask.getMapWork() |
| Modifier and Type | Method and Description |
|---|---|
Operator<? extends OperatorDesc> |
TezTask.getReducer(MapWork mapWork) |
void |
DynamicPartitionPruner.initialize(org.apache.tez.runtime.api.InputInitializerContext context,
MapWork work,
org.apache.hadoop.mapred.JobConf jobConf) |
| Constructor and Description |
|---|
HiveSplitGenerator(org.apache.hadoop.conf.Configuration conf,
MapWork work,
boolean generateSingleSplit) |
HiveSplitGenerator(org.apache.hadoop.conf.Configuration conf,
MapWork work,
boolean generateSingleSplit,
Integer numSplits) |
| Modifier and Type | Method and Description |
|---|---|
static void |
VectorizedRowBatchCtx.getPartitionValues(VectorizedRowBatchCtx vrbCtx,
MapWork mapWork,
org.apache.hadoop.mapred.FileSplit split,
Object[] partitionValues) |
| Modifier and Type | Field and Description |
|---|---|
protected MapWork |
HiveInputFormat.mrwork |
| Modifier and Type | Method and Description |
|---|---|
static void |
HiveInputFormat.pushFiltersAndAsOf(org.apache.hadoop.mapred.JobConf jobConf,
TableScanOperator tableScan,
MapWork mrwork) |
| Modifier and Type | Class and Description |
|---|---|
class |
MergeFileWork |
| Constructor and Description |
|---|
VectorizedOrcAcidRowBatchReader(OrcSplit inputSplit,
org.apache.hadoop.mapred.JobConf conf,
org.apache.hadoop.mapred.Reporter reporter,
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.NullWritable,VectorizedRowBatch> baseReader,
VectorizedRowBatchCtx rbCtx,
boolean isFlatPayload,
MapWork mapWork)
LLAP IO c'tor
|
| Modifier and Type | Class and Description |
|---|---|
class |
ColumnTruncateWork |
| Modifier and Type | Method and Description |
|---|---|
static MapWork |
GenMapRedUtils.createMergeTask(FileSinkDesc fsInputDesc,
org.apache.hadoop.fs.Path finalName,
boolean hasDynamicPartitions,
CompilationOpContext ctx)
Create a block level merge task for RCFiles or stripe level merge task for
ORCFiles
|
| Modifier and Type | Method and Description |
|---|---|
static String |
GenMapRedUtils.findAlias(MapWork work,
Operator<?> operator) |
static Set<String> |
GenMapRedUtils.findAliases(MapWork work,
Operator<?> startOp) |
static void |
GenMapRedUtils.replaceMapWork(String sourceAlias,
String targetAlias,
MapWork source,
MapWork target)
Replace the Map-side operator tree associated with targetAlias in
target with the Map-side operator tree associated with sourceAlias in source.
|
static void |
GenMapRedUtils.setMapWork(MapWork plan,
ParseContext parseCtx,
Set<ReadEntity> inputs,
PrunedPartitionList partsList,
TableScanOperator tsOp,
String alias_id,
HiveConf conf,
boolean local)
initialize MapWork
|
static void |
GenMapRedUtils.setTaskPlan(org.apache.hadoop.fs.Path path,
String alias,
Operator<? extends OperatorDesc> topOp,
MapWork plan,
boolean local,
TableDesc ttDesc)
set the current task in the mapredWork.
|
| Modifier and Type | Method and Description |
|---|---|
long |
AbstractJoinTaskDispatcher.getTotalKnownInputSize(Context context,
MapWork currWork,
Map<org.apache.hadoop.fs.Path,List<String>> pathToAliases,
Map<String,Long> aliasToSize) |
| Modifier and Type | Method and Description |
|---|---|
MapWork |
GenTezUtils.createMapWork(GenTezProcContext context,
Operator<?> root,
TezWork tezWork,
PrunedPartitionList partitions) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
GenTezUtils.setupMapWork(MapWork mapWork,
GenTezProcContext context,
PrunedPartitionList partitions,
TableScanOperator root,
String alias) |
| Modifier and Type | Method and Description |
|---|---|
MapWork |
MapredWork.getMapWork() |
| Modifier and Type | Method and Description |
|---|---|
void |
MapWork.mergingInto(MapWork mapWork) |
void |
MapredWork.setMapWork(MapWork mapWork) |
| Constructor and Description |
|---|
MapExplainVectorization(MapWork mapWork) |
Copyright © 2024 The Apache Software Foundation. All rights reserved.