| Package | Description |
|---|---|
| org.apache.hadoop.hive.ql.io | |
| org.apache.hadoop.hive.ql.txn.compactor | |
| org.apache.hadoop.hive.ql.txn.compactor.service |
| Modifier and Type | Method and Description |
|---|---|
static AcidDirectory |
AcidUtils.getAcidState(org.apache.hadoop.fs.FileSystem fileSystem,
org.apache.hadoop.fs.Path candidateDirectory,
org.apache.hadoop.conf.Configuration conf,
ValidWriteIdList writeIdList,
Ref<Boolean> useFileIds,
boolean ignoreEmptyFiles)
Get the ACID state of the given directory.
|
static AcidDirectory |
AcidUtils.getAcidState(org.apache.hadoop.fs.FileSystem fileSystem,
org.apache.hadoop.fs.Path candidateDirectory,
org.apache.hadoop.conf.Configuration conf,
ValidWriteIdList writeIdList,
Ref<Boolean> useFileIds,
boolean ignoreEmptyFiles,
Map<org.apache.hadoop.fs.Path,AcidUtils.HdfsDirSnapshot> dirSnapshots)
GetAcidState implementation which uses the provided dirSnapshot.
|
static AcidDirectory |
AcidUtils.getAcidStateFromCache(Supplier<org.apache.hadoop.fs.FileSystem> fileSystem,
org.apache.hadoop.fs.Path candidateDirectory,
org.apache.hadoop.conf.Configuration conf,
ValidWriteIdList writeIdList,
Ref<Boolean> useFileIds,
boolean ignoreEmptyFiles)
Tries to get directory details from cache.
|
| Modifier and Type | Method and Description |
|---|---|
AcidDirectory |
CompactorContext.getAcidDirectory() |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.hadoop.fs.Path |
QueryCompactor.Util.getCompactionResultDir(org.apache.hadoop.hive.metastore.api.StorageDescriptor sd,
ValidWriteIdList writeIds,
HiveConf conf,
boolean writingBase,
boolean createDeleteDelta,
boolean bucket0,
AcidDirectory directory)
Get the path of the base, delta, or delete delta directory that will be the final
destination of the files during compaction.
|
static List<org.apache.hadoop.fs.Path> |
CompactorUtil.getObsoleteDirs(AcidDirectory dir,
boolean isDynPartAbort) |
static boolean |
CompactorUtil.isMinorCompactionSupported(HiveConf conf,
Map<String,String> tblproperties,
AcidDirectory dir)
Check for that special case when minor compaction is supported or not.
|
void |
MRCompactor.run(HiveConf conf,
org.apache.hadoop.hive.metastore.api.Table t,
org.apache.hadoop.hive.metastore.api.StorageDescriptor sd,
ValidWriteIdList writeIds,
CompactionInfo ci,
AcidDirectory dir)
Run Compaction which may consist of several jobs on the cluster.
|
| Constructor and Description |
|---|
CompactorContext(HiveConf conf,
org.apache.hadoop.hive.metastore.api.Table table,
org.apache.hadoop.hive.metastore.api.Partition p,
org.apache.hadoop.hive.metastore.api.StorageDescriptor sd,
ValidWriteIdList tblValidWriteIds,
CompactionInfo ci,
AcidDirectory dir) |
| Modifier and Type | Method and Description |
|---|---|
static boolean |
AcidCompactionService.needsCleaning(AcidDirectory dir,
org.apache.hadoop.hive.metastore.api.StorageDescriptor sd)
Check for obsolete directories, and return true if any exist and Cleaner should be
run.
|
Copyright © 2024 The Apache Software Foundation. All rights reserved.