org.apache.hadoop.hive.ql.txn.compactor

Class CompactorUtil

java.lang.Object

org.apache.hadoop.hive.ql.txn.compactor.CompactorUtil

public class CompactorUtil
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CompactorUtil.METADATA_FETCH_MODE
static interface	CompactorUtil.ThrowingRunnable<E extends Exception>

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMPACTOR

Constructor Summary

Constructors

Constructor and Description
CompactorUtil()

Method Summary

Modifier and Type	Method and Description
static void	checkInterrupt(String callerClassName)
static ExecutorService	createExecutorWithThreadFactory(int parallelism, String threadNameFormat)
static org.apache.hadoop.hive.metastore.api.LockRequest	createLockRequest(HiveConf conf, CompactionInfo ci, long txnId, org.apache.hadoop.hive.metastore.api.LockType lockType, org.apache.hadoop.hive.metastore.api.DataOperationType opType)
static String	getDebugInfo(List<org.apache.hadoop.fs.Path> paths)
static String	getFullPartitionName(String dbName, String tableName, String partName)
static List<org.apache.hadoop.fs.Path>	getObsoleteDirs(AcidDirectory dir, boolean isDynPartAbort)
static List<org.apache.hadoop.hive.metastore.api.Partition>	getPartitionsByNames(HiveConf conf, String dbName, String tableName, String partName)
static boolean	isDynPartAbort(org.apache.hadoop.hive.metastore.api.Table t, String partName)
static boolean	isMinorCompactionSupported(HiveConf conf, Map<String,String> tblproperties, AcidDirectory dir) Check for that special case when minor compaction is supported or not.
static org.apache.hadoop.hive.metastore.api.Database	resolveDatabase(HiveConf conf, String dbName)
static org.apache.hadoop.hive.metastore.api.Partition	resolvePartition(HiveConf conf, IMetaStoreClient msc, String dbName, String tableName, String partName, CompactorUtil.METADATA_FETCH_MODE fetchMode)
static org.apache.hadoop.hive.metastore.api.StorageDescriptor	resolveStorageDescriptor(org.apache.hadoop.hive.metastore.api.Table t)
static org.apache.hadoop.hive.metastore.api.StorageDescriptor	resolveStorageDescriptor(org.apache.hadoop.hive.metastore.api.Table t, org.apache.hadoop.hive.metastore.api.Partition p)
static org.apache.hadoop.hive.metastore.api.Table	resolveTable(HiveConf conf, String dbName, String tableName)
static boolean	runJobAsSelf(String owner) Determine whether to run this job as the current user or whether we need a doAs to switch users.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COMPACTOR

public static final String COMPACTOR

See Also:

Constant Field Values

Constructor Detail

CompactorUtil

public CompactorUtil()

Method Detail

createExecutorWithThreadFactory

public static ExecutorService createExecutorWithThreadFactory(int parallelism,
                                                              String threadNameFormat)

resolveStorageDescriptor

public static org.apache.hadoop.hive.metastore.api.StorageDescriptor resolveStorageDescriptor(org.apache.hadoop.hive.metastore.api.Table t,
                                                                                              org.apache.hadoop.hive.metastore.api.Partition p)

resolveStorageDescriptor

public static org.apache.hadoop.hive.metastore.api.StorageDescriptor resolveStorageDescriptor(org.apache.hadoop.hive.metastore.api.Table t)

isDynPartAbort

public static boolean isDynPartAbort(org.apache.hadoop.hive.metastore.api.Table t,
                                     String partName)

getPartitionsByNames

public static List<org.apache.hadoop.hive.metastore.api.Partition> getPartitionsByNames(HiveConf conf,
                                                                                        String dbName,
                                                                                        String tableName,
                                                                                        String partName)
                                                                                 throws org.apache.hadoop.hive.metastore.api.MetaException

Throws:

org.apache.hadoop.hive.metastore.api.MetaException

resolveDatabase

public static org.apache.hadoop.hive.metastore.api.Database resolveDatabase(HiveConf conf,
                                                                            String dbName)
                                                                     throws org.apache.hadoop.hive.metastore.api.MetaException,
                                                                            org.apache.hadoop.hive.metastore.api.NoSuchObjectException

Throws:

org.apache.hadoop.hive.metastore.api.MetaException

org.apache.hadoop.hive.metastore.api.NoSuchObjectException

resolveTable

public static org.apache.hadoop.hive.metastore.api.Table resolveTable(HiveConf conf,
                                                                      String dbName,
                                                                      String tableName)
                                                               throws org.apache.hadoop.hive.metastore.api.MetaException

Throws:

org.apache.hadoop.hive.metastore.api.MetaException

getDebugInfo

public static String getDebugInfo(List<org.apache.hadoop.fs.Path> paths)

runJobAsSelf

public static boolean runJobAsSelf(String owner)

Determine whether to run this job as the current user or whether we need a doAs to switch users.

Parameters:

owner - of the directory we will be working in, as determined by TxnUtils.findUserToRunAs(String, Table, Configuration)

Returns:

true if the job should run as the current user, false if a doAs is needed.

getObsoleteDirs

public static List<org.apache.hadoop.fs.Path> getObsoleteDirs(AcidDirectory dir,
                                                              boolean isDynPartAbort)

resolvePartition

public static org.apache.hadoop.hive.metastore.api.Partition resolvePartition(HiveConf conf,
                                                                              IMetaStoreClient msc,
                                                                              String dbName,
                                                                              String tableName,
                                                                              String partName,
                                                                              CompactorUtil.METADATA_FETCH_MODE fetchMode)
                                                                       throws org.apache.hadoop.hive.metastore.api.MetaException

Throws:

org.apache.hadoop.hive.metastore.api.MetaException

getFullPartitionName

public static String getFullPartitionName(String dbName,
                                          String tableName,
                                          String partName)

checkInterrupt

public static void checkInterrupt(String callerClassName)
                           throws InterruptedException

Throws:

InterruptedException

isMinorCompactionSupported

public static boolean isMinorCompactionSupported(HiveConf conf,
                                                 Map<String,String> tblproperties,
                                                 AcidDirectory dir)

Check for that special case when minor compaction is supported or not.

• The table is Insert-only OR
• Query based compaction is not enabled OR
• The table has only acid data in it.

Parameters:

tblproperties - The properties of the table to check

dir - The AcidDirectory instance pointing to the table's folder on the filesystem.

Returns:

Returns true if minor compaction is supported based on the given parameters, false otherwise.

createLockRequest

public static org.apache.hadoop.hive.metastore.api.LockRequest createLockRequest(HiveConf conf,
                                                                                 CompactionInfo ci,
                                                                                 long txnId,
                                                                                 org.apache.hadoop.hive.metastore.api.LockType lockType,
                                                                                 org.apache.hadoop.hive.metastore.api.DataOperationType opType)