Utilities (Hive Query Language 4.0.0 API)

java.lang.Object
- org.apache.hadoop.hive.ql.exec.Utilities

public final class Utilities
extends Object

Utilities.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`Utilities.CollectionPersistenceDelegate`
`static class`	`Utilities.MissingBucketsContext`
`static class`	`Utilities.PartitionDetails`
`static class`	`Utilities.ReduceField` ReduceField: KEY: record key VALUE: record value
`static class`	`Utilities.SQLCommand<T>`
`static class`	`Utilities.StreamStatus` StreamStatus.
`static interface`	`Utilities.SupplierWithCheckedException<T,X extends Exception>`

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`BLOB_MANIFEST_FILE`
`static int`	`carriageReturnCode`
`static char[]`	`COMMON_WHITESPACE_CHARS`
`static String`	`COPY_KEYWORD`
`static int`	`ctrlaCode`
`static TableDesc`	`defaultTd`
`protected static String`	`DEPRECATED_MAPRED_DFSCLIENT_PARALLELISM_MAX` Deprecated.
`static String`	`ENSURE_OPERATORS_EXECUTED`
`static org.slf4j.Logger`	`FILE_OP_LOGGER` A logger mostly used to trace-log the details of Hive table file operations.
`static String`	`HADOOP_LOCAL_FS` The object in the reducer are composed of these top level fields.
`static String`	`HADOOP_LOCAL_FS_SCHEME`
`static String`	`HAS_MAP_WORK`
`static String`	`HAS_REDUCE_WORK`
`static String`	`HIVE_ADDED_JARS`
`static String`	`INDENT`
`static String`	`INPUT_NAME`
`static com.fasterxml.jackson.databind.ObjectMapper`	`JSON_MAPPER` Mapper to use to serialize/deserialize JSON objects ().
`static org.slf4j.Logger`	`LOGGER`
`static String`	`MAP_PLAN_NAME`
`static String`	`MAPNAME`
`static String`	`MAPRED_MAPPER_CLASS`
`static String`	`MAPRED_REDUCER_CLASS`
`static String`	`MERGE_PLAN_NAME`
`static int`	`newLineCode`
`static String`	`NSTR`
`static String`	`nullStringOutput`
`static String`	`nullStringStorage`
`static String`	`REDUCE_PLAN_NAME`
`static List<String>`	`reduceFieldNameList`
`static String`	`REDUCENAME`
`static String`	`SNAPSHOT_REF`
`static char`	`sqlEscapeChar`
`static String`	`suffix`
`static int`	`tabCode`
`static String`	`USE_VECTORIZED_INPUT_FILE_FORMAT`
`static String`	`VECTOR_MODE`

Method Summary

All Methods Static Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`static String`	`abbreviate(String str, int max)` convert "From src insert blah blah" to "From src insert ...
`static void`	`addDependencyJars(org.apache.hadoop.conf.Configuration conf, Class<?>... classes)` Sets up the job so that all necessary jars ar passed that contain classes from the given argument of this method.
`static void`	`addSchemaEvolutionToTableScanOperator(StructObjectInspector structOI, TableScanOperator tableScanOp)`
`static void`	`addSchemaEvolutionToTableScanOperator(Table table, TableScanOperator tableScanOp)`
`static void`	`addTableSchemaToConf(org.apache.hadoop.conf.Configuration conf, TableScanOperator tableScanOp)`
`static boolean`	`arePathsEqualOrWithin(org.apache.hadoop.fs.Path p1, org.apache.hadoop.fs.Path p2)`
`static void`	`cacheBaseWork(org.apache.hadoop.conf.Configuration conf, String name, BaseWork work, org.apache.hadoop.fs.Path hiveScratchDir)`
`static void`	`cacheMapWork(org.apache.hadoop.conf.Configuration conf, MapWork work, org.apache.hadoop.fs.Path hiveScratchDir)`
`static void`	`clearWork(org.apache.hadoop.conf.Configuration conf)`
`static void`	`clearWorkMap(org.apache.hadoop.conf.Configuration conf)`
`static void`	`clearWorkMapForConf(org.apache.hadoop.conf.Configuration conf)`
`static Connection`	`connectWithRetry(String connectionString, long waitWindow, int maxRetries)` Retry connecting to a database with random backoff (same as the one implemented in HDFS-767).
`static StandardStructObjectInspector`	`constructVectorizedReduceRowOI(StructObjectInspector keyInspector, StructObjectInspector valueInspector)` Create row key and value object inspectors for reduce vectorization.
`static boolean`	`contentsEqual(InputStream is1, InputStream is2, boolean ignoreWhitespace)`
`static void`	`copyJobSecretToTableProperties(TableDesc tbl)` Copy job credentials to table properties
`static void`	`copyTableJobPropertiesToConf(TableDesc tbl, org.apache.hadoop.mapred.JobConf job)` Copies the storage handler properties configured for a table descriptor to a runtime job configuration.
`static void`	`copyTablePropertiesToConf(TableDesc tbl, org.apache.hadoop.mapred.JobConf job)` Copies the storage handler properties configured for a table descriptor to a runtime job configuration.
`static OutputStream`	`createCompressedStream(org.apache.hadoop.mapred.JobConf jc, OutputStream out, boolean isCompressed)` Convert an output stream to a compressed output stream based on codecs codecs in the Job Configuration.
`static boolean`	`createDirsWithPermission(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path mkdirPath, org.apache.hadoop.fs.permission.FsPermission fsPermission, boolean recursive)`
`static RCFile.Writer`	`createRCFileWriter(org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file, boolean isCompressed, org.apache.hadoop.util.Progressable progressable)` Create a RCFile output stream based on job configuration Uses user supplied compression flag (rather than obtaining it from the Job Configuration).
`static org.apache.hadoop.io.SequenceFile.Writer`	`createSequenceWriter(org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file, Class<?> keyClass, Class<?> valClass, boolean isCompressed, org.apache.hadoop.util.Progressable progressable)` Create a sequencefile output stream based on job configuration Uses user supplied compression flag (rather than obtaining it from the Job Configuration).
`static void`	`createTmpDirs(org.apache.hadoop.conf.Configuration conf, MapWork mWork)` Hive uses tmp directories to capture the output of each FileSinkOperator.
`static void`	`createTmpDirs(org.apache.hadoop.conf.Configuration conf, ReduceWork rWork)` Hive uses tmp directories to capture the output of each FileSinkOperator.
`static List<String>`	`decodeColumnNames(String colNamesStr)`
`static String`	`encodeColumnNames(List<String> colNames)`
`static void`	`ensurePathIsWritable(org.apache.hadoop.fs.Path rootHDFSDirPath, HiveConf conf)` Checks if path passed in exists and has writable permissions.
`static String`	`escapeSqlLike(String key)` Escape the '_', '%', as well as the escape characters inside the string key.
`static int`	`estimateNumberOfReducers(HiveConf conf, org.apache.hadoop.fs.ContentSummary inputSummary, MapWork work, boolean finalMapRed)` Estimate the number of reducers needed for this job, based on job input, and configuration parameters.
`static int`	`estimateReducers(long totalInputFileSize, long bytesPerReducer, int maxReducers, boolean powersOfTwo)`
`static <T> T`	`executeWithRetry(Utilities.SQLCommand<T> cmd, PreparedStatement stmt, long baseWindow, int maxRetries)` Retry SQL execution with random backoff (same as the one implemented in HDFS-767).
`static String`	`formatBinaryString(byte[] array, int start, int length)`
`static String`	`formatMsecToStr(long msec)` Format number of milliseconds to strings
`static String`	`generateFileName(Byte tag, String bigBucketFileName)`
`static String`	`generatePath(org.apache.hadoop.fs.Path baseURI, String filename)`
`static org.apache.hadoop.fs.Path`	`generatePath(org.apache.hadoop.fs.Path basePath, String dumpFilePrefix, Byte tag, String bigBucketFileName)`
`static String`	`generateTarFileName(String name)`
`static org.apache.hadoop.fs.Path`	`generateTarPath(org.apache.hadoop.fs.Path basePath, String filename)`
`static org.apache.hadoop.fs.Path`	`generateTmpPath(org.apache.hadoop.fs.Path basePath, String id)`
`static String`	`getAclStringWithHiveModification(org.apache.hadoop.conf.Configuration tezConf, String propertyName, boolean addHs2User, String user, String hs2User)`
`static String`	`getBucketFileNameFromPathSubString(String bucketName)`
`static int`	`getBucketIdFromFile(String bucketName)`
`static int`	`getBucketingVersion(String versionStr)`
`static Collection<Class<?>>`	`getClassNamesFromConfig(HiveConf hiveConf, HiveConf.ConfVars confVar)`
`static List<String>`	`getColumnNames(org.apache.hadoop.conf.Configuration conf)`
`static List<String>`	`getColumnNames(Properties props)` Note: This will not return the correct number of columns in the case of Avro serde using an external schema URL, unless these properties have been used to initialize the Avro SerDe (which updates these properties).
`static List<String>`	`getColumnNamesFromFieldSchema(List<org.apache.hadoop.hive.metastore.api.FieldSchema> partCols)`
`static List<String>`	`getColumnNamesFromSortCols(List<org.apache.hadoop.hive.metastore.api.Order> sortCols)`
`static List<String>`	`getColumnTypes(Properties props)`
`static String[]`	`getDbTableName(String dbtable)` Deprecated. use `TableName` or `HiveTableName` instead
`static String[]`	`getDbTableName(String defaultDb, String dbtable)` Deprecated. use `TableName` or `HiveTableName` instead
`static int`	`getDefaultNotificationInterval(org.apache.hadoop.conf.Configuration hconf)` Gets the default notification interval to send progress updates to the tracker.
`static org.apache.hadoop.fs.Path[]`	`getDirectInsertDirectoryCandidates(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, int dpLevels, org.apache.hadoop.fs.PathFilter filter, long writeId, int stmtId, org.apache.hadoop.conf.Configuration conf, Boolean isBaseDir, AcidUtils.Operation acidOperation)`
`static int`	`getDPColOffset(FileSinkDesc conf)`
`static List<String>`	`getFieldSchemaString(List<org.apache.hadoop.hive.metastore.api.FieldSchema> fl)`
`static String`	`getFileExtension(org.apache.hadoop.mapred.JobConf jc, boolean isCompressed, HiveOutputFormat<?,?> hiveOutputFormat)` Based on compression option, output format, and configured output codec - get extension for output file.
`static int`	`getFooterCount(TableDesc table, org.apache.hadoop.mapred.JobConf job)` Get footer line count for a table.
`static Map<org.apache.hadoop.fs.Path,Utilities.PartitionDetails>`	`getFullDPSpecs(org.apache.hadoop.conf.Configuration conf, DynamicPartitionCtx dpCtx, Map<String,List<org.apache.hadoop.fs.Path>> dynamicPartitionSpecs)` Construct a list of full partition spec from Dynamic Partition Context and the directory names corresponding to these dynamic partitions.
`static String`	`getHdfsResourceFiles(org.apache.hadoop.conf.Configuration conf, SessionState.ResourceType type)`
`static int`	`getHeaderCount(TableDesc table)` Get header line count for a table.
`static double`	`getHighestSamplePercentage(MapWork work)` Returns the highest sample percentage of any alias in the given MapWork
`static List<org.apache.hadoop.fs.Path>`	`getInputPaths(org.apache.hadoop.mapred.JobConf job, MapWork work, org.apache.hadoop.fs.Path hiveScratchDir, Context ctx, boolean skipDummy)` Computes a list of all input paths needed to compute the given MapWork.
`static List<org.apache.hadoop.fs.Path>`	`getInputPathsTez(org.apache.hadoop.mapred.JobConf job, MapWork work)` On Tez we're not creating dummy files when getting/setting input paths.
`static org.apache.hadoop.fs.ContentSummary`	`getInputSummary(Context ctx, MapWork work, org.apache.hadoop.fs.PathFilter filter)` Calculate the total size of input files.
`static List<String>`	`getInternalColumnNamesFromSignature(List<ColumnInfo> colInfos)`
`static boolean`	`getIsVectorized(org.apache.hadoop.conf.Configuration conf)` Returns true if a plan is both configured for vectorized execution and the node is vectorized.
`static boolean`	`getIsVectorized(org.apache.hadoop.conf.Configuration conf, MapWork mapWork)`
`static String`	`getLocalResourceFiles(org.apache.hadoop.conf.Configuration conf, SessionState.ResourceType type)`
`static MapredWork`	`getMapRedWork(org.apache.hadoop.conf.Configuration conf)`
`static MapWork`	`getMapWork(org.apache.hadoop.conf.Configuration conf)`
`static BaseWork`	`getMergeWork(org.apache.hadoop.conf.Configuration jconf)`
`static BaseWork`	`getMergeWork(org.apache.hadoop.conf.Configuration jconf, String prefix)`
`static List<ExecDriver>`	`getMRTasks(List<Task<?>> tasks)`
`static String`	`getNameMessage(Throwable e)`
`static TableName`	`getNullableTableName(String dbTableName)` Deprecated. handle null values and use `TableName.fromString(String, String, String)`
`static TableName`	`getNullableTableName(String dbTableName, String defaultDb)` Deprecated. handle null values and use `TableName.fromString(String, String, String)`
`static int`	`getNumClusterJobs(List<Task<?>> tasks)`
`static List<String>`	`getPartitionColumnNames(org.apache.hadoop.conf.Configuration conf)` Returns a list with partition column names present in the configuration, or empty if there is no such information available.
`static PartitionDesc`	`getPartitionDesc(Partition part, TableDesc tableDesc)`
`static PartitionDesc`	`getPartitionDescFromTableDesc(TableDesc tblDesc, Partition part, boolean usePartSchemaProperties)`
`static String`	`getPasswdFromKeystore(String keystore, String key)`
`static String`	`getPasswdFromUri(String uriString)` Load password from the given uri.
`static org.apache.hadoop.fs.Path`	`getPlanPath(org.apache.hadoop.conf.Configuration conf)`
`static String`	`getQualifiedPath(HiveConf conf, org.apache.hadoop.fs.Path path)` Convert path to qualified path.
`static long`	`getRandomWaitTime(long baseWindow, int failures, Random r)` Introducing a random factor to the wait time before another retry.
`static ReduceWork`	`getReduceWork(org.apache.hadoop.conf.Configuration conf)`
`static String`	`getResourceFiles(org.apache.hadoop.conf.Configuration conf, SessionState.ResourceType t)`
`static ClassLoader`	`getSessionSpecifiedClassLoader()` get session specified class loader and get current class loader if fall
`static StatsPublisher`	`getStatsPublisher(org.apache.hadoop.mapred.JobConf jc)`
`static List<String>`	`getStatsTmpDirs(BaseWork work, org.apache.hadoop.conf.Configuration conf)`
`static TableDesc`	`getTableDesc(String cols, String colTypes)`
`static TableDesc`	`getTableDesc(Table tbl)`
`static String`	`getTableOrMVSuffix(Context context, boolean createTableOrMVUseSuffix)`
`static String`	`getTaskId(org.apache.hadoop.conf.Configuration hconf)` Gets the task id if we are running as a Hadoop job.
`static String`	`getTaskIdFromFilename(String filename)` Get the task id from the filename.
`static List<TezTask>`	`getTezTasks(List<Task<?>> tasks)`
`static long`	`getTotalInputFileSize(org.apache.hadoop.fs.ContentSummary inputSummary, MapWork work, double highestSamplePercentage)` Computes the total input file size.
`static long`	`getTotalInputNumFiles(org.apache.hadoop.fs.ContentSummary inputSummary, MapWork work, double highestSamplePercentage)` Computes the total number of input files.
`static List<org.apache.hadoop.fs.Path>`	`getValidMmDirectoriesFromTableOrPart(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, ValidWriteIdList validWriteIdList)`
`static VectorizedRowBatchCtx`	`getVectorizedRowBatchCtx(org.apache.hadoop.conf.Configuration conf)`
`static String`	`getVertexCounterName(String counter, String vertexName)` Appends vertex name to specified counter name.
`static void`	`handleDirectInsertTableFinalPath(org.apache.hadoop.fs.Path specPath, String unionSuffix, org.apache.hadoop.conf.Configuration hconf, boolean success, int dpLevels, int lbLevels, Utilities.MissingBucketsContext mbc, long writeId, int stmtId, org.apache.hadoop.mapred.Reporter reporter, boolean isMmTable, boolean isMmCtas, boolean isInsertOverwrite, boolean isDirectInsert, String staticSpec, AcidUtils.Operation acidOperation, FileSinkDesc conf)`
`static String`	`humanReadableByteCount(long bytes)`
`static boolean`	`isCopyFile(String filepath)`
`static boolean`	`isEmptyPath(org.apache.hadoop.conf.Configuration job, org.apache.hadoop.fs.Path dirPath)`
`static boolean`	`isEmptyPath(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.fs.Path dirPath, Context ctx)`
`static boolean`	`isHiveManagedFile(org.apache.hadoop.fs.Path path)`
`static boolean`	`isInputFileFormatSelfDescribing(PartitionDesc pd)`
`static boolean`	`isInputFileFormatVectorized(PartitionDesc pd)`
`static boolean`	`isPerfOrAboveLogging(HiveConf conf)` Checks if the current HiveServer2 logging operation level is >= PERFORMANCE.
`static boolean`	`isSchemaEvolutionEnabled(org.apache.hadoop.conf.Configuration conf, boolean isAcid)`
`static String`	`jarFinderGetJar(Class klass)` Returns the full path to the Jar containing the class.
`static String`	`join(String... elements)`
`static org.apache.hadoop.fs.FileStatus[]`	`listNonHiddenFileStatus(org.apache.hadoop.conf.Configuration job, org.apache.hadoop.fs.Path dirPath)`
`static org.apache.hadoop.fs.FileStatus[]`	`listStatusIfExists(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)` returns null if path is not exist
`static ArrayList`	`makeList(Object... olist)`
`static Properties`	`makeProperties(String... olist)`
`static List<String>`	`mergeUniqElems(List<String> src, List<String> dest)`
`static org.apache.hadoop.fs.Path`	`moveFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path srcFile, org.apache.hadoop.fs.Path destDir, String destFileName)` Rename src to dst, or in the case dst already exists, move files in src to dst.
`static void`	`moveSpecifiedFilesInParallel(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path destPath, Set<String> filesToMove)` move specified files to destination in parallel mode.
`static void`	`mvFileToFinalPath(org.apache.hadoop.fs.Path specPath, String unionSuffix, org.apache.hadoop.conf.Configuration hconf, boolean success, org.slf4j.Logger log, DynamicPartitionCtx dpCtx, FileSinkDesc conf, org.apache.hadoop.mapred.Reporter reporter)`
`static String`	`now()`
`static int`	`parseSplitBucket(org.apache.hadoop.mapred.InputSplit split)`
`static PreparedStatement`	`prepareWithRetry(Connection conn, String stmt, long waitWindow, int maxRetries)` Retry preparing a SQL statement with random backoff (same as the one implemented in HDFS-767).
`static Utilities.StreamStatus`	`readColumn(DataInput in, OutputStream out)`
`static String`	`realFile(String newFile, org.apache.hadoop.conf.Configuration conf)` Shamelessly cloned from GenericOptionsParser.
`static void`	`removeFromClassPath(String[] pathsToRemove)` Remove elements from the classpath, if possible.
`static List<org.apache.hadoop.fs.Path>`	`removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus[] fileStats, String unionSuffix, int dpLevels, int numBuckets, org.apache.hadoop.conf.Configuration hconf, Long writeId, int stmtId, boolean isMmTable, Set<org.apache.hadoop.fs.FileStatus> filesKept, boolean isBaseDir)`
`static void`	`removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration hconf, boolean isBaseDir)` Remove all temporary files and duplicate (double-committed) files from a given directory.
`static List<org.apache.hadoop.fs.Path>`	`removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, DynamicPartitionCtx dpCtx, FileSinkDesc conf, org.apache.hadoop.conf.Configuration hconf, boolean isBaseDir)`
`static String`	`removeValueTag(String column)`
`static void`	`rename(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)` Rename src to dst, or in the case dst already exists, move files in src to dst.
`static void`	`renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)` Rename src to dst, or in the case dst already exists, move files in src to dst.
`static void`	`renameOrMoveFilesInParallel(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)` Rename src to dst, or in the case dst already exists, move files in src to dst.
`static String`	`replaceTaskId(String taskId, int bucketNum)` Replace taskId with input bucketNum.
`static String`	`replaceTaskIdFromFilename(String filename, int bucketNum)` Replace the task id from the filename.
`static String`	`replaceTaskIdFromFilename(String filename, String fileId)`
`static void`	`restoreSessionSpecifiedClassLoader(ClassLoader prev)`
`static void`	`reworkMapRedWork(Task<?> task, boolean reworkMapredWork, HiveConf conf)` The check here is kind of not clean.
`static void`	`setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf, Operator op)`
`static void`	`setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf, Operator op, boolean excludeVCs)`
`static void`	`setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf, RowSchema rowSchema)`
`static void`	`setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf, RowSchema rowSchema, boolean excludeVCs)`
`static void`	`setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf, Operator op)`
`static void`	`setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf, Operator op, boolean excludeVCs)`
`static void`	`setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf, RowSchema rowSchema)`
`static void`	`setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf, RowSchema rowSchema, boolean excludeVCs)`
`static void`	`setInputAttributes(org.apache.hadoop.conf.Configuration conf, MapWork mWork)` Set hive input format, and input format file if necessary.
`static void`	`setInputPaths(org.apache.hadoop.mapred.JobConf job, List<org.apache.hadoop.fs.Path> pathsToAdd)` setInputPaths add all the paths in the provided list to the Job conf object as input paths for the job.
`static void`	`setMapRedWork(org.apache.hadoop.conf.Configuration conf, MapredWork w, org.apache.hadoop.fs.Path hiveScratchDir)`
`static void`	`setMapWork(org.apache.hadoop.conf.Configuration conf, MapWork work)`
`static org.apache.hadoop.fs.Path`	`setMapWork(org.apache.hadoop.conf.Configuration conf, MapWork w, org.apache.hadoop.fs.Path hiveScratchDir, boolean useCache)`
`static org.apache.hadoop.fs.Path`	`setMergeWork(org.apache.hadoop.mapred.JobConf conf, MergeJoinWork mergeJoinWork, org.apache.hadoop.fs.Path mrScratchDir, boolean useCache)`
`static void`	`setPartitionColumnNames(org.apache.hadoop.conf.Configuration conf, TableScanOperator tableScanOp)` Sets partition column names to the configuration, if there is available info in the operator.
`static void`	`setQueryTimeout(Statement stmt, int timeout)`
`static void`	`setReduceWork(org.apache.hadoop.conf.Configuration conf, ReduceWork work)`
`static org.apache.hadoop.fs.Path`	`setReduceWork(org.apache.hadoop.conf.Configuration conf, ReduceWork w, org.apache.hadoop.fs.Path hiveScratchDir, boolean useCache)`
`static boolean`	`shouldAvoidRename(FileSinkDesc conf, org.apache.hadoop.conf.Configuration hConf)`
`static double`	`showTime(long time)`
`static <K,V> boolean`	`skipHeader(org.apache.hadoop.mapred.RecordReader<K,V> currRecReader, int headerCount, K key, V value)` Skip header lines in the table file when reading the record.
`static long`	`sumOf(Map<String,Long> aliasToSize, Set<String> aliases)`
`static long`	`sumOfExcept(Map<String,Long> aliasToSize, Set<String> aliases, Set<String> excepts)`
`static org.apache.hadoop.fs.Path`	`toTaskTempPath(org.apache.hadoop.fs.Path orig)`
`static org.apache.hadoop.fs.Path`	`toTempPath(org.apache.hadoop.fs.Path orig)`
`static org.apache.hadoop.fs.Path`	`toTempPath(String orig)` Given a path, convert to a temporary path.
`static void`	`tryLoggingClassPaths(org.apache.hadoop.mapred.JobConf job, org.slf4j.Logger logger)` Logs the class paths of the job class loader and the thread context class loader to the passed logger.
`static void`	`unsetSchemaEvolution(org.apache.hadoop.conf.Configuration conf)`
`static void`	`validateColumnNames(List<String> colNames, List<String> checkCols)`
`static void`	`writeCommitManifest(List<org.apache.hadoop.fs.Path> commitPaths, org.apache.hadoop.fs.Path specPath, org.apache.hadoop.fs.FileSystem fs, String taskId, Long writeId, int stmtId, String unionSuffix, boolean isInsertOverwrite, boolean hasDynamicPartitions, Set<String> dynamicPartitionSpecs, String staticSpec, boolean isDelete)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail
- JSON_MAPPER
```
public static final com.fasterxml.jackson.databind.ObjectMapper JSON_MAPPER
```
  Mapper to use to serialize/deserialize JSON objects ().
- FILE_OP_LOGGER
```
public static final org.slf4j.Logger FILE_OP_LOGGER
```
  A logger mostly used to trace-log the details of Hive table file operations. Filtering the logs for FileOperations (with trace logs present) allows one to debug what Hive has done with various files and directories while committing writes, as well as reading.
- LOGGER
```
public static final org.slf4j.Logger LOGGER
```
- HADOOP_LOCAL_FS
```
public static final String HADOOP_LOCAL_FS
```
  The object in the reducer are composed of these top level fields.
  
  See Also:
  
  Constant Field Values
- HADOOP_LOCAL_FS_SCHEME
```
public static final String HADOOP_LOCAL_FS_SCHEME
```
  See Also:
  
  Constant Field Values
- MAP_PLAN_NAME
```
public static final String MAP_PLAN_NAME
```
  See Also:
  
  Constant Field Values
- REDUCE_PLAN_NAME
```
public static final String REDUCE_PLAN_NAME
```
  See Also:
  
  Constant Field Values
- MERGE_PLAN_NAME
```
public static final String MERGE_PLAN_NAME
```
  See Also:
  
  Constant Field Values
- INPUT_NAME
```
public static final String INPUT_NAME
```
  See Also:
  
  Constant Field Values
- HAS_MAP_WORK
```
public static final String HAS_MAP_WORK
```
  See Also:
  
  Constant Field Values
- HAS_REDUCE_WORK
```
public static final String HAS_REDUCE_WORK
```
  See Also:
  
  Constant Field Values
- MAPRED_MAPPER_CLASS
```
public static final String MAPRED_MAPPER_CLASS
```
  See Also:
  
  Constant Field Values
- MAPRED_REDUCER_CLASS
```
public static final String MAPRED_REDUCER_CLASS
```
  See Also:
  
  Constant Field Values
- HIVE_ADDED_JARS
```
public static final String HIVE_ADDED_JARS
```
  See Also:
  
  Constant Field Values
- VECTOR_MODE
```
public static final String VECTOR_MODE
```
  See Also:
  
  Constant Field Values
- USE_VECTORIZED_INPUT_FILE_FORMAT
```
public static final String USE_VECTORIZED_INPUT_FILE_FORMAT
```
  See Also:
  
  Constant Field Values
- MAPNAME
```
public static final String MAPNAME
```
  See Also:
  
  Constant Field Values
- REDUCENAME
```
public static final String REDUCENAME
```
  See Also:
  
  Constant Field Values
- ENSURE_OPERATORS_EXECUTED
```
public static final String ENSURE_OPERATORS_EXECUTED
```
  See Also:
  
  Constant Field Values
- SNAPSHOT_REF
```
public static final String SNAPSHOT_REF
```
  See Also:
  
  Constant Field Values
- DEPRECATED_MAPRED_DFSCLIENT_PARALLELISM_MAX
```
@Deprecated
protected static final String DEPRECATED_MAPRED_DFSCLIENT_PARALLELISM_MAX
```
  Deprecated.
  
  See Also:
  
  Constant Field Values
- COMMON_WHITESPACE_CHARS
```
public static final char[] COMMON_WHITESPACE_CHARS
```
- BLOB_MANIFEST_FILE
```
public static final String BLOB_MANIFEST_FILE
```
  See Also:
  
  Constant Field Values
- reduceFieldNameList
```
public static List<String> reduceFieldNameList
```
- defaultTd
```
public static TableDesc defaultTd
```
- carriageReturnCode
```
public static final int carriageReturnCode
```
  See Also:
  
  Constant Field Values
- newLineCode
```
public static final int newLineCode
```
  See Also:
  
  Constant Field Values
- tabCode
```
public static final int tabCode
```
  See Also:
  
  Constant Field Values
- ctrlaCode
```
public static final int ctrlaCode
```
  See Also:
  
  Constant Field Values
- INDENT
```
public static final String INDENT
```
  See Also:
  
  Constant Field Values
- nullStringStorage
```
public static final String nullStringStorage
```
  See Also:
  
  Constant Field Values
- nullStringOutput
```
public static final String nullStringOutput
```
  See Also:
  
  Constant Field Values
- NSTR
```
public static final String NSTR
```
  See Also:
  
  Constant Field Values
- COPY_KEYWORD
```
public static final String COPY_KEYWORD
```
  See Also:
  
  Constant Field Values
- suffix
```
public static final String suffix
```
  See Also:
  
  Constant Field Values
- sqlEscapeChar
```
public static final char sqlEscapeChar
```
  See Also:
  
  Constant Field Values

Method Detail

removeValueTag

public static String removeValueTag(String column)

clearWork

public static void clearWork(org.apache.hadoop.conf.Configuration conf)

getMapRedWork

public static MapredWork getMapRedWork(org.apache.hadoop.conf.Configuration conf)

cacheMapWork

public static void cacheMapWork(org.apache.hadoop.conf.Configuration conf,
                                MapWork work,
                                org.apache.hadoop.fs.Path hiveScratchDir)

setMapWork

public static void setMapWork(org.apache.hadoop.conf.Configuration conf,
                              MapWork work)

getMapWork

public static MapWork getMapWork(org.apache.hadoop.conf.Configuration conf)

setReduceWork

public static void setReduceWork(org.apache.hadoop.conf.Configuration conf,
                                 ReduceWork work)

getReduceWork

public static ReduceWork getReduceWork(org.apache.hadoop.conf.Configuration conf)

setMergeWork

public static org.apache.hadoop.fs.Path setMergeWork(org.apache.hadoop.mapred.JobConf conf,
                                                     MergeJoinWork mergeJoinWork,
                                                     org.apache.hadoop.fs.Path mrScratchDir,
                                                     boolean useCache)

getMergeWork

public static BaseWork getMergeWork(org.apache.hadoop.conf.Configuration jconf)

getMergeWork

public static BaseWork getMergeWork(org.apache.hadoop.conf.Configuration jconf,
                                    String prefix)

cacheBaseWork

public static void cacheBaseWork(org.apache.hadoop.conf.Configuration conf,
                                 String name,
                                 BaseWork work,
                                 org.apache.hadoop.fs.Path hiveScratchDir)

getFieldSchemaString

public static List<String> getFieldSchemaString(List<org.apache.hadoop.hive.metastore.api.FieldSchema> fl)

setMapRedWork

public static void setMapRedWork(org.apache.hadoop.conf.Configuration conf,
                                 MapredWork w,
                                 org.apache.hadoop.fs.Path hiveScratchDir)

setMapWork

public static org.apache.hadoop.fs.Path setMapWork(org.apache.hadoop.conf.Configuration conf,
                                                   MapWork w,
                                                   org.apache.hadoop.fs.Path hiveScratchDir,
                                                   boolean useCache)

setReduceWork

public static org.apache.hadoop.fs.Path setReduceWork(org.apache.hadoop.conf.Configuration conf,
                                                      ReduceWork w,
                                                      org.apache.hadoop.fs.Path hiveScratchDir,
                                                      boolean useCache)

getPlanPath

public static org.apache.hadoop.fs.Path getPlanPath(org.apache.hadoop.conf.Configuration conf)

getTaskId
```
public static String getTaskId(org.apache.hadoop.conf.Configuration hconf)
```
Gets the task id if we are running as a Hadoop job. Gets a random number otherwise.

makeProperties

public static Properties makeProperties(String... olist)

makeList

public static ArrayList makeList(Object... olist)

getTableDesc

public static TableDesc getTableDesc(Table tbl)

getTableDesc

public static TableDesc getTableDesc(String cols,
                                     String colTypes)

getPartitionDesc

public static PartitionDesc getPartitionDesc(Partition part,
                                             TableDesc tableDesc)
                                      throws HiveException

Throws:: HiveException

getPartitionDescFromTableDesc

public static PartitionDesc getPartitionDescFromTableDesc(TableDesc tblDesc,
                                                          Partition part,
                                                          boolean usePartSchemaProperties)
                                                   throws HiveException

Throws:: HiveException

contentsEqual

public static boolean contentsEqual(InputStream is1,
                                    InputStream is2,
                                    boolean ignoreWhitespace)
                             throws IOException

Throws:: IOException

abbreviate

public static String abbreviate(String str,
                                int max)

convert "From src insert blah blah" to "From src insert ... blah"

readColumn

public static Utilities.StreamStatus readColumn(DataInput in,
                                                OutputStream out)
                                         throws IOException

Throws:: IOException

createCompressedStream
```
public static OutputStream createCompressedStream(org.apache.hadoop.mapred.JobConf jc,
                                                  OutputStream out,
                                                  boolean isCompressed)
                                           throws IOException
```
Convert an output stream to a compressed output stream based on codecs codecs in the Job Configuration. Caller specifies directly whether file is compressed or not

Parameters:

jc - Job Configuration

out - Output Stream to be converted into compressed output stream

isCompressed - whether the output stream needs to be compressed or not

Returns:

compressed output stream

Throws:

IOException

getFileExtension
```
public static String getFileExtension(org.apache.hadoop.mapred.JobConf jc,
                                      boolean isCompressed,
                                      HiveOutputFormat<?,?> hiveOutputFormat)
```
Based on compression option, output format, and configured output codec - get extension for output file. Text files require an extension, whereas others, like sequence files, do not.
The property hive.output.file.extension is used to determine the extension - if set, it will override other logic for choosing an extension.

Parameters:

jc - Job Configuration

isCompressed - Whether the output file is compressed or not

hiveOutputFormat - The output format, used to detect if the format is text

Returns:

the required file extension (example: .gz)

createSequenceWriter

public static org.apache.hadoop.io.SequenceFile.Writer createSequenceWriter(org.apache.hadoop.mapred.JobConf jc,
                                                                            org.apache.hadoop.fs.FileSystem fs,
                                                                            org.apache.hadoop.fs.Path file,
                                                                            Class<?> keyClass,
                                                                            Class<?> valClass,
                                                                            boolean isCompressed,
                                                                            org.apache.hadoop.util.Progressable progressable)
                                                                     throws IOException

Create a sequencefile output stream based on job configuration Uses user supplied compression flag (rather than obtaining it from the Job Configuration).

Parameters:: jc - Job configuration; fs - File System to create file in; file - Path to be created; keyClass - Java Class for key; valClass - Java Class for value
Returns:: output stream over the created sequencefile
Throws:: IOException

createRCFileWriter

public static RCFile.Writer createRCFileWriter(org.apache.hadoop.mapred.JobConf jc,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.fs.Path file,
                                               boolean isCompressed,
                                               org.apache.hadoop.util.Progressable progressable)
                                        throws IOException

Create a RCFile output stream based on job configuration Uses user supplied compression flag (rather than obtaining it from the Job Configuration).

Parameters:: jc - Job configuration; fs - File System to create file in; file - Path to be created
Returns:: output stream over the created rcfile
Throws:: IOException

realFile

public static String realFile(String newFile,
                              org.apache.hadoop.conf.Configuration conf)
                       throws IOException

Shamelessly cloned from GenericOptionsParser.

Throws:: IOException

mergeUniqElems

public static List<String> mergeUniqElems(List<String> src,
                                          List<String> dest)

toTaskTempPath

public static org.apache.hadoop.fs.Path toTaskTempPath(org.apache.hadoop.fs.Path orig)

toTempPath

public static org.apache.hadoop.fs.Path toTempPath(org.apache.hadoop.fs.Path orig)

toTempPath

public static org.apache.hadoop.fs.Path toTempPath(String orig)

Given a path, convert to a temporary path.

rename
```
public static void rename(org.apache.hadoop.fs.FileSystem fs,
                          org.apache.hadoop.fs.Path src,
                          org.apache.hadoop.fs.Path dst)
                   throws IOException,
                          HiveException
```
Rename src to dst, or in the case dst already exists, move files in src to dst. If there is an existing file with the same name, the new file's name will be appended with "_1", "_2", etc.

Parameters:

fs - the FileSystem where src and dst are on.

src - the src directory

dst - the target directory

Throws:

IOException

HiveException

moveFile

public static org.apache.hadoop.fs.Path moveFile(org.apache.hadoop.fs.FileSystem fs,
                                                 org.apache.hadoop.fs.Path srcFile,
                                                 org.apache.hadoop.fs.Path destDir,
                                                 String destFileName)
                                          throws IOException,
                                                 HiveException

Rename src to dst, or in the case dst already exists, move files in src to dst. If there is an existing file with the same name, the new file's name will be generated based on the file name. If the file name confirms to hive managed file NNNNNN_Y(_copy_YY) then it will create NNNNN_Y_copy_XX else it will append _1, _2, ....

Parameters:: fs - the FileSystem where src and dst are on.; srcFile - the src file; destDir - the target directory; destFileName - the target filename
Returns:: The final path the file was moved to.
Throws:: IOException; HiveException

renameOrMoveFiles
```
public static void renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs,
                                     org.apache.hadoop.fs.Path src,
                                     org.apache.hadoop.fs.Path dst)
                              throws IOException,
                                     HiveException
```
Rename src to dst, or in the case dst already exists, move files in src to dst. If there is an existing file with the same name, the new file's name will be generated based on the file name. If the file name confirms to hive managed file NNNNNN_Y(_copy_YY) then it will create NNNNN_Y_copy_XX else it will append _1, _2, ....

Parameters:

fs - the FileSystem where src and dst are on.

src - the src directory

dst - the target directory

Throws:

IOException

HiveException

renameOrMoveFilesInParallel

public static void renameOrMoveFilesInParallel(org.apache.hadoop.conf.Configuration conf,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.fs.Path src,
                                               org.apache.hadoop.fs.Path dst)
                                        throws IOException,
                                               HiveException

Rename src to dst, or in the case dst already exists, move files in src to dst. If there is an existing file with the same name, the new file's name will be appended with "_1", "_2", etc. Happens in parallel mode.

Parameters:: conf -; fs - the FileSystem where src and dst are on.; src - the src directory; dst - the target directory
Throws:: IOException; HiveException

getTaskIdFromFilename
```
public static String getTaskIdFromFilename(String filename)
```
Get the task id from the filename. It is assumed that the filename is derived from the output of getTaskId

Parameters:

filename - filename to extract taskid from

replaceTaskIdFromFilename
```
public static String replaceTaskIdFromFilename(String filename,
                                               int bucketNum)
```
Replace the task id from the filename. It is assumed that the filename is derived from the output of getTaskId

Parameters:

filename - filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"

replaceTaskIdFromFilename

public static String replaceTaskIdFromFilename(String filename,
                                               String fileId)

replaceTaskId
```
public static String replaceTaskId(String taskId,
                                   int bucketNum)
```
Replace taskId with input bucketNum. For example, if taskId is 000000 and bucketNum is 1, return should be 000001; if taskId is (ds%3D1)000000 and bucketNum is 1, return should be (ds%3D1)000001. This method is different from the replaceTaskId(String, String) method. In this method, the pattern is in taskId.

Parameters:

taskId -

bucketNum -

Returns:

shouldAvoidRename

public static boolean shouldAvoidRename(FileSinkDesc conf,
                                        org.apache.hadoop.conf.Configuration hConf)

listStatusIfExists

public static org.apache.hadoop.fs.FileStatus[] listStatusIfExists(org.apache.hadoop.fs.Path path,
                                                                   org.apache.hadoop.fs.FileSystem fs)
                                                            throws IOException

returns null if path is not exist

Throws:: IOException

mvFileToFinalPath

public static void mvFileToFinalPath(org.apache.hadoop.fs.Path specPath,
                                     String unionSuffix,
                                     org.apache.hadoop.conf.Configuration hconf,
                                     boolean success,
                                     org.slf4j.Logger log,
                                     DynamicPartitionCtx dpCtx,
                                     FileSinkDesc conf,
                                     org.apache.hadoop.mapred.Reporter reporter)
                              throws IOException,
                                     HiveException

Throws:: IOException; HiveException

moveSpecifiedFilesInParallel

public static void moveSpecifiedFilesInParallel(org.apache.hadoop.conf.Configuration conf,
                                                org.apache.hadoop.fs.FileSystem fs,
                                                org.apache.hadoop.fs.Path srcPath,
                                                org.apache.hadoop.fs.Path destPath,
                                                Set<String> filesToMove)
                                         throws HiveException,
                                                IOException

move specified files to destination in parallel mode. Spins up multiple threads, schedules transfer and shuts down the pool.

Parameters:: conf -; fs -; srcPath -; destPath -; filesToMove -
Throws:: HiveException; IOException

removeTempOrDuplicateFiles

public static void removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs,
                                              org.apache.hadoop.fs.Path path,
                                              org.apache.hadoop.conf.Configuration hconf,
                                              boolean isBaseDir)
                                       throws IOException

Remove all temporary files and duplicate (double-committed) files from a given directory.

Throws:: IOException

removeTempOrDuplicateFiles

public static List<org.apache.hadoop.fs.Path> removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs,
                                                                         org.apache.hadoop.fs.Path path,
                                                                         DynamicPartitionCtx dpCtx,
                                                                         FileSinkDesc conf,
                                                                         org.apache.hadoop.conf.Configuration hconf,
                                                                         boolean isBaseDir)
                                                                  throws IOException

Throws:: IOException

removeTempOrDuplicateFiles

public static List<org.apache.hadoop.fs.Path> removeTempOrDuplicateFiles(org.apache.hadoop.fs.FileSystem fs,
                                                                         org.apache.hadoop.fs.FileStatus[] fileStats,
                                                                         String unionSuffix,
                                                                         int dpLevels,
                                                                         int numBuckets,
                                                                         org.apache.hadoop.conf.Configuration hconf,
                                                                         Long writeId,
                                                                         int stmtId,
                                                                         boolean isMmTable,
                                                                         Set<org.apache.hadoop.fs.FileStatus> filesKept,
                                                                         boolean isBaseDir)
                                                                  throws IOException

Throws:: IOException

isCopyFile

public static boolean isCopyFile(String filepath)

getBucketFileNameFromPathSubString

public static String getBucketFileNameFromPathSubString(String bucketName)

parseSplitBucket

public static int parseSplitBucket(org.apache.hadoop.mapred.InputSplit split)

getBucketIdFromFile

public static int getBucketIdFromFile(String bucketName)

getNameMessage

public static String getNameMessage(Throwable e)

getResourceFiles

public static String getResourceFiles(org.apache.hadoop.conf.Configuration conf,
                                      SessionState.ResourceType t)

getHdfsResourceFiles

public static String getHdfsResourceFiles(org.apache.hadoop.conf.Configuration conf,
                                          SessionState.ResourceType type)

getLocalResourceFiles

public static String getLocalResourceFiles(org.apache.hadoop.conf.Configuration conf,
                                           SessionState.ResourceType type)

getSessionSpecifiedClassLoader
```
public static ClassLoader getSessionSpecifiedClassLoader()
```
get session specified class loader and get current class loader if fall

Returns:

restoreSessionSpecifiedClassLoader

public static void restoreSessionSpecifiedClassLoader(ClassLoader prev)

removeFromClassPath
```
public static void removeFromClassPath(String[] pathsToRemove)
                                throws IOException
```
Remove elements from the classpath, if possible. This will only work if the current thread context class loader is an UDFClassLoader (i.e. if we have created it).

Parameters:

pathsToRemove - Array of classpath elements

Throws:

IOException

formatBinaryString

public static String formatBinaryString(byte[] array,
                                        int start,
                                        int length)

getColumnNamesFromSortCols

public static List<String> getColumnNamesFromSortCols(List<org.apache.hadoop.hive.metastore.api.Order> sortCols)

getColumnNamesFromFieldSchema

public static List<String> getColumnNamesFromFieldSchema(List<org.apache.hadoop.hive.metastore.api.FieldSchema> partCols)

getInternalColumnNamesFromSignature

public static List<String> getInternalColumnNamesFromSignature(List<ColumnInfo> colInfos)

getColumnNames
```
public static List<String> getColumnNames(Properties props)
```
Note: This will not return the correct number of columns in the case of Avro serde using an external schema URL, unless these properties have been used to initialize the Avro SerDe (which updates these properties).

Parameters:

props - TableDesc properties

Returns:

list of column names based on the table properties

getColumnNames

public static List<String> getColumnNames(org.apache.hadoop.conf.Configuration conf)

getColumnTypes

public static List<String> getColumnTypes(Properties props)

getDbTableName
```
@Deprecated
public static String[] getDbTableName(String dbtable)
                                           throws SemanticException
```
Deprecated. use TableName or HiveTableName instead

Extract db and table name from dbtable string, where db and table are separated by "." If there is no db name part, set the current sessions default db

Parameters:

dbtable -

Returns:

String array with two elements, first is db name, second is table name

Throws:

SemanticException

getDbTableName

@Deprecated
public static String[] getDbTableName(String defaultDb,
                                                  String dbtable)
                                           throws SemanticException

Deprecated. use TableName or HiveTableName instead

Extract db and table name from dbtable string.

Parameters:: defaultDb -; dbtable -
Returns:: String array with two elements, first is db name, second is table name
Throws:: SemanticException

validateColumnNames

public static void validateColumnNames(List<String> colNames,
                                       List<String> checkCols)
                                throws SemanticException

Throws:: SemanticException

getNullableTableName
```
@Deprecated
public static TableName getNullableTableName(String dbTableName)
                                                  throws SemanticException
```
Deprecated. handle null values and use TableName.fromString(String, String, String)

Accepts qualified name which is in the form of table, dbname.tablename or catalog.dbname.tablename and returns a TableName. All parts can be null.

Parameters:

dbTableName -

Returns:

a TableName

Throws:

SemanticException

getNullableTableName
```
@Deprecated
public static TableName getNullableTableName(String dbTableName,
                                                         String defaultDb)
                                                  throws SemanticException
```
Deprecated. handle null values and use TableName.fromString(String, String, String)

Accepts qualified name which is in the form of table, dbname.tablename or catalog.dbname.tablename and returns a TableName. All parts can be null.

Parameters:

dbTableName -

defaultDb -

Returns:

a TableName

Throws:

SemanticException

getDefaultNotificationInterval
```
public static int getDefaultNotificationInterval(org.apache.hadoop.conf.Configuration hconf)
```
Gets the default notification interval to send progress updates to the tracker. Useful for operators that may not output data for a while.

Parameters:

hconf -

Returns:

the interval in milliseconds

copyTableJobPropertiesToConf
```
public static void copyTableJobPropertiesToConf(TableDesc tbl,
                                                org.apache.hadoop.mapred.JobConf job)
                                         throws HiveException
```
Copies the storage handler properties configured for a table descriptor to a runtime job configuration.

Parameters:

tbl - table descriptor from which to read

job - configuration which receives configured properties

Throws:

HiveException

copyTablePropertiesToConf
```
public static void copyTablePropertiesToConf(TableDesc tbl,
                                             org.apache.hadoop.mapred.JobConf job)
                                      throws HiveException
```
Copies the storage handler properties configured for a table descriptor to a runtime job configuration. This differs from copyTablePropertiesToConf(org.apache.hadoop.hive.ql.plan.TableDesc, org.apache.hadoop.mapred.JobConf) in that it does not allow parameters already set in the job to override the values from the table. This is important for setting the config up for reading, as the job may already have values in it from another table.

Parameters:

tbl -

job -

Throws:

HiveException

copyJobSecretToTableProperties

public static void copyJobSecretToTableProperties(TableDesc tbl)
                                           throws IOException

Copy job credentials to table properties

Parameters:: tbl -
Throws:: IOException

getInputSummary

public static org.apache.hadoop.fs.ContentSummary getInputSummary(Context ctx,
                                                                  MapWork work,
                                                                  org.apache.hadoop.fs.PathFilter filter)
                                                           throws IOException

Calculate the total size of input files.

Parameters:: ctx - the hadoop job context; work - map reduce job plan; filter - filter to apply to the input paths before calculating size
Returns:: the summary of all the input paths.
Throws:: IOException

sumOf

public static long sumOf(Map<String,Long> aliasToSize,
                         Set<String> aliases)

sumOfExcept

public static long sumOfExcept(Map<String,Long> aliasToSize,
                               Set<String> aliases,
                               Set<String> excepts)

isEmptyPath

public static boolean isEmptyPath(org.apache.hadoop.mapred.JobConf job,
                                  org.apache.hadoop.fs.Path dirPath,
                                  Context ctx)
                           throws Exception

Throws:: Exception

isEmptyPath

public static boolean isEmptyPath(org.apache.hadoop.conf.Configuration job,
                                  org.apache.hadoop.fs.Path dirPath)
                           throws IOException

Throws:: IOException

listNonHiddenFileStatus

public static org.apache.hadoop.fs.FileStatus[] listNonHiddenFileStatus(org.apache.hadoop.conf.Configuration job,
                                                                        org.apache.hadoop.fs.Path dirPath)
                                                                 throws IOException

Throws:: IOException

getTezTasks

public static List<TezTask> getTezTasks(List<Task<?>> tasks)

getMRTasks

public static List<ExecDriver> getMRTasks(List<Task<?>> tasks)

getNumClusterJobs

public static int getNumClusterJobs(List<Task<?>> tasks)

getFullDPSpecs

public static Map<org.apache.hadoop.fs.Path,Utilities.PartitionDetails> getFullDPSpecs(org.apache.hadoop.conf.Configuration conf,
                                                                                       DynamicPartitionCtx dpCtx,
                                                                                       Map<String,List<org.apache.hadoop.fs.Path>> dynamicPartitionSpecs)
                                                                                throws HiveException

Construct a list of full partition spec from Dynamic Partition Context and the directory names corresponding to these dynamic partitions.

Throws:: HiveException

getStatsPublisher

public static StatsPublisher getStatsPublisher(org.apache.hadoop.mapred.JobConf jc)

join

public static String join(String... elements)

setColumnNameList

public static void setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
                                     RowSchema rowSchema)

setColumnNameList

public static void setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
                                     RowSchema rowSchema,
                                     boolean excludeVCs)

setColumnNameList

public static void setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
                                     Operator op)

setColumnNameList

public static void setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
                                     Operator op,
                                     boolean excludeVCs)

setColumnTypeList

public static void setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
                                     RowSchema rowSchema)

setColumnTypeList

public static void setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
                                     RowSchema rowSchema,
                                     boolean excludeVCs)

setColumnTypeList

public static void setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
                                     Operator op)

setColumnTypeList

public static void setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
                                     Operator op,
                                     boolean excludeVCs)

generatePath

public static org.apache.hadoop.fs.Path generatePath(org.apache.hadoop.fs.Path basePath,
                                                     String dumpFilePrefix,
                                                     Byte tag,
                                                     String bigBucketFileName)

generateFileName

public static String generateFileName(Byte tag,
                                      String bigBucketFileName)

generateTmpPath

public static org.apache.hadoop.fs.Path generateTmpPath(org.apache.hadoop.fs.Path basePath,
                                                        String id)

generateTarPath

public static org.apache.hadoop.fs.Path generateTarPath(org.apache.hadoop.fs.Path basePath,
                                                        String filename)

generateTarFileName

public static String generateTarFileName(String name)

generatePath

public static String generatePath(org.apache.hadoop.fs.Path baseURI,
                                  String filename)

now
```
public static String now()
```

showTime

public static double showTime(long time)

reworkMapRedWork
```
public static void reworkMapRedWork(Task<?> task,
                                    boolean reworkMapredWork,
                                    HiveConf conf)
                             throws SemanticException
```
The check here is kind of not clean. It first use a for loop to go through all input formats, and choose the ones that extend ReworkMapredInputFormat to a set. And finally go through the ReworkMapredInputFormat set, and call rework for each one. Technically all these can be avoided if all Hive's input formats can share a same interface. As in today's hive and Hadoop, it is not possible because a lot of Hive's input formats are in Hadoop's code. And most of Hadoop's input formats just extend InputFormat interface.

Parameters:

task -

reworkMapredWork -

conf -

Throws:

SemanticException

executeWithRetry
```
public static <T> T executeWithRetry(Utilities.SQLCommand<T> cmd,
                                     PreparedStatement stmt,
                                     long baseWindow,
                                     int maxRetries)
                              throws SQLException
```
Retry SQL execution with random backoff (same as the one implemented in HDFS-767). This function only retries when the SQL query throws a SQLTransientException (which might be able to succeed with a simple retry). It doesn't retry when the exception is a SQLRecoverableException or SQLNonTransientException. For SQLRecoverableException the caller needs to reconnect to the database and restart the whole transaction.

Parameters:

cmd - the SQL command

stmt - the prepared statement of SQL.

baseWindow - The base time window (in milliseconds) before the next retry. see getRandomWaitTime(long, int, java.util.Random) for details.

maxRetries - the maximum # of retries when getting a SQLTransientException.

Throws:

SQLException - throws SQLRecoverableException or SQLNonTransientException the first time it is caught, or SQLTransientException when the maxRetries has reached.

connectWithRetry
```
public static Connection connectWithRetry(String connectionString,
                                          long waitWindow,
                                          int maxRetries)
                                   throws SQLException
```
Retry connecting to a database with random backoff (same as the one implemented in HDFS-767). This function only retries when the SQL query throws a SQLTransientException (which might be able to succeed with a simple retry). It doesn't retry when the exception is a SQLRecoverableException or SQLNonTransientException. For SQLRecoverableException the caller needs to reconnect to the database and restart the whole transaction.

Parameters:

connectionString - the JDBC connection string.

waitWindow - The base time window (in milliseconds) before the next retry. see getRandomWaitTime(long, int, java.util.Random) for details.

maxRetries - the maximum # of retries when getting a SQLTransientException.

Throws:

SQLException - throws SQLRecoverableException or SQLNonTransientException the first time it is caught, or SQLTransientException when the maxRetries has reached.

prepareWithRetry
```
public static PreparedStatement prepareWithRetry(Connection conn,
                                                 String stmt,
                                                 long waitWindow,
                                                 int maxRetries)
                                          throws SQLException
```
Retry preparing a SQL statement with random backoff (same as the one implemented in HDFS-767). This function only retries when the SQL query throws a SQLTransientException (which might be able to succeed with a simple retry). It doesn't retry when the exception is a SQLRecoverableException or SQLNonTransientException. For SQLRecoverableException the caller needs to reconnect to the database and restart the whole transaction.

Parameters:

conn - a JDBC connection.

stmt - the SQL statement to be prepared.

waitWindow - The base time window (in milliseconds) before the next retry. see getRandomWaitTime(long, int, java.util.Random) for details.

maxRetries - the maximum # of retries when getting a SQLTransientException.

Throws:

SQLException - throws SQLRecoverableException or SQLNonTransientException the first time it is caught, or SQLTransientException when the maxRetries has reached.

setQueryTimeout

public static void setQueryTimeout(Statement stmt,
                                   int timeout)
                            throws SQLException

Throws:: SQLException

getRandomWaitTime
```
public static long getRandomWaitTime(long baseWindow,
                                     int failures,
                                     Random r)
```
Introducing a random factor to the wait time before another retry. The wait time is dependent on # of failures and a random factor. At the first time of getting an exception , the wait time is a random number between 0..baseWindow msec. If the first retry still fails, we will wait baseWindow msec grace period before the 2nd retry. Also at the second retry, the waiting window is expanded to 2*baseWindow msec alleviating the request rate from the server. Similarly the 3rd retry will wait 2*baseWindow msec. grace period before retry and the waiting window is expanded to 3*baseWindow msec and so on.

Parameters:

baseWindow - the base waiting window.

failures - number of failures so far.

r - a random generator.

Returns:

number of milliseconds for the next wait time.

escapeSqlLike
```
public static String escapeSqlLike(String key)
```
Escape the '_', '%', as well as the escape characters inside the string key.

Parameters:

key - the string that will be used for the SQL LIKE operator.

Returns:

a string with escaped '_' and '%'.

formatMsecToStr
```
public static String formatMsecToStr(long msec)
```
Format number of milliseconds to strings

Parameters:

msec - milliseconds

Returns:

a formatted string like "x days y hours z minutes a seconds b msec"

estimateNumberOfReducers

public static int estimateNumberOfReducers(HiveConf conf,
                                           org.apache.hadoop.fs.ContentSummary inputSummary,
                                           MapWork work,
                                           boolean finalMapRed)
                                    throws IOException

Estimate the number of reducers needed for this job, based on job input, and configuration parameters. The output of this method should only be used if the output of this MapRedTask is not being used to populate a bucketed table and the user has not specified the number of reducers to use.

Returns:: the number of reducers.
Throws:: IOException

estimateReducers

public static int estimateReducers(long totalInputFileSize,
                                   long bytesPerReducer,
                                   int maxReducers,
                                   boolean powersOfTwo)

getTotalInputFileSize

public static long getTotalInputFileSize(org.apache.hadoop.fs.ContentSummary inputSummary,
                                         MapWork work,
                                         double highestSamplePercentage)

Computes the total input file size. If block sampling was used it will scale this value by the highest sample percentage (as an estimate for input).

Parameters:: inputSummary -; work -; highestSamplePercentage -
Returns:: estimated total input size for job

getTotalInputNumFiles

public static long getTotalInputNumFiles(org.apache.hadoop.fs.ContentSummary inputSummary,
                                         MapWork work,
                                         double highestSamplePercentage)

Computes the total number of input files. If block sampling was used it will scale this value by the highest sample percentage (as an estimate for # input files).

Parameters:: inputSummary -; work -; highestSamplePercentage -
Returns:

getHighestSamplePercentage
```
public static double getHighestSamplePercentage(MapWork work)
```
Returns the highest sample percentage of any alias in the given MapWork

getInputPathsTez

public static List<org.apache.hadoop.fs.Path> getInputPathsTez(org.apache.hadoop.mapred.JobConf job,
                                                               MapWork work)
                                                        throws Exception

On Tez we're not creating dummy files when getting/setting input paths. We let Tez handle the situation. We're also setting the paths in the AM so we don't want to depend on scratch dir and context.

Throws:: Exception

getVertexCounterName
```
public static String getVertexCounterName(String counter,
                                          String vertexName)
```
Appends vertex name to specified counter name.

Parameters:

counter - counter to be appended with

vertexName - vertex name

Returns:

counter name with vertex name appended

getInputPaths

public static List<org.apache.hadoop.fs.Path> getInputPaths(org.apache.hadoop.mapred.JobConf job,
                                                            MapWork work,
                                                            org.apache.hadoop.fs.Path hiveScratchDir,
                                                            Context ctx,
                                                            boolean skipDummy)
                                                     throws Exception

Computes a list of all input paths needed to compute the given MapWork. All aliases are considered and a merged list of input paths is returned. If any input path points to an empty table or partition a dummy file in the scratch dir is instead created and added to the list. This is needed to avoid special casing the operator pipeline for these cases.

Parameters:: job - JobConf used to run the job; work - MapWork encapsulating the info about the task; hiveScratchDir - The tmp dir used to create dummy files if needed; ctx - Context object
Returns:: List of paths to process for the given MapWork
Throws:: Exception

setInputPaths

public static void setInputPaths(org.apache.hadoop.mapred.JobConf job,
                                 List<org.apache.hadoop.fs.Path> pathsToAdd)

setInputPaths add all the paths in the provided list to the Job conf object as input paths for the job.

Parameters:: job -; pathsToAdd -

setInputAttributes

public static void setInputAttributes(org.apache.hadoop.conf.Configuration conf,
                                      MapWork mWork)

Set hive input format, and input format file if necessary.

createTmpDirs
```
public static void createTmpDirs(org.apache.hadoop.conf.Configuration conf,
                                 MapWork mWork)
                          throws IOException
```
Hive uses tmp directories to capture the output of each FileSinkOperator. This method creates all necessary tmp directories for FileSinks in the Mapwork.

Parameters:

conf - Used to get the right FileSystem

mWork - Used to find FileSinkOperators

Throws:

IOException

createTmpDirs
```
public static void createTmpDirs(org.apache.hadoop.conf.Configuration conf,
                                 ReduceWork rWork)
                          throws IOException
```
Hive uses tmp directories to capture the output of each FileSinkOperator. This method creates all necessary tmp directories for FileSinks in the ReduceWork.

Parameters:

conf - Used to get the right FileSystem

rWork - Used to find FileSinkOperators

Throws:

IOException

createDirsWithPermission

public static boolean createDirsWithPermission(org.apache.hadoop.conf.Configuration conf,
                                               org.apache.hadoop.fs.Path mkdirPath,
                                               org.apache.hadoop.fs.permission.FsPermission fsPermission,
                                               boolean recursive)
                                        throws IOException

Throws:: IOException

getIsVectorized
```
public static boolean getIsVectorized(org.apache.hadoop.conf.Configuration conf)
```
Returns true if a plan is both configured for vectorized execution and the node is vectorized. The plan may be configured for vectorization but vectorization disallowed eg. for FetchOperator execution.

getIsVectorized

public static boolean getIsVectorized(org.apache.hadoop.conf.Configuration conf,
                                      MapWork mapWork)

getVectorizedRowBatchCtx

public static VectorizedRowBatchCtx getVectorizedRowBatchCtx(org.apache.hadoop.conf.Configuration conf)

Parameters:: conf -
Returns:: the configured VectorizedRowBatchCtx for a MapWork task.

clearWorkMapForConf

public static void clearWorkMapForConf(org.apache.hadoop.conf.Configuration conf)

clearWorkMap

public static void clearWorkMap(org.apache.hadoop.conf.Configuration conf)

skipHeader
```
public static <K,V> boolean skipHeader(org.apache.hadoop.mapred.RecordReader<K,V> currRecReader,
                                       int headerCount,
                                       K key,
                                       V value)
                                throws IOException
```
Skip header lines in the table file when reading the record.

Parameters:

currRecReader - Record reader.

headerCount - Header line number of the table files.

key - Key of current reading record.

value - Value of current reading record.

Returns:

Return true if there are 0 or more records left in the file after skipping all headers, otherwise return false.

Throws:

IOException

getHeaderCount
```
public static int getHeaderCount(TableDesc table)
                          throws IOException
```
Get header line count for a table.

Parameters:

table - Table description for target table.

Throws:

IOException

getFooterCount

public static int getFooterCount(TableDesc table,
                                 org.apache.hadoop.mapred.JobConf job)
                          throws IOException

Get footer line count for a table.

Parameters:: table - Table description for target table.; job - Job configuration for current job.
Throws:: IOException

getQualifiedPath

public static String getQualifiedPath(HiveConf conf,
                                      org.apache.hadoop.fs.Path path)
                               throws HiveException

Convert path to qualified path.

Parameters:: conf - Hive configuration.; path - Path to convert.
Returns:: Qualified path
Throws:: HiveException

isPerfOrAboveLogging
```
public static boolean isPerfOrAboveLogging(HiveConf conf)
```
Checks if the current HiveServer2 logging operation level is >= PERFORMANCE.

Parameters:

conf - Hive configuration.

Returns:

true if current HiveServer2 logging operation level is >= PERFORMANCE. Else, false.

jarFinderGetJar
```
public static String jarFinderGetJar(Class klass)
```
Returns the full path to the Jar containing the class. It always return a JAR.

Parameters:

klass - class.

Returns:

path to the Jar containing the class.

addDependencyJars
```
public static void addDependencyJars(org.apache.hadoop.conf.Configuration conf,
                                     Class<?>... classes)
                              throws IOException
```
Sets up the job so that all necessary jars ar passed that contain classes from the given argument of this method.

Parameters:

conf - jobConf instance to setup

classes - the classes to look in jars for

Throws:

IOException

getDPColOffset

public static int getDPColOffset(FileSinkDesc conf)

getStatsTmpDirs

public static List<String> getStatsTmpDirs(BaseWork work,
                                           org.apache.hadoop.conf.Configuration conf)

isSchemaEvolutionEnabled

public static boolean isSchemaEvolutionEnabled(org.apache.hadoop.conf.Configuration conf,
                                               boolean isAcid)

isInputFileFormatSelfDescribing

public static boolean isInputFileFormatSelfDescribing(PartitionDesc pd)

isInputFileFormatVectorized

public static boolean isInputFileFormatVectorized(PartitionDesc pd)

getClassNamesFromConfig

public static Collection<Class<?>> getClassNamesFromConfig(HiveConf hiveConf,
                                                           HiveConf.ConfVars confVar)

addSchemaEvolutionToTableScanOperator

public static void addSchemaEvolutionToTableScanOperator(Table table,
                                                         TableScanOperator tableScanOp)

addSchemaEvolutionToTableScanOperator

public static void addSchemaEvolutionToTableScanOperator(StructObjectInspector structOI,
                                                         TableScanOperator tableScanOp)

unsetSchemaEvolution

public static void unsetSchemaEvolution(org.apache.hadoop.conf.Configuration conf)

addTableSchemaToConf

public static void addTableSchemaToConf(org.apache.hadoop.conf.Configuration conf,
                                        TableScanOperator tableScanOp)

setPartitionColumnNames

public static void setPartitionColumnNames(org.apache.hadoop.conf.Configuration conf,
                                           TableScanOperator tableScanOp)

Sets partition column names to the configuration, if there is available info in the operator.

getPartitionColumnNames
```
public static List<String> getPartitionColumnNames(org.apache.hadoop.conf.Configuration conf)
```
Returns a list with partition column names present in the configuration, or empty if there is no such information available.

constructVectorizedReduceRowOI

public static StandardStructObjectInspector constructVectorizedReduceRowOI(StructObjectInspector keyInspector,
                                                                           StructObjectInspector valueInspector)
                                                                    throws HiveException

Create row key and value object inspectors for reduce vectorization. The row object inspector used by ReduceWork needs to be a **standard** struct object inspector, not just any struct object inspector.

Parameters:: keyInspector -; valueInspector -
Returns:: OI
Throws:: HiveException

humanReadableByteCount

public static String humanReadableByteCount(long bytes)

getDirectInsertDirectoryCandidates

public static org.apache.hadoop.fs.Path[] getDirectInsertDirectoryCandidates(org.apache.hadoop.fs.FileSystem fs,
                                                                             org.apache.hadoop.fs.Path path,
                                                                             int dpLevels,
                                                                             org.apache.hadoop.fs.PathFilter filter,
                                                                             long writeId,
                                                                             int stmtId,
                                                                             org.apache.hadoop.conf.Configuration conf,
                                                                             Boolean isBaseDir,
                                                                             AcidUtils.Operation acidOperation)
                                                                      throws IOException

Throws:: IOException

writeCommitManifest

public static void writeCommitManifest(List<org.apache.hadoop.fs.Path> commitPaths,
                                       org.apache.hadoop.fs.Path specPath,
                                       org.apache.hadoop.fs.FileSystem fs,
                                       String taskId,
                                       Long writeId,
                                       int stmtId,
                                       String unionSuffix,
                                       boolean isInsertOverwrite,
                                       boolean hasDynamicPartitions,
                                       Set<String> dynamicPartitionSpecs,
                                       String staticSpec,
                                       boolean isDelete)
                                throws HiveException

Throws:: HiveException

handleDirectInsertTableFinalPath

public static void handleDirectInsertTableFinalPath(org.apache.hadoop.fs.Path specPath,
                                                    String unionSuffix,
                                                    org.apache.hadoop.conf.Configuration hconf,
                                                    boolean success,
                                                    int dpLevels,
                                                    int lbLevels,
                                                    Utilities.MissingBucketsContext mbc,
                                                    long writeId,
                                                    int stmtId,
                                                    org.apache.hadoop.mapred.Reporter reporter,
                                                    boolean isMmTable,
                                                    boolean isMmCtas,
                                                    boolean isInsertOverwrite,
                                                    boolean isDirectInsert,
                                                    String staticSpec,
                                                    AcidUtils.Operation acidOperation,
                                                    FileSinkDesc conf)
                                             throws IOException,
                                                    HiveException

Throws:: IOException; HiveException

getValidMmDirectoriesFromTableOrPart

public static List<org.apache.hadoop.fs.Path> getValidMmDirectoriesFromTableOrPart(org.apache.hadoop.fs.Path path,
                                                                                   org.apache.hadoop.conf.Configuration conf,
                                                                                   ValidWriteIdList validWriteIdList)
                                                                            throws IOException

Returns:: the complete list of valid MM directories under a table/partition path; null if the entire directory is valid (has no uncommitted/temporary files).
Throws:: IOException

getAclStringWithHiveModification

public static String getAclStringWithHiveModification(org.apache.hadoop.conf.Configuration tezConf,
                                                      String propertyName,
                                                      boolean addHs2User,
                                                      String user,
                                                      String hs2User)
                                               throws IOException

Throws:: IOException

isHiveManagedFile

public static boolean isHiveManagedFile(org.apache.hadoop.fs.Path path)

ensurePathIsWritable

public static void ensurePathIsWritable(org.apache.hadoop.fs.Path rootHDFSDirPath,
                                        HiveConf conf)
                                 throws IOException

Checks if path passed in exists and has writable permissions. The path will be created if it does not exist.

Parameters:: rootHDFSDirPath -; conf -
Throws:: IOException

getBucketingVersion

public static int getBucketingVersion(String versionStr)

getPasswdFromKeystore

public static String getPasswdFromKeystore(String keystore,
                                           String key)
                                    throws IOException

Throws:: IOException

getPasswdFromUri

public static String getPasswdFromUri(String uriString)
                               throws IOException,
                                      URISyntaxException,
                                      HiveException

Load password from the given uri.

Parameters:: uriString - The URI which is used to load the password.
Returns:: null if the uri is empty or null, else the password represented by the URI.
Throws:: IOException; URISyntaxException; HiveException

encodeColumnNames

public static String encodeColumnNames(List<String> colNames)
                                throws SemanticException

Throws:: SemanticException

decodeColumnNames

public static List<String> decodeColumnNames(String colNamesStr)
                                      throws SemanticException

Throws:: SemanticException

tryLoggingClassPaths
```
public static void tryLoggingClassPaths(org.apache.hadoop.mapred.JobConf job,
                                        org.slf4j.Logger logger)
```
Logs the class paths of the job class loader and the thread context class loader to the passed logger. Checks both loaders if getURLs method is available; if not, prints a message about this (instead of the class path) Note: all messages will always be logged with DEBUG log level.

arePathsEqualOrWithin

public static boolean arePathsEqualOrWithin(org.apache.hadoop.fs.Path p1,
                                            org.apache.hadoop.fs.Path p2)

getTableOrMVSuffix

public static String getTableOrMVSuffix(Context context,
                                        boolean createTableOrMVUseSuffix)

Class Utilities

Nested Class Summary

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

JSON_MAPPER

FILE_OP_LOGGER

LOGGER

HADOOP_LOCAL_FS

HADOOP_LOCAL_FS_SCHEME

MAP_PLAN_NAME

REDUCE_PLAN_NAME

MERGE_PLAN_NAME

INPUT_NAME

HAS_MAP_WORK

HAS_REDUCE_WORK

MAPRED_MAPPER_CLASS

MAPRED_REDUCER_CLASS

HIVE_ADDED_JARS

VECTOR_MODE

USE_VECTORIZED_INPUT_FILE_FORMAT

MAPNAME

REDUCENAME

ENSURE_OPERATORS_EXECUTED

SNAPSHOT_REF

DEPRECATED_MAPRED_DFSCLIENT_PARALLELISM_MAX

COMMON_WHITESPACE_CHARS

BLOB_MANIFEST_FILE

reduceFieldNameList

defaultTd

carriageReturnCode

newLineCode

tabCode

ctrlaCode

INDENT

nullStringStorage

nullStringOutput

NSTR

COPY_KEYWORD

suffix

sqlEscapeChar

Method Detail

removeValueTag

clearWork

getMapRedWork

cacheMapWork

setMapWork

getMapWork

setReduceWork

getReduceWork

setMergeWork

getMergeWork

getMergeWork

cacheBaseWork

getFieldSchemaString

setMapRedWork

setMapWork

setReduceWork

getPlanPath

getTaskId

makeProperties

makeList

getTableDesc

getTableDesc

getPartitionDesc

getPartitionDescFromTableDesc

contentsEqual

abbreviate

readColumn

createCompressedStream

getFileExtension

createSequenceWriter

createRCFileWriter

realFile

mergeUniqElems

toTaskTempPath

toTempPath

toTempPath

rename