org.apache.hadoop.hive.llap

Class LlapHiveUtils

java.lang.Object

org.apache.hadoop.hive.llap.LlapHiveUtils

public final class LlapHiveUtils
extends Object

Covers utility functions that are used by LLAP code and depend on Hive constructs e.g. ql code.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Method Summary

Modifier and Type	Method and Description
static Object	createFileIdUsingFS(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration daemonConf) Determines the fileID for the given path using the FileSystem type provided while considering daemon configuration.
static MapWork	findMapWork(org.apache.hadoop.mapred.JobConf job) Returns MapWork based what is serialized in the JobConf instance provided.
static CacheTag	getDbAndTableNameForMetrics(org.apache.hadoop.fs.Path path, boolean includeParts, PartitionDesc part)
static int	getSchemaHash(PartitionDesc part)
static boolean	isLlapMode(org.apache.hadoop.conf.Configuration conf)
static PartitionDesc	partitionDescForPath(org.apache.hadoop.fs.Path path, Map<org.apache.hadoop.fs.Path,PartitionDesc> partitionDescMap) Takes a Path and looks up the PartitionDesc instance associated with it in a map of Path->PartitionDesc entries.
static void	throwIfCacheOnlyRead(boolean isCacheOnlyRead)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Method Detail

partitionDescForPath

public static PartitionDesc partitionDescForPath(org.apache.hadoop.fs.Path path,
                                                 Map<org.apache.hadoop.fs.Path,PartitionDesc> partitionDescMap)

Takes a Path and looks up the PartitionDesc instance associated with it in a map of Path->PartitionDesc entries. If it is not found (e.g. Path denotes a partition path, but map contains table level instances only) we will try to do the same with the parent of this path, traversing up until there's a match, if any.

Parameters:

path - the absolute path used for the look up

partitionDescMap - the map

Returns:

the PartitionDesc instance if found, null if not found

getDbAndTableNameForMetrics

public static CacheTag getDbAndTableNameForMetrics(org.apache.hadoop.fs.Path path,
                                                   boolean includeParts,
                                                   PartitionDesc part)

getSchemaHash

public static int getSchemaHash(PartitionDesc part)

findMapWork

public static MapWork findMapWork(org.apache.hadoop.mapred.JobConf job)

Returns MapWork based what is serialized in the JobConf instance provided.

Parameters:

job -

Returns:

the MapWork instance. Might be null if missing.

throwIfCacheOnlyRead

public static void throwIfCacheOnlyRead(boolean isCacheOnlyRead)
                                 throws IOException

Throws:

IOException

isLlapMode

public static boolean isLlapMode(org.apache.hadoop.conf.Configuration conf)

createFileIdUsingFS

public static Object createFileIdUsingFS(org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.fs.Path path,
                                         org.apache.hadoop.conf.Configuration daemonConf)
                                  throws IOException

Determines the fileID for the given path using the FileSystem type provided while considering daemon configuration. Invokes HdfsUtils.getFileId(), the resulting file ID can be of types Long (inode) or SyntheticFileId depending on the FS type and the actual daemon configuration. Can be costly on cloud file systems.

Parameters:

fs - FileSystem type

path - Path associated to this file

daemonConf - Llap daemon configuration

Returns:

the generated fileID, can be null in special cases (e.g. conf disallows synthetic ID on a non-HDFS FS)

Throws:

IOException