com.hotels.hcommon.hive.metastore.util

Class LocationUtils

java.lang.Object

com.hotels.hcommon.hive.metastore.util.LocationUtils

public final class LocationUtils
extends Object

On hadoop Path objects:

A Path fetched from the hive metastore are assumed stored encoded e.g.:

 table.getSd().getLocation()
 partition.getSd().getLocation()
 Will have url encoded string, e.g. /folder/a_percent_%25_encoded/
 

Strings that are encoded can be turned into a Path like this: new Path(String). Calling new Path(String).toUri() on such a path will return a double encoded URI.

Calling new Path(String).toString() on such a path will return the correct path representation(single encoded).

The Hadoop DistributedFileSystem expects to have a double encoded URI as it does path.toURI().getPath() to get the unencoded path (without scheme) back. For all our purposes in Circus Train code we can always create Paths with the new Path(String) constructor. We can get the string representation back by calling path.toString().

Method Summary

Modifier and Type	Method and Description
static boolean	hasLocation(org.apache.hadoop.hive.metastore.api.Partition partition)
static boolean	hasLocation(org.apache.hadoop.hive.metastore.api.Table table)
static org.apache.hadoop.fs.Path	locationAsPath(org.apache.hadoop.hive.metastore.api.Partition partition)
static org.apache.hadoop.fs.Path	locationAsPath(org.apache.hadoop.hive.metastore.api.Table table)
static URI	locationAsUri(org.apache.hadoop.hive.metastore.api.Partition partition)
static URI	locationAsUri(org.apache.hadoop.hive.metastore.api.Table table)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

hasLocation

public static boolean hasLocation(org.apache.hadoop.hive.metastore.api.Table table)

Parameters:

table -

Returns:

true if the table has location, false otherwise

hasLocation

public static boolean hasLocation(org.apache.hadoop.hive.metastore.api.Partition partition)

Parameters:

partition -

Returns:

true if the partition has location, false otherwise

locationAsUri

public static URI locationAsUri(org.apache.hadoop.hive.metastore.api.Table table)

Parameters:

table -

Returns:

the table.getSd().getLocation() as a URI (correctly URL encoded)

locationAsUri

public static URI locationAsUri(org.apache.hadoop.hive.metastore.api.Partition partition)

Parameters:

partition -

Returns:

the partition.getSd().getLocation() as a URI (correctly URL encoded)

locationAsPath

public static org.apache.hadoop.fs.Path locationAsPath(org.apache.hadoop.hive.metastore.api.Table table)

Parameters:

table -

Returns:

the table.getSd().getLocation() as a Path (correctly URL encoded)

locationAsPath

public static org.apache.hadoop.fs.Path locationAsPath(org.apache.hadoop.hive.metastore.api.Partition partition)

Parameters:

partition -

Returns:

the partition.getSd().getLocation() as a Path (correctly URL encoded)