org.apache.hadoop.hive.dynamodb.split

Class HiveDynamoDBSegmentsSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapred.FileSplit

org.apache.hadoop.hive.dynamodb.split.HiveDynamoDBSegmentsSplit

All Implemented Interfaces:

DynamoDBSplit, org.apache.hadoop.io.Writable, org.apache.hadoop.mapred.InputSplit, org.apache.hadoop.mapred.InputSplitWithLocationInfo

public class HiveDynamoDBSegmentsSplit
extends org.apache.hadoop.mapred.FileSplit
implements DynamoDBSplit

This InputSplit implementation is meant to be used with Hive, where path is non-null, and takes a form of, e.g., hdfs://10.239.164.224:9000/mnt/hive_0110/warehouse/dynamodbtable that maps to a DynamoDB table. DynamoDBSegmentsSplit. We cannot use DynamoDBSegmentsSplit with Hive because HiveInputFormat in Aws157Hive (as of version 0.11.0) requires the injected InputSplit object be a FileSplit. Otherwise, it falls back to using new Path(""), which always throws an IllegalArgumentException: Can not create a Path from an empty string.

Constructor Summary

Constructors

Constructor and Description
HiveDynamoDBSegmentsSplit()
HiveDynamoDBSegmentsSplit(org.apache.hadoop.fs.Path path, long approxItemCount, int splitId, List<Integer> segments, int totalSegments, DynamoDBQueryFilter filterPushdown)

Method Summary

Modifier and Type	Method and Description
long	getApproxItemCount()
DynamoDBQueryFilter	getFilterPushdown()
long	getLength()
String[]	getLocations()
List<Integer>	getSegments()
int	getSplitId()
int	getTotalSegments()
void	readFields(DataInput in)
void	setDynamoDBFilterPushdown(DynamoDBQueryFilter filterPushdown)
void	write(DataOutput out)

Methods inherited from class org.apache.hadoop.mapred.FileSplit

getLocationInfo, getPath, getStart, toString

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HiveDynamoDBSegmentsSplit

public HiveDynamoDBSegmentsSplit()

HiveDynamoDBSegmentsSplit

public HiveDynamoDBSegmentsSplit(org.apache.hadoop.fs.Path path,
                                 long approxItemCount,
                                 int splitId,
                                 List<Integer> segments,
                                 int totalSegments,
                                 DynamoDBQueryFilter filterPushdown)

Method Detail

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class org.apache.hadoop.mapred.FileSplit

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class org.apache.hadoop.mapred.FileSplit

Throws:

IOException

getLength

public long getLength()

Specified by:

getLength in interface org.apache.hadoop.mapred.InputSplit

Overrides:

getLength in class org.apache.hadoop.mapred.FileSplit

getLocations

public String[] getLocations()
                      throws IOException

Specified by:

getLocations in interface org.apache.hadoop.mapred.InputSplit

Overrides:

getLocations in class org.apache.hadoop.mapred.FileSplit

Throws:

IOException

getApproxItemCount

public long getApproxItemCount()

Specified by:

getApproxItemCount in interface DynamoDBSplit

getSplitId

public int getSplitId()

Specified by:

getSplitId in interface DynamoDBSplit

getSegments

public List<Integer> getSegments()

Specified by:

getSegments in interface DynamoDBSplit

getTotalSegments

public int getTotalSegments()

Specified by:

getTotalSegments in interface DynamoDBSplit

getFilterPushdown

public DynamoDBQueryFilter getFilterPushdown()

Specified by:

getFilterPushdown in interface DynamoDBSplit

setDynamoDBFilterPushdown

public void setDynamoDBFilterPushdown(DynamoDBQueryFilter filterPushdown)

Specified by:

setDynamoDBFilterPushdown in interface DynamoDBSplit