org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast

Class VectorMapJoinFastTableContainer

java.lang.Object

org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer

All Implemented Interfaces:

MemoryEstimate, MapJoinTableContainer, VectorMapJoinHashTable, VectorMapJoinTableContainer

public class VectorMapJoinFastTableContainer
extends Object
implements VectorMapJoinTableContainer, VectorMapJoinHashTable

HashTableLoader for Tez constructs the hashtable from records read from a broadcast edge.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainer

MapJoinTableContainer.NonMatchedSmallTableIterator, MapJoinTableContainer.ReusableGetAdaptor

Constructor Summary

Constructors

Constructor and Description
VectorMapJoinFastTableContainer(MapJoinDesc desc, org.apache.hadoop.conf.Configuration hconf, long estimatedKeys, int numHTs)

Method Summary

Modifier and Type	Method and Description
void	clear() Clears the contents of the table.
boolean	containsLongKey(long currentKey)
MapJoinTableContainer.ReusableGetAdaptor	createGetter(MapJoinKey keyTypeFromLoader) Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.
MatchTracker	createMatchTracker()
VectorMapJoinNonMatchedIterator	createNonMatchedIterator(MatchTracker matchTracker)
MapJoinTableContainer.NonMatchedSmallTableIterator	createNonMatchedSmallTableIterator(MatchTracker matchTracker) Creates an iterator for going through the hash table and returns the key and value rows for any non-matched keys.
void	dumpMetrics()
MapJoinKey	getAnyKey()
long	getEstimatedMemorySize()
long	getHashCode(org.apache.hadoop.io.BytesWritable currentKey)
String	getKey() Return the assigned key.
boolean	hasSpill() Checks if the container has spilled any data onto disk.
void	putRow(long hashCode, org.apache.hadoop.io.BytesWritable currentKey, org.apache.hadoop.io.BytesWritable currentValue)
MapJoinKey	putRow(org.apache.hadoop.io.Writable currentKey, org.apache.hadoop.io.Writable currentValue) Adds row from input to the table.
void	seal() Indicates to the container that the puts have ended; table is now r/o.
void	setKey(String key) Assign a key to the container, which can be used to cache it.
void	setSerde(MapJoinObjectSerDeContext keyCtx, MapJoinObjectSerDeContext valCtx)
int	size() Return the size of the hash table.
int	spillPartitionId()
VectorMapJoinHashTable	vectorMapJoinHashTable()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorMapJoinFastTableContainer

public VectorMapJoinFastTableContainer(MapJoinDesc desc,
                                       org.apache.hadoop.conf.Configuration hconf,
                                       long estimatedKeys,
                                       int numHTs)
                                throws SerDeException

Throws:

SerDeException

Method Detail

vectorMapJoinHashTable

public VectorMapJoinHashTable vectorMapJoinHashTable()

Specified by:

vectorMapJoinHashTable in interface VectorMapJoinTableContainer

setKey

public void setKey(String key)

Description copied from interface: MapJoinTableContainer

Assign a key to the container, which can be used to cache it.

Specified by:

setKey in interface MapJoinTableContainer

getKey

public String getKey()

Description copied from interface: MapJoinTableContainer

Return the assigned key.

Specified by:

getKey in interface MapJoinTableContainer

getHashCode

public long getHashCode(org.apache.hadoop.io.BytesWritable currentKey)
                 throws HiveException,
                        IOException

Throws:

HiveException

IOException

putRow

public MapJoinKey putRow(org.apache.hadoop.io.Writable currentKey,
                         org.apache.hadoop.io.Writable currentValue)
                  throws SerDeException,
                         HiveException,
                         IOException

Description copied from interface: MapJoinTableContainer

Adds row from input to the table.

Specified by:

putRow in interface MapJoinTableContainer

Throws:

SerDeException

HiveException

IOException

putRow

public void putRow(long hashCode,
                   org.apache.hadoop.io.BytesWritable currentKey,
                   org.apache.hadoop.io.BytesWritable currentValue)
            throws SerDeException,
                   HiveException,
                   IOException

Specified by:

putRow in interface VectorMapJoinHashTable

Parameters:

hashCode - current HashCode to avoid re-computation

currentKey - The current Key in bytes

currentValue - The current Value in bytes

Throws:

SerDeException

HiveException

IOException

seal

public void seal()

Description copied from interface: MapJoinTableContainer

Indicates to the container that the puts have ended; table is now r/o.

Specified by:

seal in interface MapJoinTableContainer

createGetter

public MapJoinTableContainer.ReusableGetAdaptor createGetter(MapJoinKey keyTypeFromLoader)

Description copied from interface: MapJoinTableContainer

Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.

Specified by:

createGetter in interface MapJoinTableContainer

Parameters:

keyTypeFromLoader - Last key from hash table loader, to determine key type used when loading hashtable (if it can vary).

createNonMatchedSmallTableIterator

public MapJoinTableContainer.NonMatchedSmallTableIterator createNonMatchedSmallTableIterator(MatchTracker matchTracker)

Description copied from interface: MapJoinTableContainer

Creates an iterator for going through the hash table and returns the key and value rows for any non-matched keys. Supports FULL OUTER MapJoin.

Specified by:

createNonMatchedSmallTableIterator in interface MapJoinTableContainer

clear

public void clear()

Description copied from interface: MapJoinTableContainer

Clears the contents of the table.

Specified by:

clear in interface MapJoinTableContainer

getAnyKey

public MapJoinKey getAnyKey()

Specified by:

getAnyKey in interface MapJoinTableContainer

dumpMetrics

public void dumpMetrics()

Specified by:

dumpMetrics in interface MapJoinTableContainer

hasSpill

public boolean hasSpill()

Description copied from interface: MapJoinTableContainer

Checks if the container has spilled any data onto disk. This is only applicable for HybridHashTableContainer.

Specified by:

hasSpill in interface MapJoinTableContainer

containsLongKey

public boolean containsLongKey(long currentKey)

Specified by:

containsLongKey in interface VectorMapJoinHashTable

Parameters:

currentKey - The key to check for existence.

Returns:

true If HashTable contains the given key.

size

public int size()

Description copied from interface: MapJoinTableContainer

Return the size of the hash table.

Specified by:

size in interface MapJoinTableContainer

Specified by:

size in interface VectorMapJoinHashTable

createMatchTracker

public MatchTracker createMatchTracker()

Specified by:

createMatchTracker in interface VectorMapJoinHashTable

createNonMatchedIterator

public VectorMapJoinNonMatchedIterator createNonMatchedIterator(MatchTracker matchTracker)

Specified by:

createNonMatchedIterator in interface VectorMapJoinHashTable

spillPartitionId

public int spillPartitionId()

Specified by:

spillPartitionId in interface VectorMapJoinHashTable

getEstimatedMemorySize

public long getEstimatedMemorySize()

Specified by:

getEstimatedMemorySize in interface MemoryEstimate

setSerde

public void setSerde(MapJoinObjectSerDeContext keyCtx,
                     MapJoinObjectSerDeContext valCtx)
              throws SerDeException

Specified by:

setSerde in interface MapJoinTableContainer

Throws:

SerDeException