org.apache.tez.runtime.library.common.writers

Class UnorderedPartitionedKVWriter

java.lang.Object

org.apache.tez.runtime.api.Writer

org.apache.tez.runtime.library.api.KeyValueWriter

org.apache.tez.runtime.library.api.KeyValuesWriter

org.apache.tez.runtime.library.common.writers.BaseUnorderedPartitionedKVWriter

org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter

public class UnorderedPartitionedKVWriter
extends BaseUnorderedPartitionedKVWriter

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.tez.common.counters.TezCounter	outputLargeRecordsCounter Represents final number of records written (spills are not counted)

Fields inherited from class org.apache.tez.runtime.library.common.writers.BaseUnorderedPartitionedKVWriter

additionalSpillBytesReadCounter, additionalSpillBytesWritternCounter, codec, conf, fileOutputBytesCounter, ifileBufferSize, ifileReadAhead, ifileReadAheadLength, keyClass, keySerializer, numAdditionalSpillsCounter, numPartitions, outputBytesWithOverheadCounter, outputContext, outputFileHandler, outputRecordBytesCounter, outputRecordsCounter, partitioner, serializationFactory, spilledRecordsCounter, valClass, valSerializer

Constructor Summary

Constructors

Constructor and Description
UnorderedPartitionedKVWriter(org.apache.tez.runtime.api.OutputContext outputContext, org.apache.hadoop.conf.Configuration conf, int numOutputs, long availableMemoryBytes)

Method Summary

Modifier and Type	Method and Description
List<org.apache.tez.runtime.api.Event>	close()
static long	getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf, long maxAvailableTaskMemory)
boolean	reportDetailedPartitionStats()
void	write(Object key, Object value) Writes a key/value pair.

Methods inherited from class org.apache.tez.runtime.library.common.writers.BaseUnorderedPartitionedKVWriter

write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

outputLargeRecordsCounter

protected final org.apache.tez.common.counters.TezCounter outputLargeRecordsCounter

Represents final number of records written (spills are not counted)

Constructor Detail

UnorderedPartitionedKVWriter

public UnorderedPartitionedKVWriter(org.apache.tez.runtime.api.OutputContext outputContext,
                                    org.apache.hadoop.conf.Configuration conf,
                                    int numOutputs,
                                    long availableMemoryBytes)
                             throws IOException

Throws:

IOException

Method Detail

write

public void write(Object key,
                  Object value)
           throws IOException

Description copied from class: KeyValueWriter

Writes a key/value pair.

Specified by:

write in class BaseUnorderedPartitionedKVWriter

Parameters:

key - the key to write

value - the value to write

Throws:

IOException - if an error occurs

getInitialMemoryRequirement

public static long getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf,
                                               long maxAvailableTaskMemory)

close

public List<org.apache.tez.runtime.api.Event> close()
                                             throws IOException,
                                                    InterruptedException

Specified by:

close in class BaseUnorderedPartitionedKVWriter

Throws:

IOException

InterruptedException

reportDetailedPartitionStats

public boolean reportDetailedPartitionStats()