org.apache.parquet.hadoop

Class ParquetRecordWriter<T>

java.lang.Object

org.apache.hadoop.mapreduce.RecordWriter<Void,T>

org.apache.parquet.hadoop.ParquetRecordWriter<T>

Type Parameters:

T - the type of the materialized records

public class ParquetRecordWriter<T>
extends org.apache.hadoop.mapreduce.RecordWriter<Void,T>

Writes records to a Parquet file

See Also:

ParquetOutputFormat

Constructor Summary

Constructors

Constructor and Description
ParquetRecordWriter(ParquetFileWriter w, WriteSupport<T> writeSupport, MessageType schema, Map<String,String> extraMetaData, int blockSize, int pageSize, CodecFactory.BytesCompressor compressor, int dictionaryPageSize, boolean enableDictionary, boolean validating, ParquetProperties.WriterVersion writerVersion) Deprecated.
ParquetRecordWriter(ParquetFileWriter w, WriteSupport<T> writeSupport, MessageType schema, Map<String,String> extraMetaData, int blockSize, int pageSize, CompressionCodecFactory.BytesInputCompressor compressor, int dictionaryPageSize, boolean enableDictionary, boolean validating, ParquetProperties.WriterVersion writerVersion) Deprecated.
ParquetRecordWriter(ParquetFileWriter w, WriteSupport<T> writeSupport, MessageType schema, Map<String,String> extraMetaData, long blockSize, int pageSize, CodecFactory.BytesCompressor compressor, int dictionaryPageSize, boolean enableDictionary, boolean validating, ParquetProperties.WriterVersion writerVersion, MemoryManager memoryManager) Deprecated.
ParquetRecordWriter(ParquetFileWriter w, WriteSupport<T> writeSupport, MessageType schema, Map<String,String> extraMetaData, long blockSize, int pageSize, CompressionCodecFactory.BytesInputCompressor compressor, int dictionaryPageSize, boolean enableDictionary, boolean validating, ParquetProperties.WriterVersion writerVersion, MemoryManager memoryManager) Deprecated.

Method Summary

Modifier and Type	Method and Description
void	close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
void	write(Void key, T value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetRecordWriter

@Deprecated
public ParquetRecordWriter(ParquetFileWriter w,
                                       WriteSupport<T> writeSupport,
                                       MessageType schema,
                                       Map<String,String> extraMetaData,
                                       int blockSize,
                                       int pageSize,
                                       CodecFactory.BytesCompressor compressor,
                                       int dictionaryPageSize,
                                       boolean enableDictionary,
                                       boolean validating,
                                       ParquetProperties.WriterVersion writerVersion)

Deprecated.

ParquetRecordWriter

@Deprecated
public ParquetRecordWriter(ParquetFileWriter w,
                                       WriteSupport<T> writeSupport,
                                       MessageType schema,
                                       Map<String,String> extraMetaData,
                                       int blockSize,
                                       int pageSize,
                                       CompressionCodecFactory.BytesInputCompressor compressor,
                                       int dictionaryPageSize,
                                       boolean enableDictionary,
                                       boolean validating,
                                       ParquetProperties.WriterVersion writerVersion)

Deprecated.

Parameters:

w - the file to write to

writeSupport - the class to convert incoming records

schema - the schema of the records

extraMetaData - extra meta data to write in the footer of the file

blockSize - the size of a block in the file (this will be approximate)

pageSize - the size of a page in the file (this will be approximate)

compressor - the compressor used to compress the pages

dictionaryPageSize - the threshold for dictionary size

enableDictionary - to enable the dictionary

validating - if schema validation should be turned on

writerVersion - writer compatibility version

ParquetRecordWriter

@Deprecated
public ParquetRecordWriter(ParquetFileWriter w,
                                       WriteSupport<T> writeSupport,
                                       MessageType schema,
                                       Map<String,String> extraMetaData,
                                       long blockSize,
                                       int pageSize,
                                       CodecFactory.BytesCompressor compressor,
                                       int dictionaryPageSize,
                                       boolean enableDictionary,
                                       boolean validating,
                                       ParquetProperties.WriterVersion writerVersion,
                                       MemoryManager memoryManager)

Deprecated.

ParquetRecordWriter

@Deprecated
public ParquetRecordWriter(ParquetFileWriter w,
                                       WriteSupport<T> writeSupport,
                                       MessageType schema,
                                       Map<String,String> extraMetaData,
                                       long blockSize,
                                       int pageSize,
                                       CompressionCodecFactory.BytesInputCompressor compressor,
                                       int dictionaryPageSize,
                                       boolean enableDictionary,
                                       boolean validating,
                                       ParquetProperties.WriterVersion writerVersion,
                                       MemoryManager memoryManager)

Deprecated.

Parameters:

w - the file to write to

writeSupport - the class to convert incoming records

schema - the schema of the records

extraMetaData - extra meta data to write in the footer of the file

blockSize - the size of a block in the file (this will be approximate)

pageSize - the size of a page in the file (this will be approximate)

compressor - the compressor used to compress the pages

dictionaryPageSize - the threshold for dictionary size

enableDictionary - to enable the dictionary

validating - if schema validation should be turned on

writerVersion - writer compatibility version

memoryManager - memory manager for the write

Method Detail

close

public void close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
           throws IOException,
                  InterruptedException

Specified by:

close in class org.apache.hadoop.mapreduce.RecordWriter<Void,T>

Throws:

IOException

InterruptedException

write

public void write(Void key,
                  T value)
           throws IOException,
                  InterruptedException

Specified by:

write in class org.apache.hadoop.mapreduce.RecordWriter<Void,T>

Throws:

IOException

InterruptedException