org.apache.parquet.hadoop.example

Class ExampleOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Void,T>

org.apache.parquet.hadoop.ParquetOutputFormat<Group>

org.apache.parquet.hadoop.example.ExampleOutputFormat

public class ExampleOutputFormat
extends ParquetOutputFormat<Group>

An example output format

must be provided the schema up front

See Also:

setSchema(Job, MessageType), GroupWriteSupport.PARQUET_EXAMPLE_SCHEMA

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.parquet.hadoop.ParquetOutputFormat

ParquetOutputFormat.JobSummaryLevel

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.Counter

Field Summary

Fields inherited from class org.apache.parquet.hadoop.ParquetOutputFormat

ADAPTIVE_BLOOM_FILTER_ENABLED, BLOCK_SIZE, BLOOM_FILTER_CANDIDATES_NUMBER, BLOOM_FILTER_ENABLED, BLOOM_FILTER_EXPECTED_NDV, BLOOM_FILTER_FPP, BLOOM_FILTER_MAX_BYTES, COLUMN_INDEX_TRUNCATE_LENGTH, COMPRESSION, DICTIONARY_PAGE_SIZE, ENABLE_DICTIONARY, ENABLE_JOB_SUMMARY, ESTIMATE_PAGE_SIZE_CHECK, JOB_SUMMARY_LEVEL, MAX_PADDING_BYTES, MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK, MEMORY_POOL_RATIO, MIN_MEMORY_ALLOCATION, MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK, PAGE_ROW_COUNT_LIMIT, PAGE_SIZE, PAGE_VALUE_COUNT_THRESHOLD, PAGE_WRITE_CHECKSUM_ENABLED, STATISTICS_TRUNCATE_LENGTH, VALIDATION, WRITE_SUPPORT_CLASS, WRITER_VERSION

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART

Constructor Summary

Constructors

Constructor and Description
ExampleOutputFormat()

Method Summary

Modifier and Type	Method and Description
static MessageType	getSchema(org.apache.hadoop.mapreduce.Job job) retrieve the schema from the conf
static void	setSchema(org.apache.hadoop.mapreduce.Job job, MessageType schema) set the schema being written to the job conf

Methods inherited from class org.apache.parquet.hadoop.ParquetOutputFormat

createEncryptionProperties, getAdaptiveBloomFilterEnabled, getBlockSize, getBlockSize, getBloomFilterEnabled, getBloomFilterMaxBytes, getCompression, getCompression, getDictionaryPageSize, getDictionaryPageSize, getEnableDictionary, getEnableDictionary, getEstimatePageSizeCheck, getJobSummaryLevel, getLongBlockSize, getMaxRowCountForPageSizeCheck, getMemoryManager, getMinRowCountForPageSizeCheck, getOutputCommitter, getPageSize, getPageSize, getPageWriteChecksumEnabled, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getValidation, getValidation, getValueCountThreshold, getWriterVersion, getWriteSupport, getWriteSupportClass, isCompressionSet, isCompressionSet, setBlockSize, setColumnIndexTruncateLength, setColumnIndexTruncateLength, setCompression, setDictionaryPageSize, setEnableDictionary, setMaxPaddingSize, setMaxPaddingSize, setPageRowCountLimit, setPageRowCountLimit, setPageSize, setPageWriteChecksumEnabled, setPageWriteChecksumEnabled, setStatisticsTruncateLength, setValidation, setValidation, setWriteSupportClass, setWriteSupportClass

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExampleOutputFormat

public ExampleOutputFormat()

Method Detail

setSchema

public static void setSchema(org.apache.hadoop.mapreduce.Job job,
                             MessageType schema)

set the schema being written to the job conf

Parameters:

job - a job

schema - the schema of the data

getSchema

public static MessageType getSchema(org.apache.hadoop.mapreduce.Job job)

retrieve the schema from the conf

Parameters:

job - a job

Returns:

the schema