ParquetWriter.Builder (Apache Parquet Hadoop 1.14.1 API)

java.lang.Object
- org.apache.parquet.hadoop.ParquetWriter.Builder<T,SELF>

Type Parameters:

T - The type of objects written by the constructed ParquetWriter.

SELF - The type of this builder that is returned by builder methods

Direct Known Subclasses:

ExampleParquetWriter.Builder

Enclosing class:

ParquetWriter<T>
```
public abstract static class ParquetWriter.Builder<T,SELF extends ParquetWriter.Builder<T,SELF>>
extends Object
```
An abstract builder class for ParquetWriter instances.
Object models should extend this builder to provide writer configuration options.

Constructor Summary

Constructors
Modifier Constructor and Description

protected Builder(OutputFile path)

protected Builder(org.apache.hadoop.fs.Path path)

Constructors
Modifier	Constructor and Description
`protected`	`Builder(OutputFile path)`
`protected`	`Builder(org.apache.hadoop.fs.Path path)`

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`ParquetWriter<T>`	`build()` Build a `ParquetWriter` with the accumulated configuration.
`SELF`	`config(String property, String value)` Set a property that will be available to the read path.
`SELF`	`enableDictionaryEncoding()` Enables dictionary encoding for the constructed writer.
`SELF`	`enablePageWriteChecksum()` Enables writing page level checksums for the constructed writer.
`SELF`	`enableValidation()` Enables validation for the constructed writer.
`protected abstract WriteSupport<T>`	`getWriteSupport(org.apache.hadoop.conf.Configuration conf)` Deprecated. Use `getWriteSupport(ParquetConfiguration)` instead
`protected WriteSupport<T>`	`getWriteSupport(ParquetConfiguration conf)`
`protected abstract SELF`	`self()`
`SELF`	`withAdaptiveBloomFilterEnabled(boolean enabled)` When NDV (number of distinct values) for a specified column is not set, whether to use `AdaptiveBloomFilter` to automatically adjust the BloomFilter size according to `parquet.bloom.filter.max.bytes`
`SELF`	`withAllocator(ByteBufferAllocator allocator)` Sets the ByteBuffer allocator instance to be used for allocating memory for writing.
`SELF`	`withBloomFilterCandidateNumber(String columnPath, int number)` When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.
`SELF`	`withBloomFilterEnabled(boolean enabled)` Sets the bloom filter enabled/disabled
`SELF`	`withBloomFilterEnabled(String columnPath, boolean enabled)` Sets the bloom filter enabled/disabled for the specified column.
`SELF`	`withBloomFilterFPP(String columnPath, double fpp)`
`SELF`	`withBloomFilterNDV(String columnPath, long ndv)` Sets the NDV (number of distinct values) for the specified column.
`SELF`	`withByteStreamSplitEncoding(boolean enableByteStreamSplit)`
`SELF`	`withCodecFactory(CompressionCodecFactory codecFactory)` Set the `codec factory` used by the constructed writer.
`SELF`	`withColumnIndexTruncateLength(int length)` Sets the length to be used for truncating binary values in a binary column index.
`SELF`	`withCompressionCodec(CompressionCodecName codecName)` Set the `compression codec` used by the constructed writer.
`SELF`	`withConf(org.apache.hadoop.conf.Configuration conf)` Set the `Configuration` used by the constructed writer.
`SELF`	`withConf(ParquetConfiguration conf)` Set the `ParquetConfiguration` used by the constructed writer.
`SELF`	`withDictionaryEncoding(boolean enableDictionary)` Enable or disable dictionary encoding for the constructed writer.
`SELF`	`withDictionaryEncoding(String columnPath, boolean enableDictionary)` Enable or disable dictionary encoding of the specified column for the constructed writer.
`SELF`	`withDictionaryPageSize(int dictionaryPageSize)` Set the Parquet format dictionary page size used by the constructed writer.
`SELF`	`withEncryption(FileEncryptionProperties encryptionProperties)` Set the `file encryption properties` used by the constructed writer.
`SELF`	`withExtraMetaData(Map<String,String> extraMetaData)` Sets additional metadata entries to be included in the file footer.
`SELF`	`withMaxBloomFilterBytes(int maxBloomFilterBytes)` Set max Bloom filter bytes for related columns.
`SELF`	`withMaxPaddingSize(int maxPaddingSize)` Set the maximum amount of padding, in bytes, that will be used to align row groups with blocks in the underlying filesystem.
`SELF`	`withMaxRowCountForPageSizeCheck(int max)` Sets the maximum number of rows to write before a page size check is done.
`SELF`	`withMinRowCountForPageSizeCheck(int min)` Sets the minimum number of rows to write before a page size check is done.
`SELF`	`withPageRowCountLimit(int rowCount)` Sets the Parquet format page row count limit used by the constructed writer.
`SELF`	`withPageSize(int pageSize)` Set the Parquet format page size used by the constructed writer.
`SELF`	`withPageWriteChecksumEnabled(boolean enablePageWriteChecksum)` Enables writing page level checksums for the constructed writer.
`SELF`	`withRowGroupSize(int rowGroupSize)` Deprecated. Use `withRowGroupSize(long)` instead
`SELF`	`withRowGroupSize(long rowGroupSize)` Set the Parquet format row group size used by the constructed writer.
`SELF`	`withStatisticsTruncateLength(int length)` Sets the length which the min/max binary values in row groups are truncated to.
`SELF`	`withValidation(boolean enableValidation)` Enable or disable validation for the constructed writer.
`SELF`	`withWriteMode(ParquetFileWriter.Mode mode)` Set the `write mode` used when creating the backing file for this writer.
`SELF`	`withWriterVersion(ParquetProperties.WriterVersion version)` Set the `format version` used by the constructed writer.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - Builder
```
protected Builder(org.apache.hadoop.fs.Path path)
```
  - Builder
```
protected Builder(OutputFile path)
```
- Method Detail
  - self
```
protected abstract SELF self()
```
    Returns:
    
    this as the correct subclass of ParquetWriter.Builder.
  - getWriteSupport
```
@Deprecated
protected abstract WriteSupport<T> getWriteSupport(org.apache.hadoop.conf.Configuration conf)
```
    Deprecated. Use getWriteSupport(ParquetConfiguration) instead
    
    Parameters:
    
    conf - a configuration
    
    Returns:
    
    an appropriate WriteSupport for the object model.
  - getWriteSupport
```
protected WriteSupport<T> getWriteSupport(ParquetConfiguration conf)
```
    Parameters:
    
    conf - a configuration
    
    Returns:
    
    an appropriate WriteSupport for the object model.
  - withConf
```
public SELF withConf(org.apache.hadoop.conf.Configuration conf)
```
    Set the Configuration used by the constructed writer.
    
    Parameters:
    
    conf - a Configuration
    
    Returns:
    
    this builder for method chaining.
  - withConf
```
public SELF withConf(ParquetConfiguration conf)
```
    Set the ParquetConfiguration used by the constructed writer.
    
    Parameters:
    
    conf - a ParquetConfiguration
    
    Returns:
    
    this builder for method chaining.
  - withWriteMode
```
public SELF withWriteMode(ParquetFileWriter.Mode mode)
```
    Set the write mode used when creating the backing file for this writer.
    
    Parameters:
    
    mode - a ParquetFileWriter.Mode
    
    Returns:
    
    this builder for method chaining.
  - withCompressionCodec
```
public SELF withCompressionCodec(CompressionCodecName codecName)
```
    Set the compression codec used by the constructed writer.
    
    Parameters:
    
    codecName - a CompressionCodecName
    
    Returns:
    
    this builder for method chaining.
  - withCodecFactory
```
public SELF withCodecFactory(CompressionCodecFactory codecFactory)
```
    Set the codec factory used by the constructed writer.
    
    Parameters:
    
    codecFactory - a CompressionCodecFactory
    
    Returns:
    
    this builder for method chaining.
  - withEncryption
```
public SELF withEncryption(FileEncryptionProperties encryptionProperties)
```
    Set the file encryption properties used by the constructed writer.
    
    Parameters:
    
    encryptionProperties - a FileEncryptionProperties
    
    Returns:
    
    this builder for method chaining.
  - withRowGroupSize
```
@Deprecated
public SELF withRowGroupSize(int rowGroupSize)
```
    Deprecated. Use withRowGroupSize(long) instead
    
    Set the Parquet format row group size used by the constructed writer.
    
    Parameters:
    
    rowGroupSize - an integer size in bytes
    
    Returns:
    
    this builder for method chaining.
  - withRowGroupSize
```
public SELF withRowGroupSize(long rowGroupSize)
```
    Set the Parquet format row group size used by the constructed writer.
    
    Parameters:
    
    rowGroupSize - an integer size in bytes
    
    Returns:
    
    this builder for method chaining.
  - withPageSize
```
public SELF withPageSize(int pageSize)
```
    Set the Parquet format page size used by the constructed writer.
    
    Parameters:
    
    pageSize - an integer size in bytes
    
    Returns:
    
    this builder for method chaining.
  - withPageRowCountLimit
```
public SELF withPageRowCountLimit(int rowCount)
```
    Sets the Parquet format page row count limit used by the constructed writer.
    
    Parameters:
    
    rowCount - limit for the number of rows stored in a page
    
    Returns:
    
    this builder for method chaining
  - withDictionaryPageSize
```
public SELF withDictionaryPageSize(int dictionaryPageSize)
```
    Set the Parquet format dictionary page size used by the constructed writer.
    
    Parameters:
    
    dictionaryPageSize - an integer size in bytes
    
    Returns:
    
    this builder for method chaining.
  - withMaxPaddingSize
```
public SELF withMaxPaddingSize(int maxPaddingSize)
```
    Set the maximum amount of padding, in bytes, that will be used to align row groups with blocks in the underlying filesystem. If the underlying filesystem is not a block filesystem like HDFS, this has no effect.
    
    Parameters:
    
    maxPaddingSize - an integer size in bytes
    
    Returns:
    
    this builder for method chaining.
  - enableDictionaryEncoding
```
public SELF enableDictionaryEncoding()
```
    Enables dictionary encoding for the constructed writer.
    
    Returns:
    
    this builder for method chaining.
  - withDictionaryEncoding
```
public SELF withDictionaryEncoding(boolean enableDictionary)
```
    Enable or disable dictionary encoding for the constructed writer.
    
    Parameters:
    
    enableDictionary - whether dictionary encoding should be enabled
    
    Returns:
    
    this builder for method chaining.
  - withByteStreamSplitEncoding
```
public SELF withByteStreamSplitEncoding(boolean enableByteStreamSplit)
```
  - withDictionaryEncoding
```
public SELF withDictionaryEncoding(String columnPath,
                                   boolean enableDictionary)
```
    Enable or disable dictionary encoding of the specified column for the constructed writer.
    
    Parameters:
    
    columnPath - the path of the column (dot-string)
    
    enableDictionary - whether dictionary encoding should be enabled
    
    Returns:
    
    this builder for method chaining.
  - enableValidation
```
public SELF enableValidation()
```
    Enables validation for the constructed writer.
    
    Returns:
    
    this builder for method chaining.
  - withValidation
```
public SELF withValidation(boolean enableValidation)
```
    Enable or disable validation for the constructed writer.
    
    Parameters:
    
    enableValidation - whether validation should be enabled
    
    Returns:
    
    this builder for method chaining.
  - withWriterVersion
```
public SELF withWriterVersion(ParquetProperties.WriterVersion version)
```
    Set the format version used by the constructed writer.
    
    Parameters:
    
    version - a WriterVersion
    
    Returns:
    
    this builder for method chaining.
  - enablePageWriteChecksum
```
public SELF enablePageWriteChecksum()
```
    Enables writing page level checksums for the constructed writer.
    
    Returns:
    
    this builder for method chaining.
  - withPageWriteChecksumEnabled
```
public SELF withPageWriteChecksumEnabled(boolean enablePageWriteChecksum)
```
    Enables writing page level checksums for the constructed writer.
    
    Parameters:
    
    enablePageWriteChecksum - whether page checksums should be written out
    
    Returns:
    
    this builder for method chaining.
  - withMaxBloomFilterBytes
```
public SELF withMaxBloomFilterBytes(int maxBloomFilterBytes)
```
    Set max Bloom filter bytes for related columns.
    
    Parameters:
    
    maxBloomFilterBytes - the max bytes of a Bloom filter bitset for a column.
    
    Returns:
    
    this builder for method chaining
  - withBloomFilterNDV
```
public SELF withBloomFilterNDV(String columnPath,
                               long ndv)
```
    Sets the NDV (number of distinct values) for the specified column.
    
    Parameters:
    
    columnPath - the path of the column (dot-string)
    
    ndv - the NDV of the column
    
    Returns:
    
    this builder for method chaining.
  - withBloomFilterFPP
```
public SELF withBloomFilterFPP(String columnPath,
                               double fpp)
```
  - withAdaptiveBloomFilterEnabled
```
public SELF withAdaptiveBloomFilterEnabled(boolean enabled)
```
    When NDV (number of distinct values) for a specified column is not set, whether to use `AdaptiveBloomFilter` to automatically adjust the BloomFilter size according to `parquet.bloom.filter.max.bytes`
    
    Parameters:
    
    enabled - whether to write bloom filter for the column
  - withBloomFilterCandidateNumber
```
public SELF withBloomFilterCandidateNumber(String columnPath,
                                           int number)
```
    When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.
    
    Parameters:
    
    columnPath - the path of the column (dot-string)
    
    number - the number of candidate
  - withBloomFilterEnabled
```
public SELF withBloomFilterEnabled(boolean enabled)
```
    Sets the bloom filter enabled/disabled
    
    Parameters:
    
    enabled - whether to write bloom filters
    
    Returns:
    
    this builder for method chaining
  - withBloomFilterEnabled
```
public SELF withBloomFilterEnabled(String columnPath,
                                   boolean enabled)
```
    Sets the bloom filter enabled/disabled for the specified column. If not set for the column specifically the default enabled/disabled state will take place. See withBloomFilterEnabled(boolean).
    
    Parameters:
    
    columnPath - the path of the column (dot-string)
    
    enabled - whether to write bloom filter for the column
    
    Returns:
    
    this builder for method chaining
  - withMinRowCountForPageSizeCheck
```
public SELF withMinRowCountForPageSizeCheck(int min)
```
    Sets the minimum number of rows to write before a page size check is done.
    
    Parameters:
    
    min - writes at least `min` rows before invoking a page size check
    
    Returns:
    
    this builder for method chaining
  - withMaxRowCountForPageSizeCheck
```
public SELF withMaxRowCountForPageSizeCheck(int max)
```
    Sets the maximum number of rows to write before a page size check is done.
    
    Parameters:
    
    max - makes a page size check after `max` rows have been written
    
    Returns:
    
    this builder for method chaining
  - withColumnIndexTruncateLength
```
public SELF withColumnIndexTruncateLength(int length)
```
    Sets the length to be used for truncating binary values in a binary column index.
    
    Parameters:
    
    length - the length to truncate to
    
    Returns:
    
    this builder for method chaining
  - withStatisticsTruncateLength
```
public SELF withStatisticsTruncateLength(int length)
```
    Sets the length which the min/max binary values in row groups are truncated to.
    
    Parameters:
    
    length - the length to truncate to
    
    Returns:
    
    this builder for method chaining
  - withExtraMetaData
```
public SELF withExtraMetaData(Map<String,String> extraMetaData)
```
    Sets additional metadata entries to be included in the file footer.
    
    Parameters:
    
    extraMetaData - a Map of additional stringly-typed metadata entries
    
    Returns:
    
    this builder for method chaining
  - withAllocator
```
public SELF withAllocator(ByteBufferAllocator allocator)
```
    Sets the ByteBuffer allocator instance to be used for allocating memory for writing.
    
    Parameters:
    
    allocator - the allocator instance
    
    Returns:
    
    this builder for method chaining
  - config
```
public SELF config(String property,
                   String value)
```
    Set a property that will be available to the read path. For writers that use a Hadoop configuration, this is the recommended way to add configuration values.
    
    Parameters:
    
    property - a String property name
    
    value - a String property value
    
    Returns:
    
    this builder for method chaining.
  - build
```
public ParquetWriter<T> build()
                       throws IOException
```
    Build a ParquetWriter with the accumulated configuration.
    
    Returns:
    
    a configured ParquetWriter instance.
    
    Throws:
    
    IOException - if there is an error while creating the writer

Class ParquetWriter.Builder<T,SELF extends ParquetWriter.Builder<T,SELF>>

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

Builder

Builder

Method Detail

self

getWriteSupport

getWriteSupport

withConf

withConf

withWriteMode

withCompressionCodec

withCodecFactory

withEncryption

withRowGroupSize

withRowGroupSize

withPageSize

withPageRowCountLimit

withDictionaryPageSize

withMaxPaddingSize

enableDictionaryEncoding

withDictionaryEncoding

withByteStreamSplitEncoding

withDictionaryEncoding

enableValidation

withValidation

withWriterVersion

enablePageWriteChecksum

withPageWriteChecksumEnabled

withMaxBloomFilterBytes

withBloomFilterNDV

withBloomFilterFPP

withAdaptiveBloomFilterEnabled

withBloomFilterCandidateNumber

withBloomFilterEnabled

withBloomFilterEnabled

withMinRowCountForPageSizeCheck

withMaxRowCountForPageSizeCheck

withColumnIndexTruncateLength

withStatisticsTruncateLength

withExtraMetaData

withAllocator

config

build