org.apache.hudi.io.hadoop

Class HoodieAvroParquetWriter

java.lang.Object

org.apache.hudi.io.hadoop.HoodieBaseParquetWriter<org.apache.avro.generic.IndexedRecord>

org.apache.hudi.io.hadoop.HoodieAvroParquetWriter

All Implemented Interfaces:

Closeable, AutoCloseable, HoodieAvroFileWriter, HoodieFileWriter

@NotThreadSafe
public class HoodieAvroParquetWriter
extends HoodieBaseParquetWriter<org.apache.avro.generic.IndexedRecord>
implements HoodieAvroFileWriter

HoodieParquetWriter extends the ParquetWriter to help limit the size of underlying file. Provides a way to check if the current file can take more records with the canWrite() ATTENTION: HoodieParquetWriter is not thread safe and developer should take care of the order of write and close

Field Summary

Fields inherited from class org.apache.hudi.io.hadoop.HoodieBaseParquetWriter

BLOOM_FILTER_ENABLED, BLOOM_FILTER_EXPECTED_NDV

Constructor Summary

Constructors

Constructor and Description
HoodieAvroParquetWriter(StoragePath file, HoodieParquetConfig<HoodieAvroWriteSupport> parquetConfig, String instantTime, TaskContextSupplier taskContextSupplier, boolean populateMetaFields)

Method Summary

Modifier and Type	Method and Description
void	close()
void	writeAvro(String key, org.apache.avro.generic.IndexedRecord object)
void	writeAvroWithMetadata(HoodieKey key, org.apache.avro.generic.IndexedRecord avroRecord)

Methods inherited from class org.apache.hudi.io.hadoop.HoodieBaseParquetWriter

canWrite, getDataSize, getRecordCountForNextSizeCheck, getWrittenRecordCount, handleParquetBloomFilters, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.io.storage.HoodieAvroFileWriter

canWrite, prepRecordWithMetadata, write, writeWithMetadata

Methods inherited from interface org.apache.hudi.io.storage.HoodieFileWriter

write, writeWithMetadata

Constructor Detail

HoodieAvroParquetWriter

public HoodieAvroParquetWriter(StoragePath file,
                               HoodieParquetConfig<HoodieAvroWriteSupport> parquetConfig,
                               String instantTime,
                               TaskContextSupplier taskContextSupplier,
                               boolean populateMetaFields)
                        throws IOException

Throws:

IOException

Method Detail

writeAvroWithMetadata

public void writeAvroWithMetadata(HoodieKey key,
                                  org.apache.avro.generic.IndexedRecord avroRecord)
                           throws IOException

Specified by:

writeAvroWithMetadata in interface HoodieAvroFileWriter

Throws:

IOException

writeAvro

public void writeAvro(String key,
                      org.apache.avro.generic.IndexedRecord object)
               throws IOException

Specified by:

writeAvro in interface HoodieAvroFileWriter

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface HoodieAvroFileWriter

Specified by:

close in interface HoodieFileWriter

Overrides:

close in class HoodieBaseParquetWriter<org.apache.avro.generic.IndexedRecord>

Throws:

IOException