org.apache.hudi.common.table.log.block

Class HoodieAvroDataBlock

java.lang.Object

org.apache.hudi.common.table.log.block.HoodieLogBlock

org.apache.hudi.common.table.log.block.HoodieDataBlock

org.apache.hudi.common.table.log.block.HoodieAvroDataBlock

public class HoodieAvroDataBlock
extends HoodieDataBlock

HoodieAvroDataBlock contains a list of records serialized using Avro. It is used with the Parquet base file format.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.common.table.log.block.HoodieLogBlock

HoodieLogBlock.FooterMetadataType, HoodieLogBlock.HeaderMetadataType, HoodieLogBlock.HoodieLogBlockContentLocation, HoodieLogBlock.HoodieLogBlockType

Field Summary

Fields inherited from class org.apache.hudi.common.table.log.block.HoodieDataBlock

internalSchema, readerSchema

Fields inherited from class org.apache.hudi.common.table.log.block.HoodieLogBlock

readBlockLazily, version

Constructor Summary

Constructors

Constructor and Description
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField)
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField, InternalSchema internalSchema)
HoodieAvroDataBlock(List<org.apache.avro.generic.IndexedRecord> records, Map<HoodieLogBlock.HeaderMetadataType,String> header, String keyField)
HoodieAvroDataBlock(List<org.apache.avro.generic.IndexedRecord> records, org.apache.avro.Schema schema) Deprecated.

Method Summary

Modifier and Type	Method and Description
protected ClosableIterator<org.apache.avro.generic.IndexedRecord>	deserializeRecords(byte[] content)
static HoodieAvroDataBlock	getBlock(byte[] content, org.apache.avro.Schema readerSchema)
static HoodieAvroDataBlock	getBlock(byte[] content, org.apache.avro.Schema readerSchema, InternalSchema internalSchema) Deprecated.
HoodieLogBlock.HoodieLogBlockType	getBlockType()
byte[]	getBytes(org.apache.avro.Schema schema) Deprecated.
protected byte[]	serializeRecords(List<org.apache.avro.generic.IndexedRecord> records)

Methods inherited from class org.apache.hudi.common.table.log.block.HoodieDataBlock

getContentBytes, getKeyField, getRecordIterator, getRecordIterator, getRecordKey, getSchema, getWriterSchema, lookupRecords, readRecordsFromBlockPayload

Methods inherited from class org.apache.hudi.common.table.log.block.HoodieLogBlock

deflate, getBlockContentLocation, getContent, getLogBlockFooter, getLogBlockHeader, getLogBlockLength, getLogMetadata, getLogMetadataBytes, getMagic, inflate, tryReadContent

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieAvroDataBlock

public HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream,
                           Option<byte[]> content,
                           boolean readBlockLazily,
                           HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation,
                           Option<org.apache.avro.Schema> readerSchema,
                           Map<HoodieLogBlock.HeaderMetadataType,String> header,
                           Map<HoodieLogBlock.HeaderMetadataType,String> footer,
                           String keyField,
                           InternalSchema internalSchema)

HoodieAvroDataBlock

public HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream,
                           Option<byte[]> content,
                           boolean readBlockLazily,
                           HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation,
                           Option<org.apache.avro.Schema> readerSchema,
                           Map<HoodieLogBlock.HeaderMetadataType,String> header,
                           Map<HoodieLogBlock.HeaderMetadataType,String> footer,
                           String keyField)

HoodieAvroDataBlock

public HoodieAvroDataBlock(@Nonnull
                           List<org.apache.avro.generic.IndexedRecord> records,
                           @Nonnull
                           Map<HoodieLogBlock.HeaderMetadataType,String> header,
                           @Nonnull
                           String keyField)

HoodieAvroDataBlock

@Deprecated
public HoodieAvroDataBlock(List<org.apache.avro.generic.IndexedRecord> records,
                                       org.apache.avro.Schema schema)

Deprecated.

This constructor is retained to provide backwards compatibility to HoodieArchivedLogs which were written using HoodieLogFormat V1.

Method Detail

getBlockType

public HoodieLogBlock.HoodieLogBlockType getBlockType()

Specified by:

getBlockType in class HoodieDataBlock

serializeRecords

protected byte[] serializeRecords(List<org.apache.avro.generic.IndexedRecord> records)
                           throws IOException

Specified by:

serializeRecords in class HoodieDataBlock

Throws:

IOException

deserializeRecords

protected ClosableIterator<org.apache.avro.generic.IndexedRecord> deserializeRecords(byte[] content)
                                                                              throws IOException

Specified by:

deserializeRecords in class HoodieDataBlock

Throws:

IOException

getBlock

public static HoodieAvroDataBlock getBlock(byte[] content,
                                           org.apache.avro.Schema readerSchema)
                                    throws IOException

Throws:

IOException

getBlock

@Deprecated
public static HoodieAvroDataBlock getBlock(byte[] content,
                                                       org.apache.avro.Schema readerSchema,
                                                       InternalSchema internalSchema)
                                                throws IOException

Deprecated.

This method is retained to provide backwards compatibility to HoodieArchivedLogs which were written using HoodieLogFormat V1.

Throws:

IOException

getBytes

@Deprecated
public byte[] getBytes(org.apache.avro.Schema schema)
                            throws IOException

Deprecated.

Throws:

IOException