org.apache.hudi.io.hadoop

Class HoodieHBaseAvroHFileReader

java.lang.Object

org.apache.hudi.io.storage.HoodieAvroFileReader

org.apache.hudi.io.storage.HoodieAvroHFileReaderImplBase

org.apache.hudi.io.hadoop.HoodieHBaseAvroHFileReader

All Implemented Interfaces:

AutoCloseable, HoodieFileReader<org.apache.avro.generic.IndexedRecord>, HoodieSeekingFileReader<org.apache.avro.generic.IndexedRecord>

public class HoodieHBaseAvroHFileReader
extends HoodieAvroHFileReaderImplBase

NOTE: PLEASE READ DOCS & COMMENTS CAREFULLY BEFORE MAKING CHANGES

HoodieFileReader implementation allowing to read from HFile.

Field Summary

Fields inherited from class org.apache.hudi.io.storage.HoodieAvroHFileReaderImplBase

KEY_BLOOM_FILTER_META_BLOCK, KEY_BLOOM_FILTER_TYPE_CODE, KEY_FIELD_NAME, KEY_MAX_RECORD, KEY_MIN_RECORD, SCHEMA_KEY

Constructor Summary

Constructors

Constructor and Description
HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf, StoragePath path)
HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf, StoragePath path, HoodieStorage storage, byte[] content, Option<org.apache.avro.Schema> schemaOpt)
HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf, StoragePath path, Option<org.apache.avro.Schema> schemaOpt)
HoodieHBaseAvroHFileReader(StoragePath path, HoodieStorage storage, StorageConfiguration<?> storageConf, Option<org.apache.avro.Schema> schemaOpt, Option<byte[]> content)

Method Summary

Modifier and Type	Method and Description
void	close()
Set<String>	filterRowKeys(Set<String> candidateRowKeys) Filter keys by availability.
ClosableIterator<org.apache.avro.generic.IndexedRecord>	getIndexedRecordIterator(org.apache.avro.Schema readerSchema, org.apache.avro.Schema requestedSchema)
ClosableIterator<org.apache.avro.generic.IndexedRecord>	getIndexedRecordsByKeyPrefixIterator(List<String> sortedKeyPrefixes, org.apache.avro.Schema readerSchema)
ClosableIterator<org.apache.avro.generic.IndexedRecord>	getIndexedRecordsByKeysIterator(List<String> keys, org.apache.avro.Schema readerSchema)
ClosableIterator<String>	getRecordKeyIterator()
ClosableIterator<HoodieRecord<org.apache.avro.generic.IndexedRecord>>	getRecordsByKeyPrefixIterator(List<String> sortedKeyPrefixes, org.apache.avro.Schema schema)
ClosableIterator<HoodieRecord<org.apache.avro.generic.IndexedRecord>>	getRecordsByKeysIterator(List<String> sortedKeys, org.apache.avro.Schema schema)
org.apache.avro.Schema	getSchema()
long	getTotalRecords()
BloomFilter	readBloomFilter()
String[]	readMinMaxRecordKeys()

Methods inherited from class org.apache.hudi.io.storage.HoodieAvroHFileReaderImplBase

deserialize, deserialize, readAllRecords, readRecords, readRecords

Methods inherited from class org.apache.hudi.io.storage.HoodieAvroFileReader

getIndexedRecordIterator, getRecordIterator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.io.storage.HoodieSeekingFileReader

getRecordsByKeyPrefixIterator, getRecordsByKeysIterator

Methods inherited from interface org.apache.hudi.io.storage.HoodieFileReader

getRecordIterator, getRecordIterator, getRecordIterator

Constructor Detail

HoodieHBaseAvroHFileReader

public HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf,
                                  StoragePath path,
                                  Option<org.apache.avro.Schema> schemaOpt)
                           throws IOException

Throws:

IOException

HoodieHBaseAvroHFileReader

public HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf,
                                  StoragePath path,
                                  HoodieStorage storage,
                                  byte[] content,
                                  Option<org.apache.avro.Schema> schemaOpt)
                           throws IOException

Throws:

IOException

HoodieHBaseAvroHFileReader

public HoodieHBaseAvroHFileReader(StorageConfiguration<?> storageConf,
                                  StoragePath path)
                           throws IOException

Throws:

IOException

HoodieHBaseAvroHFileReader

public HoodieHBaseAvroHFileReader(StoragePath path,
                                  HoodieStorage storage,
                                  StorageConfiguration<?> storageConf,
                                  Option<org.apache.avro.Schema> schemaOpt,
                                  Option<byte[]> content)
                           throws IOException

Throws:

IOException

Method Detail

getRecordsByKeysIterator

public ClosableIterator<HoodieRecord<org.apache.avro.generic.IndexedRecord>> getRecordsByKeysIterator(List<String> sortedKeys,
                                                                                                      org.apache.avro.Schema schema)
                                                                                               throws IOException

Throws:

IOException

getRecordsByKeyPrefixIterator

public ClosableIterator<HoodieRecord<org.apache.avro.generic.IndexedRecord>> getRecordsByKeyPrefixIterator(List<String> sortedKeyPrefixes,
                                                                                                           org.apache.avro.Schema schema)
                                                                                                    throws IOException

Throws:

IOException

readMinMaxRecordKeys

public String[] readMinMaxRecordKeys()

readBloomFilter

public BloomFilter readBloomFilter()

getSchema

public org.apache.avro.Schema getSchema()

filterRowKeys

public Set<String> filterRowKeys(Set<String> candidateRowKeys)

Filter keys by availability.

Note: This method is performant when the caller passes in a sorted candidate keys.

Parameters:

candidateRowKeys - - Keys to check for the availability

Returns:

Subset of candidate keys that are available

getIndexedRecordIterator

public ClosableIterator<org.apache.avro.generic.IndexedRecord> getIndexedRecordIterator(org.apache.avro.Schema readerSchema,
                                                                                        org.apache.avro.Schema requestedSchema)

Specified by:

getIndexedRecordIterator in class HoodieAvroFileReader

getIndexedRecordsByKeysIterator

public ClosableIterator<org.apache.avro.generic.IndexedRecord> getIndexedRecordsByKeysIterator(List<String> keys,
                                                                                               org.apache.avro.Schema readerSchema)
                                                                                        throws IOException

Specified by:

getIndexedRecordsByKeysIterator in class HoodieAvroHFileReaderImplBase

Throws:

IOException

getIndexedRecordsByKeyPrefixIterator

public ClosableIterator<org.apache.avro.generic.IndexedRecord> getIndexedRecordsByKeyPrefixIterator(List<String> sortedKeyPrefixes,
                                                                                                    org.apache.avro.Schema readerSchema)
                                                                                             throws IOException

Specified by:

getIndexedRecordsByKeyPrefixIterator in class HoodieAvroHFileReaderImplBase

Throws:

IOException

getTotalRecords

public long getTotalRecords()

close

public void close()

getRecordKeyIterator

public ClosableIterator<String> getRecordKeyIterator()