org.apache.hudi.common.table.log

Class HoodieLogFileReader

java.lang.Object

org.apache.hudi.common.table.log.HoodieLogFileReader

All Implemented Interfaces:

Closeable, AutoCloseable, Iterator<HoodieLogBlock>, HoodieLogFormat.Reader

public class HoodieLogFileReader
extends Object
implements HoodieLogFormat.Reader

Scans a log file and provides block level iterator on the log file Loads the entire block contents in memory Can emit either a DataBlock, CommandBlock, DeleteBlock or CorruptBlock (if one is found).

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean readBlockLazily)
HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean readBlockLazily, boolean reverseReader)
HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups, String keyField)
HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups, String keyField, InternalSchema internalSchema)

Method Summary

Modifier and Type	Method and Description
void	close()
HoodieLogFile	getLogFile()
boolean	hasNext()
boolean	hasPrev() hasPrev is not idempotent.
long	moveToPrev() Reverse pointer, does not read the block.
HoodieLogBlock	next()
HoodieLogBlock	prev() This is a reverse iterator Note: At any point, an instance of HoodieLogFileReader should either iterate reverse (prev) or forward (next).
void	remove()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

DEFAULT_BUFFER_SIZE

public static final int DEFAULT_BUFFER_SIZE

See Also:

Constant Field Values

Constructor Detail

HoodieLogFileReader

public HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean readBlockLazily)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean readBlockLazily,
                           boolean reverseReader)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean readBlockLazily,
                           boolean reverseReader,
                           boolean enableRecordLookups,
                           String keyField)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(org.apache.hadoop.fs.FileSystem fs,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean readBlockLazily,
                           boolean reverseReader,
                           boolean enableRecordLookups,
                           String keyField,
                           InternalSchema internalSchema)
                    throws IOException

Throws:

IOException

Method Detail

getLogFile

public HoodieLogFile getLogFile()

Specified by:

getLogFile in interface HoodieLogFormat.Reader

Returns:

the path to this HoodieLogFormat

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<HoodieLogBlock>

next

public HoodieLogBlock next()

Specified by:

next in interface Iterator<HoodieLogBlock>

hasPrev

public boolean hasPrev()

hasPrev is not idempotent.

Specified by:

hasPrev in interface HoodieLogFormat.Reader

Returns:

prev

public HoodieLogBlock prev()
                    throws IOException

This is a reverse iterator Note: At any point, an instance of HoodieLogFileReader should either iterate reverse (prev) or forward (next). Doing both in the same instance is not supported WARNING : Every call to prev() should be preceded with hasPrev()

Specified by:

prev in interface HoodieLogFormat.Reader

Returns:

Throws:

IOException

moveToPrev

public long moveToPrev()
                throws IOException

Reverse pointer, does not read the block. Return the current position of the log file (in reverse) If the pointer (inputstream) is moved in any way, it is the job of the client of this class to seek/reset it back to the file position returned from the method to expect correct results

Throws:

IOException

remove

public void remove()

Specified by:

remove in interface Iterator<HoodieLogBlock>