org.apache.samza.system.hdfs.reader

Class AvroFileHdfsReader

java.lang.Object

org.apache.samza.system.hdfs.reader.AvroFileHdfsReader

All Implemented Interfaces:

SingleFileHdfsReader

public class AvroFileHdfsReader
extends java.lang.Object
implements SingleFileHdfsReader

An implementation of the HdfsReader that reads and processes avro format files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroFileHdfsReader.AvroFileCheckpoint An avro file looks something like this: Byte offset: 0 103 271 391 ┌────────┬──────────────┬───────────┬───────────┐ Avro file: │ Header │ Block 1 │ Block 2 │ Block 3 │ ...

Constructor Summary

Constructors

Constructor and Description
AvroFileHdfsReader(org.apache.samza.system.SystemStreamPartition systemStreamPartition)

Method Summary

Modifier and Type	Method and Description
void	close() Close the reader.
boolean	hasNext() Whether there are still records to be read
java.lang.String	nextOffset() Get the next offset, which is the offset for the next message that will be returned by readNext
static int	offsetComparator(java.lang.String offset1, java.lang.String offset2)
void	open(java.lang.String pathStr, java.lang.String singleFileOffset) Open the file and seek to specific offset for reading.
org.apache.samza.system.IncomingMessageEnvelope	readNext() Construct and return the next message envelope
void	seek(java.lang.String singleFileOffset) Seek to a specific offset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroFileHdfsReader

public AvroFileHdfsReader(org.apache.samza.system.SystemStreamPartition systemStreamPartition)

Method Detail

open

public void open(java.lang.String pathStr,
                 java.lang.String singleFileOffset)

Description copied from interface: SingleFileHdfsReader

Open the file and seek to specific offset for reading.

Specified by:

open in interface SingleFileHdfsReader

Parameters:

pathStr - path of the file to be read

singleFileOffset - offset the reader should start from

seek

public void seek(java.lang.String singleFileOffset)

Description copied from interface: SingleFileHdfsReader

Seek to a specific offset

Specified by:

seek in interface SingleFileHdfsReader

Parameters:

singleFileOffset - offset the reader should seek to

readNext

public org.apache.samza.system.IncomingMessageEnvelope readNext()

Description copied from interface: SingleFileHdfsReader

Construct and return the next message envelope

Specified by:

readNext in interface SingleFileHdfsReader

Returns:

constructed IncomeMessageEnvelope

hasNext

public boolean hasNext()

Description copied from interface: SingleFileHdfsReader

Whether there are still records to be read

Specified by:

hasNext in interface SingleFileHdfsReader

Returns:

true of false based on whether the reader has hit end of file

close

public void close()

Description copied from interface: SingleFileHdfsReader

Close the reader.

Specified by:

close in interface SingleFileHdfsReader

nextOffset

public java.lang.String nextOffset()

Description copied from interface: SingleFileHdfsReader

Get the next offset, which is the offset for the next message that will be returned by readNext

Specified by:

nextOffset in interface SingleFileHdfsReader

Returns:

next offset

offsetComparator

public static int offsetComparator(java.lang.String offset1,
                                   java.lang.String offset2)