org.apache.beam.sdk.extensions.avro.io

Class AvroSource.AvroReader<T>

java.lang.Object

org.apache.beam.sdk.io.Source.Reader<T>

org.apache.beam.sdk.io.BoundedSource.BoundedReader<T>

org.apache.beam.sdk.io.OffsetBasedSource.OffsetBasedReader<T>

org.apache.beam.sdk.io.FileBasedSource.FileBasedReader<T>

org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader<T>

Type Parameters:

T - The type of records contained in the block.

All Implemented Interfaces:

java.lang.AutoCloseable

Enclosing class:

AvroSource<T>

public static class AvroSource.AvroReader<T>
extends org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

A BlockBasedReader for reading blocks from Avro files.

An Avro Object Container File consists of a header followed by a 16-bit sync marker and then a sequence of blocks, where each block begins with two encoded longs representing the total number of records in the block and the block's size in bytes, followed by the block's (optionally-encoded) records. Each block is terminated by a 16-bit sync marker.

Field Summary

Fields inherited from class org.apache.beam.sdk.io.BoundedSource.BoundedReader

SPLIT_POINTS_UNKNOWN

Constructor Summary

Constructors

Constructor and Description
AvroReader(AvroSource<T> source) Reads Avro records of type T from the specified source.

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroBlock<T>	getCurrentBlock()
long	getCurrentBlockOffset()
long	getCurrentBlockSize()
AvroSource<T>	getCurrentSource()
long	getSplitPointsRemaining()
boolean	readNextBlock()
protected void	startReading(java.nio.channels.ReadableByteChannel channel)

Methods inherited from class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader

getCurrent, getCurrentOffset, getFractionConsumed, isAtSplitPoint, readNextRecord

Methods inherited from class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader

advanceImpl, allowsDynamicSplitting, close, startImpl

Methods inherited from class org.apache.beam.sdk.io.OffsetBasedSource.OffsetBasedReader

advance, getSplitPointsConsumed, isDone, isStarted, splitAtFraction, start

Methods inherited from class org.apache.beam.sdk.io.BoundedSource.BoundedReader

getCurrentTimestamp

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroReader

public AvroReader(AvroSource<T> source)

Reads Avro records of type T from the specified source.

Method Detail

getCurrentSource

public AvroSource<T> getCurrentSource()

Overrides:

getCurrentSource in class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader<T>

readNextBlock

public boolean readNextBlock()

Specified by:

readNextBlock in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

getCurrentBlock

public org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroBlock<T> getCurrentBlock()

Specified by:

getCurrentBlock in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

getCurrentBlockOffset

public long getCurrentBlockOffset()

Specified by:

getCurrentBlockOffset in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

getCurrentBlockSize

public long getCurrentBlockSize()

Specified by:

getCurrentBlockSize in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

getSplitPointsRemaining

public long getSplitPointsRemaining()

Overrides:

getSplitPointsRemaining in class org.apache.beam.sdk.io.OffsetBasedSource.OffsetBasedReader<T>

startReading

protected void startReading(java.nio.channels.ReadableByteChannel channel)
                     throws java.io.IOException

Specified by:

startReading in class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader<T>

Throws:

java.io.IOException