Package org.apache.beam.sdk.extensions.avro.io

Class AvroSource.AvroReader<T>

  • Type Parameters:
    T - The type of records contained in the block.
    All Implemented Interfaces:
    java.lang.AutoCloseable
    Enclosing class:
    AvroSource<T>
    public static class AvroSource.AvroReader<T>
extends org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>
    A BlockBasedSource.BlockBasedReader for reading blocks from Avro files.

    An Avro Object Container File consists of a header followed by a 16-bit sync marker and then a sequence of blocks, where each block begins with two encoded longs representing the total number of records in the block and the block's size in bytes, followed by the block's (optionally-encoded) records. Each block is terminated by a 16-bit sync marker.

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.io.BoundedSource.BoundedReader

        SPLIT_POINTS_UNKNOWN

    • Constructor Summary

      Constructors 
      Constructor Description
      AvroReader​(AvroSource<T> source)
      Reads Avro records of type T from the specified source.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroBlock<T> getCurrentBlock()  
      long getCurrentBlockOffset()  
      long getCurrentBlockSize()  
      AvroSource<T> getCurrentSource()  
      long getSplitPointsRemaining()  
      boolean readNextBlock()  
      protected void startReading​(java.nio.channels.ReadableByteChannel channel)  

      • Methods inherited from class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader

        getCurrent, getCurrentOffset, getFractionConsumed, isAtSplitPoint, readNextRecord

      • Methods inherited from class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader

        advanceImpl, allowsDynamicSplitting, close, startImpl

      • Methods inherited from class org.apache.beam.sdk.io.OffsetBasedSource.OffsetBasedReader

        advance, getSplitPointsConsumed, isDone, isStarted, splitAtFraction, start

      • Methods inherited from class org.apache.beam.sdk.io.BoundedSource.BoundedReader

        getCurrentTimestamp

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • AvroReader

        public AvroReader​(AvroSource<T> source)
        Reads Avro records of type T from the specified source.

    • Method Detail

      • getCurrentSource

        public AvroSource<T> getCurrentSource()
        Overrides:
        getCurrentSource in class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader<T>

      • readNextBlock

        public boolean readNextBlock()
        Specified by:
        readNextBlock in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

      • getCurrentBlock

        public org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroBlock<T> getCurrentBlock()
        Specified by:
        getCurrentBlock in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

      • getCurrentBlockOffset

        public long getCurrentBlockOffset()
        Specified by:
        getCurrentBlockOffset in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

      • getCurrentBlockSize

        public long getCurrentBlockSize()
        Specified by:
        getCurrentBlockSize in class org.apache.beam.sdk.io.BlockBasedSource.BlockBasedReader<T>

      • getSplitPointsRemaining

        public long getSplitPointsRemaining()
        Overrides:
        getSplitPointsRemaining in class org.apache.beam.sdk.io.OffsetBasedSource.OffsetBasedReader<T>

      • startReading

        protected void startReading​(java.nio.channels.ReadableByteChannel channel)
                     throws java.io.IOException
        Specified by:
        startReading in class org.apache.beam.sdk.io.FileBasedSource.FileBasedReader<T>
        Throws:
        java.io.IOException