org.apache.hadoop.fs.azurebfs.services

Class AbfsInputStream

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

org.apache.hadoop.fs.azurebfs.services.AbfsInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.fs.CanUnbuffer, org.apache.hadoop.fs.PositionedReadable, org.apache.hadoop.fs.Seekable, org.apache.hadoop.fs.statistics.IOStatisticsSource, org.apache.hadoop.fs.StreamCapabilities

public class AbfsInputStream
extends org.apache.hadoop.fs.FSInputStream
implements org.apache.hadoop.fs.CanUnbuffer, org.apache.hadoop.fs.StreamCapabilities, org.apache.hadoop.fs.statistics.IOStatisticsSource

The AbfsInputStream for AbfsClient.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

org.apache.hadoop.fs.StreamCapabilities.StreamCapability

Field Summary

Fields

Modifier and Type	Field and Description
static int	FOOTER_SIZE
static int	MAX_OPTIMIZED_READ_ATTEMPTS

Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO

Constructor Summary

Constructors

Constructor and Description
AbfsInputStream(AbfsClient client, org.apache.hadoop.fs.FileSystem.Statistics statistics, String path, long contentLength, AbfsInputStreamContext abfsInputStreamContext, String eTag, TracingContext tracingContext)

Method Summary

Modifier and Type	Method and Description
int	available() Return the size of the remaining available bytes if the size is less than or equal to Integer.MAX_VALUE, otherwise, return Integer.MAX_VALUE.
void	close()
int	getBufferSize()
long	getBytesFromReadAhead() Getter for bytes read from readAhead buffer that fills asynchronously.
long	getBytesFromRemoteRead() Getter for bytes read remotely from the data store.
protected int	getFooterReadBufferSize()
org.apache.hadoop.fs.statistics.IOStatistics	getIOStatistics()
String	getPath()
long	getPos() Return the current offset from the start of the file
int	getReadAheadQueueDepth()
int	getReadAheadRange()
String	getStreamID()
AbfsInputStreamStatistics	getStreamStatistics() Getter for AbfsInputStreamStatistics.
TracingContext	getTracingContext()
boolean	hasCapability(String capability)
boolean	isReadAheadEnabled()
long	length() Returns the length of the file that this stream refers to.
void	mark(int readlimit) Not supported by this stream.
boolean	markSupported() gets whether mark and reset are supported by ADLFileInputStream.
int	read()
int	read(byte[] b, int off, int len)
int	read(long position, byte[] buffer, int offset, int length)
void	registerListener(Listener listener1)
void	reset() Not supported by this stream.
void	seek(long n) Seek to given position in stream.
boolean	seekToNewSource(long l) Seeks a different copy of the data.
protected void	setCachedSasToken(CachedSASToken cachedSasToken)
boolean	shouldAlwaysReadBufferSize()
long	skip(long n)
String	toString() Get the statistics of the stream.
void	unbuffer()

Methods inherited from class org.apache.hadoop.fs.FSInputStream

readFully, readFully, validatePositionedReadArgs

Methods inherited from class java.io.InputStream

read

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

maxReadSizeForVectorReads, minSeekForVectorReads, readVectored

Field Detail

FOOTER_SIZE

public static final int FOOTER_SIZE

See Also:

Constant Field Values

MAX_OPTIMIZED_READ_ATTEMPTS

public static final int MAX_OPTIMIZED_READ_ATTEMPTS

See Also:

Constant Field Values

Constructor Detail

AbfsInputStream

public AbfsInputStream(AbfsClient client,
                       org.apache.hadoop.fs.FileSystem.Statistics statistics,
                       String path,
                       long contentLength,
                       AbfsInputStreamContext abfsInputStreamContext,
                       String eTag,
                       TracingContext tracingContext)

Method Detail

getPath

public String getPath()

read

public int read(long position,
                byte[] buffer,
                int offset,
                int length)
         throws IOException

Specified by:

read in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

read in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int off,
                int len)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

seek

public void seek(long n)
          throws IOException

Seek to given position in stream.

Specified by:

seek in interface org.apache.hadoop.fs.Seekable

Specified by:

seek in class org.apache.hadoop.fs.FSInputStream

Parameters:

n - position to seek to

Throws:

IOException - if there is an error

EOFException - if attempting to seek past end of file

skip

public long skip(long n)
          throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

available

public int available()
              throws IOException

Return the size of the remaining available bytes if the size is less than or equal to Integer.MAX_VALUE, otherwise, return Integer.MAX_VALUE. This is to match the behavior of DFSInputStream.available(), which some clients may rely on (HBase write-ahead log reading in particular).

Overrides:

available in class InputStream

Throws:

IOException

length

public long length()
            throws IOException

Returns the length of the file that this stream refers to. Note that the length returned is the length as of the time the Stream was opened. Specifically, if there have been subsequent appends to the file, they wont be reflected in the returned length.

Returns:

length of the file.

Throws:

IOException - if the stream is closed

getPos

public long getPos()
            throws IOException

Return the current offset from the start of the file

Specified by:

getPos in interface org.apache.hadoop.fs.Seekable

Specified by:

getPos in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException - throws IOException if there is an error

getTracingContext

public TracingContext getTracingContext()

seekToNewSource

public boolean seekToNewSource(long l)
                        throws IOException

Seeks a different copy of the data. Returns true if found a new source, false otherwise.

Specified by:

seekToNewSource in interface org.apache.hadoop.fs.Seekable

Specified by:

seekToNewSource in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException - throws IOException if there is an error

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

mark

public void mark(int readlimit)

Not supported by this stream. Throws UnsupportedOperationException

Overrides:

mark in class InputStream

Parameters:

readlimit - ignored

reset

public void reset()
           throws IOException

Not supported by this stream. Throws UnsupportedOperationException

Overrides:

reset in class InputStream

Throws:

IOException

markSupported

public boolean markSupported()

gets whether mark and reset are supported by ADLFileInputStream. Always returns false.

Overrides:

markSupported in class InputStream

Returns:

always false

unbuffer

public void unbuffer()

Specified by:

unbuffer in interface org.apache.hadoop.fs.CanUnbuffer

hasCapability

public boolean hasCapability(String capability)

Specified by:

hasCapability in interface org.apache.hadoop.fs.StreamCapabilities

isReadAheadEnabled

@VisibleForTesting
public boolean isReadAheadEnabled()

getReadAheadRange

@VisibleForTesting
public int getReadAheadRange()

setCachedSasToken

@VisibleForTesting
protected void setCachedSasToken(CachedSASToken cachedSasToken)

getStreamID

@VisibleForTesting
public String getStreamID()

getStreamStatistics

@VisibleForTesting
public AbfsInputStreamStatistics getStreamStatistics()

Getter for AbfsInputStreamStatistics.

Returns:

an instance of AbfsInputStreamStatistics.

registerListener

@VisibleForTesting
public void registerListener(Listener listener1)

getBytesFromReadAhead

@VisibleForTesting
public long getBytesFromReadAhead()

Getter for bytes read from readAhead buffer that fills asynchronously.

Returns:

value of the counter in long.

getBytesFromRemoteRead

@VisibleForTesting
public long getBytesFromRemoteRead()

Getter for bytes read remotely from the data store.

Returns:

value of the counter in long.

getBufferSize

@VisibleForTesting
public int getBufferSize()

getFooterReadBufferSize

@VisibleForTesting
protected int getFooterReadBufferSize()

getReadAheadQueueDepth

@VisibleForTesting
public int getReadAheadQueueDepth()

shouldAlwaysReadBufferSize

@VisibleForTesting
public boolean shouldAlwaysReadBufferSize()

getIOStatistics

public org.apache.hadoop.fs.statistics.IOStatistics getIOStatistics()

Specified by:

getIOStatistics in interface org.apache.hadoop.fs.statistics.IOStatisticsSource

toString

public String toString()

Get the statistics of the stream.

Overrides:

toString in class org.apache.hadoop.fs.FSInputStream

Returns:

a string value.