org.apache.hadoop.fs.s3a.select

Class SelectInputStream

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

org.apache.hadoop.fs.s3a.select.SelectInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.fs.CanSetReadahead, org.apache.hadoop.fs.PositionedReadable, org.apache.hadoop.fs.Seekable

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public class SelectInputStream
extends org.apache.hadoop.fs.FSInputStream
implements org.apache.hadoop.fs.CanSetReadahead

An input stream for S3 Select return values. This is simply an end-to-end GET request, without any form of seek or recovery from connectivity failures. Currently only seek and positioned read operations on the current location are supported. The normal S3 input counters are updated by this stream.

Field Summary

Fields

Modifier and Type	Field and Description
static String	SEEK_UNSUPPORTED

Constructor Summary

Constructors

Constructor and Description
SelectInputStream(S3AReadOpContext readContext, S3ObjectAttributes objectAttributes, com.amazonaws.services.s3.model.SelectObjectContentResult selectResponse) Create the stream.

Method Summary

Modifier and Type	Method and Description
void	abort() Aborts the IO.
int	available()
void	close()
long	getPos()
long	getReadahead() Get the current readahead value.
S3AInputStreamStatistics	getS3AStreamStatistics() Get the Stream statistics.
void	mark(int readLimit)
boolean	markSupported()
int	read() Read a byte.
int	read(byte[] buf, int off, int len)
int	read(long position, byte[] buffer, int offset, int length) Read at a specific position.
void	reset()
void	seek(long newPos) Forward seeks are supported, but not backwards ones.
boolean	seekToNewSource(long targetPos)
void	setReadahead(Long readahead) Set the readahead.
long	skip(long n)
String	toString() String value includes statistics as well as stream state.
protected org.apache.hadoop.fs.PathIOException	unsupported(String action) Build an exception to raise when an operation is not supported here.

Methods inherited from class org.apache.hadoop.fs.FSInputStream

readFully, readFully, validatePositionedReadArgs

Methods inherited from class java.io.InputStream

read

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

SEEK_UNSUPPORTED

public static final String SEEK_UNSUPPORTED

See Also:

Constant Field Values

Constructor Detail

SelectInputStream

@Retries.OnceTranslated
public SelectInputStream(S3AReadOpContext readContext,
                                                 S3ObjectAttributes objectAttributes,
                                                 com.amazonaws.services.s3.model.SelectObjectContentResult selectResponse)
                                          throws IOException

Create the stream. The read attempt is initiated immediately.

Parameters:

readContext - read context

objectAttributes - object attributes from a HEAD request

selectResponse - response from the already executed call

Throws:

IOException - failure

Method Detail

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

available

public int available()
              throws IOException

Overrides:

available in class InputStream

Throws:

IOException

skip

@Retries.OnceTranslated
public long skip(long n)
                                  throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

getPos

public long getPos()

Specified by:

getPos in interface org.apache.hadoop.fs.Seekable

Specified by:

getPos in class org.apache.hadoop.fs.FSInputStream

setReadahead

public void setReadahead(Long readahead)

Set the readahead.

Specified by:

setReadahead in interface org.apache.hadoop.fs.CanSetReadahead

Parameters:

readahead - The readahead to use. null means to use the default.

getReadahead

public long getReadahead()

Get the current readahead value.

Returns:

the readahead

read

@Retries.OnceTranslated
public int read()
                                 throws IOException

Read a byte. There's no attempt to recover, but AWS-SDK exceptions such as SelectObjectContentEventException are translated into IOExceptions.

Specified by:

read in class InputStream

Returns:

a byte read or -1 for an end of file.

Throws:

IOException - failure.

read

@Retries.OnceTranslated
public int read(byte[] buf,
                                        int off,
                                        int len)
                                 throws IOException

Overrides:

read in class InputStream

Throws:

IOException

seek

@Retries.OnceTranslated
public void seek(long newPos)
                                  throws IOException

Forward seeks are supported, but not backwards ones. Forward seeks are implemented using read, so means that long-distance seeks will be (literally) expensive.

Specified by:

seek in interface org.apache.hadoop.fs.Seekable

Specified by:

seek in class org.apache.hadoop.fs.FSInputStream

Parameters:

newPos - new seek position.

Throws:

org.apache.hadoop.fs.PathIOException - Backwards seek attempted.

EOFException - attempt to seek past the end of the stream.

IOException - IO failure while skipping bytes

unsupported

protected org.apache.hadoop.fs.PathIOException unsupported(String action)

Build an exception to raise when an operation is not supported here.

Parameters:

action - action which is unsupported.

Returns:

an exception to throw.

seekToNewSource

public boolean seekToNewSource(long targetPos)
                        throws IOException

Specified by:

seekToNewSource in interface org.apache.hadoop.fs.Seekable

Specified by:

seekToNewSource in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

markSupported

public boolean markSupported()

Overrides:

markSupported in class InputStream

mark

public void mark(int readLimit)

Overrides:

mark in class InputStream

reset

public void reset()
           throws IOException

Overrides:

reset in class InputStream

Throws:

IOException

abort

public void abort()

Aborts the IO.

read

public int read(long position,
                byte[] buffer,
                int offset,
                int length)
         throws IOException

Read at a specific position. Reads at a position earlier than the current getPos() position will fail with a PathIOException. See seek(long). Unlike the base implementation And the requirements of the filesystem specification, this updates the stream position as returned in getPos().

Specified by:

read in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

read in class org.apache.hadoop.fs.FSInputStream

Parameters:

position - offset in the stream.

buffer - buffer to read in to.

offset - offset within the buffer

length - amount of data to read.

Returns:

the result.

Throws:

org.apache.hadoop.fs.PathIOException - Backwards seek attempted.

EOFException - attempt to seek past the end of the stream.

IOException - IO failure while seeking in the stream or reading data.

getS3AStreamStatistics

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public S3AInputStreamStatistics getS3AStreamStatistics()

Get the Stream statistics.

Returns:

the statistics for this stream.

toString

@InterfaceStability.Unstable
public String toString()

String value includes statistics as well as stream state. Important: there are no guarantees as to the stability of this value.

Overrides:

toString in class org.apache.hadoop.fs.FSInputStream

Returns:

a string value for printing in logs/diagnostics