com.aliyun.odps.io

Class SourceInputStream

java.lang.Object

java.io.InputStream

com.aliyun.odps.io.SourceInputStream

All Implemented Interfaces:

Closeable, AutoCloseable

public abstract class SourceInputStream
extends InputStream

An extension of Java InputStream used to interface with the input file byte stream

Constructor Summary

Constructors

Constructor and Description
SourceInputStream()

Method Summary

Modifier and Type	Method and Description
abstract void	adjustMaxCloneLimit(int limit) Allow adjustment on the upper limit for number of maximum cloned streams allowed.
int	available()
abstract SourceInputStream	cloneStream() Clone a stream, the clone is a separate handle to the source data.
void	close()
abstract long	getBytesRead() Get total number of bytes read by current inputstream so far Note1: this can be less than (e.g., not all bytes read) or larger than (e.g., repeated reads from part of the input via seek operation) the physical size of the file underlying the input stream Note2: this denotes the "meaningful" size of read, bytes read into buffer but discarded (e.g., due to seek operation) are not accounted for.
abstract long	getCurrentPos() Getter for current position of the cursor, within the physical file
abstract String	getFileName() Getter for the file name associated with the file being streamed in.
abstract long	getFileSize() Getter for the size in [bytes] of the physical file currently being processed.
abstract long	getSplitSize() Getter for the split size supposedly assigned to the input stream.
abstract long	getSplitStart() Getter for the start position (within the physical stream) that is assigned to the input stream to process.
void	mark(int readLimit)
boolean	markSupported()
abstract int	read() Read one byte from stream.
abstract int	read(byte[] b) Reads some number of bytes from the input stream and stores them into the buffer array b.
abstract int	read(byte[] b, int offset, int length) Reads some number of bytes from the input stream and stores them into the buffer array b.
abstract int	readToEnd(byte[] buffer) An attempt to read rest of file content from current position (init to begin of file) to the end of current file split (when file is not split up, it will read the entire file) into the supplied buffer.
void	reset()
abstract long	skip(long n)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SourceInputStream

public SourceInputStream()

Method Detail

read

public abstract int read(byte[] b,
                         int offset,
                         int length)
                  throws IOException

Reads some number of bytes from the input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer.

Overrides:

read in class InputStream

Parameters:

b - the buffer into which the data is read

offset - the start offset in array b at which the data is written

length - the maximum number of bytes to read

Returns:

the total number of bytes read into the buffer, or -1 if there is no more data

Throws:

IOException

read

public abstract int read(byte[] b)
                  throws IOException

Reads some number of bytes from the input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer.

Overrides:

read in class InputStream

Parameters:

b - the buffer into which the data is read

Returns:

the total number of bytes read into the buffer, or -1 if there is no more data

Throws:

IOException

read

public abstract int read()
                  throws IOException

Read one byte from stream. This method is very inefficient. Use read(byte[], int, int) instead.

Specified by:

read in class InputStream

Returns:

The byte value read. -1 if no more data.

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

available

public int available()
              throws IOException

Overrides:

available in class InputStream

Throws:

IOException

mark

public void mark(int readLimit)

Overrides:

mark in class InputStream

markSupported

public boolean markSupported()

Overrides:

markSupported in class InputStream

reset

public void reset()

Overrides:

reset in class InputStream

skip

public abstract long skip(long n)
                   throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

getFileName

public abstract String getFileName()

Getter for the file name associated with the file being streamed in.

getFileSize

public abstract long getFileSize()

Getter for the size in [bytes] of the physical file currently being processed.

getSplitStart

public abstract long getSplitStart()

Getter for the start position (within the physical stream) that is assigned to the input stream to process. See SourceInputStream::getSplitSize() for more details.

Returns:

split start

getSplitSize

public abstract long getSplitSize()

Getter for the split size supposedly assigned to the input stream. Note that the combination of split start and split size is merely an indication of what portion of data is assigned to current input stream for processing. However, the input stream can still be used to access data anywhere in the physical stream, although doing so may result in duplicate reading of the same data across multiple input streams.

Returns:

split size

getCurrentPos

public abstract long getCurrentPos()
                            throws IOException

Getter for current position of the cursor, within the physical file

Returns:

current pos

Throws:

IOException

readToEnd

public abstract int readToEnd(byte[] buffer)
                       throws IOException,
                              BufferOverflowException

An attempt to read rest of file content from current position (init to begin of file) to the end of current file split (when file is not split up, it will read the entire file) into the supplied buffer. The supplied byte buffer will be first checked to ensure that it is large enough to hold read-out content, and may throw BufferOverflowException if the checks fail, before actual reading. Once successful, the read-out byte count will be returned. This could be used to read an entire file of less than Integer.MAX_VALUE bytes, for files larger than such limit, use a series of read(byte[] buffer, int offset, int length) instead.

Parameters:

buffer: - the byte buffer to host read-out bytes in a successful read

Throws:

IOException

BufferOverflowException

cloneStream

public abstract SourceInputStream cloneStream()
                                       throws IOException

Clone a stream, the clone is a separate handle to the source data. This allows flexible data manipulations, such as operations on different segments of the source files, without frequent seek/reset operations.

Returns:

cloned stream

Throws:

IOException

adjustMaxCloneLimit

public abstract void adjustMaxCloneLimit(int limit)

Allow adjustment on the upper limit for number of maximum cloned streams allowed. The system will set a reasonable upper limit by default, so unless absolutely necessary, it is NOT recommended that user adjusts this value.

Parameters:

limit -

getBytesRead

public abstract long getBytesRead()

Get total number of bytes read by current inputstream so far Note1: this can be less than (e.g., not all bytes read) or larger than (e.g., repeated reads from part of the input via seek operation) the physical size of the file underlying the input stream Note2: this denotes the "meaningful" size of read, bytes read into buffer but discarded (e.g., due to seek operation) are not accounted for.

Returns:

number of bytes read