org.apache.hadoop.fs.s3a.impl.streams

Class StreamIntegration

java.lang.Object

org.apache.hadoop.fs.s3a.impl.streams.StreamIntegration

public final class StreamIntegration
extends Object

Stream integration, including factory construction.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ANALYTICS Enum/config name of the analytics input stream: "analytics".
static String	CLASSIC Enum/config name of a classic S3AInputStream: "classic".
static String	CUSTOM Reads in a classname : "custom".
static String	DEFAULT Special string for configuration only; is mapped to the default stream type: "default".
static InputStreamType	DEFAULT_STREAM_TYPE What is the default type?
static String	E_EMPTY_CUSTOM_CLASSNAME
static String	E_INVALID_STREAM_TYPE
static String	PREFETCH Enum/config name of of the Pinterest S3APrefetchingInputStream: "prefetch".

Method Summary

Modifier and Type	Method and Description
static InputStreamType	determineInputStreamType(org.apache.hadoop.conf.Configuration conf) Determine the input stream type for the supplied configuration.
static ObjectInputStreamFactory	factoryFromConfig(org.apache.hadoop.conf.Configuration conf) Create the input stream factory the configuration asks for.
static VectoredIOContext	populateVectoredIOContext(org.apache.hadoop.conf.Configuration conf) Populates the configurations related to vectored IO operations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASSIC

public static final String CLASSIC

Enum/config name of a classic S3AInputStream: "classic".

See Also:

Constant Field Values

PREFETCH

public static final String PREFETCH

Enum/config name of of the Pinterest S3APrefetchingInputStream: "prefetch".

See Also:

Constant Field Values

ANALYTICS

public static final String ANALYTICS

Enum/config name of the analytics input stream: "analytics".

See Also:

Constant Field Values

CUSTOM

public static final String CUSTOM

Reads in a classname : "custom".

See Also:

Constant Field Values

DEFAULT

public static final String DEFAULT

Special string for configuration only; is mapped to the default stream type: "default".

See Also:

Constant Field Values

DEFAULT_STREAM_TYPE

public static final InputStreamType DEFAULT_STREAM_TYPE

What is the default type?

E_EMPTY_CUSTOM_CLASSNAME

public static final String E_EMPTY_CUSTOM_CLASSNAME

See Also:

Constant Field Values

E_INVALID_STREAM_TYPE

public static final String E_INVALID_STREAM_TYPE

See Also:

Constant Field Values

Method Detail

factoryFromConfig

public static ObjectInputStreamFactory factoryFromConfig(org.apache.hadoop.conf.Configuration conf)

Create the input stream factory the configuration asks for.

This does not initialize the factory.

See determineInputStreamType(Configuration) for the resolution algorithm.

Parameters:

conf - configuration

Returns:

a stream factory.

Throws:

RuntimeException - any binding/loading/instantiation problem

determineInputStreamType

public static InputStreamType determineInputStreamType(org.apache.hadoop.conf.Configuration conf)

Determine the input stream type for the supplied configuration.

This does not perform any instantiation.

If the option "fs.s3a.prefetch.enabled" is set, the prefetch stream is selected, after printing a warning the first time this happens.

If the input stream type is declared as "default", then whatever the current default stream type is returned, as defined by DEFAULT_STREAM_TYPE.

Parameters:

conf - configuration

Returns:

a stream factory.

populateVectoredIOContext

public static VectoredIOContext populateVectoredIOContext(org.apache.hadoop.conf.Configuration conf)

Populates the configurations related to vectored IO operations. The context is still mutable at this point.

Parameters:

conf - configuration object.

Returns:

VectoredIOContext.