org.apache.hadoop.fs.s3a.impl

Class OpenFileSupport

java.lang.Object

org.apache.hadoop.fs.s3a.impl.OpenFileSupport

public class OpenFileSupport
extends Object

Helper class for openFile() logic, especially processing file status args and length/etag/versionID.

This got complex enough it merited removal from S3AFileSystem -which also permits unit testing.

The default values are those from the FileSystem configuration. in openFile(), they can all be changed by specific options; in FileSystem.open(path, buffersize) only the buffer size is set.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OpenFileSupport.OpenFileInformation The information on a file needed to open it.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.store.LogExactlyOnce	LOG_NO_SQL_SELECT

Constructor Summary

Constructors

Constructor and Description
OpenFileSupport(ChangeDetectionPolicy changePolicy, long defaultReadAhead, String username, int defaultBufferSize, long defaultAsyncDrainThreshold, S3AInputPolicy defaultInputPolicy) Instantiate with the default options from the filesystem.

Method Summary

Modifier and Type	Method and Description
S3AReadOpContext	applyDefaultOptions(S3AReadOpContext roc) Propagate the default options to the operation context being built up.
ChangeDetectionPolicy	getChangePolicy()
long	getDefaultAsyncDrainThreshold()
int	getDefaultBufferSize()
long	getDefaultReadAhead()
OpenFileSupport.OpenFileInformation	openSimpleFile(int bufferSize) Open a simple file, using all the default options.
OpenFileSupport.OpenFileInformation	prepareToOpenFile(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.impl.OpenFileParameters parameters, long blockSize) Prepare to open a file from the openFile parameters.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG_NO_SQL_SELECT

public static final org.apache.hadoop.fs.store.LogExactlyOnce LOG_NO_SQL_SELECT

Constructor Detail

OpenFileSupport

public OpenFileSupport(ChangeDetectionPolicy changePolicy,
                       long defaultReadAhead,
                       String username,
                       int defaultBufferSize,
                       long defaultAsyncDrainThreshold,
                       S3AInputPolicy defaultInputPolicy)

Instantiate with the default options from the filesystem.

Parameters:

changePolicy - change detection policy

defaultReadAhead - read ahead range

username - username

defaultBufferSize - buffer size

defaultAsyncDrainThreshold - drain threshold

defaultInputPolicy - input policy

Method Detail

getChangePolicy

public ChangeDetectionPolicy getChangePolicy()

getDefaultReadAhead

public long getDefaultReadAhead()

getDefaultBufferSize

public int getDefaultBufferSize()

getDefaultAsyncDrainThreshold

public long getDefaultAsyncDrainThreshold()

applyDefaultOptions

public S3AReadOpContext applyDefaultOptions(S3AReadOpContext roc)

Propagate the default options to the operation context being built up.

Parameters:

roc - context

Returns:

the context

prepareToOpenFile

public OpenFileSupport.OpenFileInformation prepareToOpenFile(org.apache.hadoop.fs.Path path,
                                                             org.apache.hadoop.fs.impl.OpenFileParameters parameters,
                                                             long blockSize)
                                                      throws IOException

Prepare to open a file from the openFile parameters. S3Select SQL is rejected if a mandatory opt, ignored if optional.

Parameters:

path - path to the file

parameters - open file parameters from the builder.

blockSize - for fileStatus

Returns:

open file options

Throws:

IOException - failure to resolve the link.

IllegalArgumentException - unknown mandatory key

UnsupportedOperationException - for S3 Select options.

openSimpleFile

public OpenFileSupport.OpenFileInformation openSimpleFile(int bufferSize)

Open a simple file, using all the default options.

Parameters:

bufferSize - buffer size

Returns:

the parameters needed to open a file through open(path, bufferSize).

toString

public String toString()

Overrides:

toString in class Object