org.apache.hadoop.fs.s3a.select

Class SelectBinding

java.lang.Object

org.apache.hadoop.fs.s3a.select.SelectBinding

public class SelectBinding
extends Object

Class to do the S3 select binding and build a select request from the supplied arguments/configuration. This class is intended to be instantiated by the owning S3AFileSystem instance to handle the construction of requests: IO is still done exclusively in the filesystem.

Constructor Summary

Constructors

Constructor and Description
SelectBinding(WriteOperationHelper operations) Constructor.

Method Summary

Modifier and Type	Method and Description
com.amazonaws.services.s3.model.InputSerialization	buildCsvInputRequest(org.apache.hadoop.conf.Configuration ownerConf, org.apache.hadoop.conf.Configuration builderOptions) Build the CSV input request.
com.amazonaws.services.s3.model.OutputSerialization	buildCSVOutput(org.apache.hadoop.conf.Configuration ownerConf, org.apache.hadoop.conf.Configuration builderOptions) Build CSV output for a request.
com.amazonaws.services.s3.model.SelectObjectContentRequest	buildSelectRequest(org.apache.hadoop.fs.Path path, String expression, org.apache.hadoop.conf.Configuration builderOptions) Build a select request.
boolean	isEnabled() Is the service supported?
static boolean	isSelectEnabled(org.apache.hadoop.conf.Configuration conf) Static probe for select being enabled.
org.apache.hadoop.fs.FSDataInputStream	select(S3AReadOpContext readContext, String expression, org.apache.hadoop.conf.Configuration builderOptions, S3ObjectAttributes objectAttributes) Build and execute a select request.
static String	toString(com.amazonaws.services.s3.model.SelectObjectContentRequest request) Stringify the given SelectObjectContentRequest, as its toString() operator doesn't.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SelectBinding

public SelectBinding(WriteOperationHelper operations)

Constructor.

Parameters:

operations - callback to owner FS, with associated span.

Method Detail

isEnabled

public boolean isEnabled()

Is the service supported?

Returns:

true iff select is enabled.

isSelectEnabled

public static boolean isSelectEnabled(org.apache.hadoop.conf.Configuration conf)

Static probe for select being enabled.

Parameters:

conf - configuration

Returns:

true iff select is enabled.

select

@Retries.RetryTranslated
public org.apache.hadoop.fs.FSDataInputStream select(S3AReadOpContext readContext,
                                                                              String expression,
                                                                              org.apache.hadoop.conf.Configuration builderOptions,
                                                                              S3ObjectAttributes objectAttributes)
                                                                       throws IOException

Build and execute a select request.

Parameters:

readContext - the read context, which includes the source path.

expression - the SQL expression.

builderOptions - query options

objectAttributes - object attributes from a HEAD request

Returns:

an FSDataInputStream whose wrapped stream is a SelectInputStream

Throws:

IllegalArgumentException - argument failure

IOException - failure building, validating or executing the request.

org.apache.hadoop.fs.PathIOException - source path is a directory.

buildSelectRequest

public com.amazonaws.services.s3.model.SelectObjectContentRequest buildSelectRequest(org.apache.hadoop.fs.Path path,
                                                                                     String expression,
                                                                                     org.apache.hadoop.conf.Configuration builderOptions)
                                                                              throws IOException

Build a select request.

Parameters:

path - source path.

expression - the SQL expression.

builderOptions - config to extract other query options from

Returns:

the request to serve

Throws:

IllegalArgumentException - argument failure

IOException - problem building/validating the request

buildCsvInputRequest

public com.amazonaws.services.s3.model.InputSerialization buildCsvInputRequest(org.apache.hadoop.conf.Configuration ownerConf,
                                                                               org.apache.hadoop.conf.Configuration builderOptions)
                                                                        throws IllegalArgumentException,
                                                                               IOException

Build the CSV input request.

Parameters:

ownerConf - FS owner configuration

builderOptions - options on the specific request

Returns:

the constructed request

Throws:

IllegalArgumentException - argument failure

IOException - validation failure

buildCSVOutput

public com.amazonaws.services.s3.model.OutputSerialization buildCSVOutput(org.apache.hadoop.conf.Configuration ownerConf,
                                                                          org.apache.hadoop.conf.Configuration builderOptions)
                                                                   throws IllegalArgumentException,
                                                                          IOException

Build CSV output for a request.

Parameters:

ownerConf - FS owner configuration

builderOptions - options on the specific request

Returns:

the constructed request

Throws:

IllegalArgumentException - argument failure

IOException - validation failure

toString

public static String toString(com.amazonaws.services.s3.model.SelectObjectContentRequest request)

Stringify the given SelectObjectContentRequest, as its toString() operator doesn't.

Parameters:

request - request to convert to a string

Returns:

a string to print. Does not contain secrets.