org.apache.hadoop.fs.s3a.api

Interface RequestFactory

All Known Implementing Classes:

RequestFactoryImpl

public interface RequestFactory

Factory for S3 objects. This is where the owner FS's prepareRequest() callback is invoked to mark up a request for this span. All AWS request objects MUST BE created through this, in the active audit span. The standard implementation provides a callback for the S3AFS or tests to be invoked to prepare each request. Such callbacks SHOULD NOT raise exceptions other than argument validation exceptions. as there are no guarantees how they are processed. That is: no guarantees of retry or translation.

Method Summary

Modifier and Type	Method and Description
Optional<com.amazonaws.services.s3.model.SSEAwsKeyManagementParams>	generateSSEAwsKeyParams() Create the AWS SDK structure used to configure SSE, if the encryption secrets contain the information/settings for this.
Optional<com.amazonaws.services.s3.model.SSECustomerKey>	generateSSECustomerKey() Create the SSE-C structure for the AWS SDK, if the encryption secrets contain the information/settings for this.
com.amazonaws.services.s3.model.CannedAccessControlList	getCannedACL() Get the canned ACL of this FS.
S3AEncryptionMethods	getServerSideEncryptionAlgorithm() Get the encryption algorithm of this endpoint.
com.amazonaws.services.s3.model.AbortMultipartUploadRequest	newAbortMultipartUploadRequest(String destKey, String uploadId) Abort a multipart upload.
com.amazonaws.services.s3.model.DeleteObjectsRequest	newBulkDeleteRequest(List<com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion> keysToDelete, boolean quiet) Bulk delete request.
com.amazonaws.services.s3.model.CompleteMultipartUploadRequest	newCompleteMultipartUploadRequest(String destKey, String uploadId, List<com.amazonaws.services.s3.model.PartETag> partETags) Complete a multipart upload.
com.amazonaws.services.s3.model.CopyObjectRequest	newCopyObjectRequest(String srcKey, String dstKey, com.amazonaws.services.s3.model.ObjectMetadata srcom) Create a copy request.
com.amazonaws.services.s3.model.DeleteObjectRequest	newDeleteObjectRequest(String key) Create a request to delete a single object.
com.amazonaws.services.s3.model.PutObjectRequest	newDirectoryMarkerRequest(String directory) Create a PutObjectRequest request for creating an empty directory.
com.amazonaws.services.s3.model.GetObjectMetadataRequest	newGetObjectMetadataRequest(String key) Create a HEAD request.
com.amazonaws.services.s3.model.GetObjectRequest	newGetObjectRequest(String key) Create a GET request.
com.amazonaws.services.s3.model.ListMultipartUploadsRequest	newListMultipartUploadsRequest(String prefix) List all multipart uploads under a prefix.
com.amazonaws.services.s3.model.ListNextBatchOfObjectsRequest	newListNextBatchOfObjectsRequest(com.amazonaws.services.s3.model.ObjectListing prev) Create the next V1 page list request, following on from the previous response.
com.amazonaws.services.s3.model.ListObjectsRequest	newListObjectsV1Request(String key, String delimiter, int maxKeys) Create the (legacy) V1 list request.
com.amazonaws.services.s3.model.ListObjectsV2Request	newListObjectsV2Request(String key, String delimiter, int maxKeys) Create a V2 list request.
com.amazonaws.services.s3.model.InitiateMultipartUploadRequest	newMultipartUploadRequest(String destKey) Start a multipart upload.
com.amazonaws.services.s3.model.ObjectMetadata	newObjectMetadata(long length) Create a new object metadata instance.
com.amazonaws.services.s3.model.PutObjectRequest	newPutObjectRequest(String key, com.amazonaws.services.s3.model.ObjectMetadata metadata, File srcfile) Create a putObject request.
com.amazonaws.services.s3.model.PutObjectRequest	newPutObjectRequest(String key, com.amazonaws.services.s3.model.ObjectMetadata metadata, InputStream inputStream) Create a PutObjectRequest request.
com.amazonaws.services.s3.model.SelectObjectContentRequest	newSelectRequest(String key) Create a S3 Select request for the destination object.
com.amazonaws.services.s3.model.UploadPartRequest	newUploadPartRequest(String destKey, String uploadId, int partNumber, int size, InputStream uploadStream, File sourceFile, long offset) Create and initialize a part request of a multipart upload.
void	setEncryptionSecrets(EncryptionSecrets secrets) Set the encryption secrets for all subsequent requests.

Method Detail

setEncryptionSecrets

void setEncryptionSecrets(EncryptionSecrets secrets)

Set the encryption secrets for all subsequent requests.

Parameters:

secrets - encryption secrets.

getCannedACL

com.amazonaws.services.s3.model.CannedAccessControlList getCannedACL()

Get the canned ACL of this FS.

Returns:

an ACL, if any

generateSSEAwsKeyParams

Optional<com.amazonaws.services.s3.model.SSEAwsKeyManagementParams> generateSSEAwsKeyParams()

Create the AWS SDK structure used to configure SSE, if the encryption secrets contain the information/settings for this.

Returns:

an optional set of KMS Key settings

generateSSECustomerKey

Optional<com.amazonaws.services.s3.model.SSECustomerKey> generateSSECustomerKey()

Create the SSE-C structure for the AWS SDK, if the encryption secrets contain the information/settings for this. This will contain a secret extracted from the bucket/configuration.

Returns:

an optional customer key.

getServerSideEncryptionAlgorithm

S3AEncryptionMethods getServerSideEncryptionAlgorithm()

Get the encryption algorithm of this endpoint.

Returns:

the encryption algorithm.

newObjectMetadata

com.amazonaws.services.s3.model.ObjectMetadata newObjectMetadata(long length)

Create a new object metadata instance. Any standard metadata headers are added here, for example: encryption.

Parameters:

length - length of data to set in header; Ignored if negative

Returns:

a new metadata instance

newCopyObjectRequest

com.amazonaws.services.s3.model.CopyObjectRequest newCopyObjectRequest(String srcKey,
                                                                       String dstKey,
                                                                       com.amazonaws.services.s3.model.ObjectMetadata srcom)

Create a copy request. This includes the work of copying the relevant parts of the metadata from the source

Parameters:

srcKey - source

dstKey - destination

srcom - source object metadata.

Returns:

the request

newPutObjectRequest

com.amazonaws.services.s3.model.PutObjectRequest newPutObjectRequest(String key,
                                                                     com.amazonaws.services.s3.model.ObjectMetadata metadata,
                                                                     File srcfile)

Create a putObject request. Adds the ACL and metadata

Parameters:

key - key of object

metadata - metadata header

srcfile - source file

Returns:

the request

newPutObjectRequest

com.amazonaws.services.s3.model.PutObjectRequest newPutObjectRequest(String key,
                                                                     com.amazonaws.services.s3.model.ObjectMetadata metadata,
                                                                     InputStream inputStream)

Create a PutObjectRequest request. The metadata is assumed to have been configured with the size of the operation.

Parameters:

key - key of object

metadata - metadata header

inputStream - source data.

Returns:

the request

newDirectoryMarkerRequest

com.amazonaws.services.s3.model.PutObjectRequest newDirectoryMarkerRequest(String directory)

Create a PutObjectRequest request for creating an empty directory.

Parameters:

directory - destination directory.

Returns:

request for a zero byte upload.

newListMultipartUploadsRequest

com.amazonaws.services.s3.model.ListMultipartUploadsRequest newListMultipartUploadsRequest(@Nullable
                                                                                           String prefix)

List all multipart uploads under a prefix.

Parameters:

prefix - prefix to list under

Returns:

the request.

newAbortMultipartUploadRequest

com.amazonaws.services.s3.model.AbortMultipartUploadRequest newAbortMultipartUploadRequest(String destKey,
                                                                                           String uploadId)

Abort a multipart upload.

Parameters:

destKey - destination object key

uploadId - ID of initiated upload

Returns:

the request.

newMultipartUploadRequest

com.amazonaws.services.s3.model.InitiateMultipartUploadRequest newMultipartUploadRequest(String destKey)

Start a multipart upload.

Parameters:

destKey - destination object key

Returns:

the request.

newCompleteMultipartUploadRequest

com.amazonaws.services.s3.model.CompleteMultipartUploadRequest newCompleteMultipartUploadRequest(String destKey,
                                                                                                 String uploadId,
                                                                                                 List<com.amazonaws.services.s3.model.PartETag> partETags)

Complete a multipart upload.

Parameters:

destKey - destination object key

uploadId - ID of initiated upload

partETags - ordered list of etags

Returns:

the request.

newGetObjectMetadataRequest

com.amazonaws.services.s3.model.GetObjectMetadataRequest newGetObjectMetadataRequest(String key)

Create a HEAD request.

Parameters:

key - key, may have trailing /

Returns:

the request.

newGetObjectRequest

com.amazonaws.services.s3.model.GetObjectRequest newGetObjectRequest(String key)

Create a GET request.

Parameters:

key - object key

Returns:

the request.

newUploadPartRequest

com.amazonaws.services.s3.model.UploadPartRequest newUploadPartRequest(String destKey,
                                                                       String uploadId,
                                                                       int partNumber,
                                                                       int size,
                                                                       InputStream uploadStream,
                                                                       File sourceFile,
                                                                       long offset)
                                                                throws org.apache.hadoop.fs.PathIOException

Create and initialize a part request of a multipart upload. Exactly one of: uploadStream or sourceFile must be specified. A subset of the file may be posted, by providing the starting point in offset and a length of block in size equal to or less than the remaining bytes.

Parameters:

destKey - destination key of ongoing operation

uploadId - ID of ongoing upload

partNumber - current part number of the upload

size - amount of data

uploadStream - source of data to upload

sourceFile - optional source file.

offset - offset in file to start reading.

Returns:

the request.

Throws:

org.apache.hadoop.fs.PathIOException - if the part number is out of range.

newSelectRequest

com.amazonaws.services.s3.model.SelectObjectContentRequest newSelectRequest(String key)

Create a S3 Select request for the destination object. This does not build the query.

Parameters:

key - object key

Returns:

the request

newListObjectsV1Request

com.amazonaws.services.s3.model.ListObjectsRequest newListObjectsV1Request(String key,
                                                                           String delimiter,
                                                                           int maxKeys)

Create the (legacy) V1 list request.

Parameters:

key - key to list under

delimiter - delimiter for keys

maxKeys - maximum number in a list page.

Returns:

the request

newListNextBatchOfObjectsRequest

com.amazonaws.services.s3.model.ListNextBatchOfObjectsRequest newListNextBatchOfObjectsRequest(com.amazonaws.services.s3.model.ObjectListing prev)

Create the next V1 page list request, following on from the previous response.

Parameters:

prev - previous response

Returns:

the request

newListObjectsV2Request

com.amazonaws.services.s3.model.ListObjectsV2Request newListObjectsV2Request(String key,
                                                                             String delimiter,
                                                                             int maxKeys)

Create a V2 list request. This will be recycled for any subsequent requests.

Parameters:

key - key to list under

delimiter - delimiter for keys

maxKeys - maximum number in a list page.

Returns:

the request

newDeleteObjectRequest

com.amazonaws.services.s3.model.DeleteObjectRequest newDeleteObjectRequest(String key)

Create a request to delete a single object.

Parameters:

key - object to delete

Returns:

the request

newBulkDeleteRequest

com.amazonaws.services.s3.model.DeleteObjectsRequest newBulkDeleteRequest(List<com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion> keysToDelete,
                                                                          boolean quiet)

Bulk delete request.

Parameters:

keysToDelete - list of keys to delete.

quiet - should a bulk query be quiet, or should its result list all deleted keys?

Returns:

the request