org.apache.hadoop.fs.s3a.impl

Class CSES3AFileSystemOperations

java.lang.Object

org.apache.hadoop.fs.s3a.impl.CSES3AFileSystemOperations

All Implemented Interfaces:

S3AFileSystemOperations

Direct Known Subclasses:

CSEV1CompatibleS3AFileSystemOperations

public class CSES3AFileSystemOperations
extends Object
implements S3AFileSystemOperations

An implementation of the S3AFileSystemOperations interface. This handles certain filesystem operations when s3 client side encryption is enabled.

Constructor Summary

Constructors

Constructor and Description
CSES3AFileSystemOperations() Constructs a new instance of CSES3AFileSystemOperations.

Method Summary

Modifier and Type	Method and Description
CSEMaterials	getClientSideEncryptionMaterials(org.apache.hadoop.conf.Configuration conf, String bucket, S3AEncryptionMethods algorithm) Retrieves the cse materials.
software.amazon.awssdk.core.ResponseInputStream<software.amazon.awssdk.services.s3.model.GetObjectResponse>	getObject(S3AStore store, software.amazon.awssdk.services.s3.model.GetObjectRequest request, RequestFactory factory) Retrieves an object from the S3 using encrypted S3 client.
S3ClientFactory	getS3ClientFactory(org.apache.hadoop.conf.Configuration conf) Retrieves the S3 client factory for the specified class and configuration.
long	getS3ObjectSize(String key, long length, S3AStore store, software.amazon.awssdk.services.s3.model.HeadObjectResponse response) Retrieves the unencrypted length of an object in the S3 bucket.
S3ClientFactory	getUnencryptedS3ClientFactory(org.apache.hadoop.conf.Configuration conf) Retrieves the S3 client factory for the specified class and configuration.
void	setCSEGauge(org.apache.hadoop.fs.statistics.impl.IOStatisticsStore ioStatisticsStore) Set the client side encryption gauge 1.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CSES3AFileSystemOperations

public CSES3AFileSystemOperations()

Constructs a new instance of CSES3AFileSystemOperations.

Method Detail

getObject

public software.amazon.awssdk.core.ResponseInputStream<software.amazon.awssdk.services.s3.model.GetObjectResponse> getObject(S3AStore store,
                                                                                                                             software.amazon.awssdk.services.s3.model.GetObjectRequest request,
                                                                                                                             RequestFactory factory)
                                                                                                                      throws IOException

Retrieves an object from the S3 using encrypted S3 client.

Specified by:

getObject in interface S3AFileSystemOperations

Parameters:

store - The S3AStore object representing the S3 bucket.

request - The GetObjectRequest containing the details of the object to retrieve.

factory - The RequestFactory used to create the GetObjectRequest.

Returns:

A ResponseInputStream containing the GetObjectResponse.

Throws:

IOException - If an error occurs while retrieving the object.

setCSEGauge

public void setCSEGauge(org.apache.hadoop.fs.statistics.impl.IOStatisticsStore ioStatisticsStore)

Set the client side encryption gauge 1.

Specified by:

setCSEGauge in interface S3AFileSystemOperations

Parameters:

ioStatisticsStore - The IOStatisticsStore of the filesystem.

getClientSideEncryptionMaterials

public CSEMaterials getClientSideEncryptionMaterials(org.apache.hadoop.conf.Configuration conf,
                                                     String bucket,
                                                     S3AEncryptionMethods algorithm)
                                              throws IOException

Retrieves the cse materials.

Specified by:

getClientSideEncryptionMaterials in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

bucket - The name of the S3 bucket.

algorithm - The client-side encryption algorithm to use.

Returns:

The client-side encryption materials.

Throws:

IOException - If an error occurs while retrieving the encryption materials.

getS3ClientFactory

public S3ClientFactory getS3ClientFactory(org.apache.hadoop.conf.Configuration conf)

Retrieves the S3 client factory for the specified class and configuration.

Specified by:

getS3ClientFactory in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

Returns:

The S3 client factory instance.

getUnencryptedS3ClientFactory

public S3ClientFactory getUnencryptedS3ClientFactory(org.apache.hadoop.conf.Configuration conf)

Retrieves the S3 client factory for the specified class and configuration.

Specified by:

getUnencryptedS3ClientFactory in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

Returns:

null

getS3ObjectSize

public long getS3ObjectSize(String key,
                            long length,
                            S3AStore store,
                            software.amazon.awssdk.services.s3.model.HeadObjectResponse response)
                     throws IOException

Retrieves the unencrypted length of an object in the S3 bucket.

Specified by:

getS3ObjectSize in interface S3AFileSystemOperations

Parameters:

key - The key (path) of the object in the S3 bucket.

length - The length of the object.

store - The S3AStore object representing the S3 bucket.

response - The HeadObjectResponse containing the metadata of the object.

Returns:

The unencrypted size of the object in bytes.

Throws:

IOException - If an error occurs while retrieving the object size.