org.apache.hadoop.fs.s3a.impl

Class BaseS3AFileSystemOperations

java.lang.Object

org.apache.hadoop.fs.s3a.impl.BaseS3AFileSystemOperations

All Implemented Interfaces:

S3AFileSystemOperations

public class BaseS3AFileSystemOperations
extends Object
implements S3AFileSystemOperations

An implementation of the S3AFileSystemOperations interface. This handles certain filesystem operations when s3 client side encryption is disabled.

Constructor Summary

Constructors

Constructor and Description
BaseS3AFileSystemOperations() Constructs a new instance of BaseS3AFileSystemOperations.

Method Summary

Modifier and Type	Method and Description
CSEMaterials	getClientSideEncryptionMaterials(org.apache.hadoop.conf.Configuration conf, String bucket, S3AEncryptionMethods algorithm) Retrieves the client-side encryption materials for the given bucket and encryption algorithm.
software.amazon.awssdk.core.ResponseInputStream<software.amazon.awssdk.services.s3.model.GetObjectResponse>	getObject(S3AStore store, software.amazon.awssdk.services.s3.model.GetObjectRequest request, RequestFactory factory) Retrieves an object from the S3.
S3ClientFactory	getS3ClientFactory(org.apache.hadoop.conf.Configuration conf) Retrieves the S3 client factory for the specified class and configuration.
long	getS3ObjectSize(String key, long length, S3AStore store, software.amazon.awssdk.services.s3.model.HeadObjectResponse response) Return the size of S3 object.
S3ClientFactory	getUnencryptedS3ClientFactory(org.apache.hadoop.conf.Configuration conf) Retrieves the S3 client factory for the specified class and configuration.
void	setCSEGauge(org.apache.hadoop.fs.statistics.impl.IOStatisticsStore ioStatisticsStore) Set the client side encryption gauge to 0.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseS3AFileSystemOperations

public BaseS3AFileSystemOperations()

Constructs a new instance of BaseS3AFileSystemOperations.

Method Detail

getObject

public software.amazon.awssdk.core.ResponseInputStream<software.amazon.awssdk.services.s3.model.GetObjectResponse> getObject(S3AStore store,
                                                                                                                             software.amazon.awssdk.services.s3.model.GetObjectRequest request,
                                                                                                                             RequestFactory factory)
                                                                                                                      throws IOException

Retrieves an object from the S3.

Specified by:

getObject in interface S3AFileSystemOperations

Parameters:

store - The S3AStore object representing the S3 bucket.

request - The GetObjectRequest containing the details of the object to retrieve.

factory - The RequestFactory used to create the GetObjectRequest.

Returns:

A ResponseInputStream containing the GetObjectResponse.

Throws:

IOException - If an error occurs while retrieving the object.

setCSEGauge

public void setCSEGauge(org.apache.hadoop.fs.statistics.impl.IOStatisticsStore ioStatisticsStore)

Set the client side encryption gauge to 0.

Specified by:

setCSEGauge in interface S3AFileSystemOperations

Parameters:

ioStatisticsStore - The IOStatisticsStore of the filesystem.

getClientSideEncryptionMaterials

public CSEMaterials getClientSideEncryptionMaterials(org.apache.hadoop.conf.Configuration conf,
                                                     String bucket,
                                                     S3AEncryptionMethods algorithm)

Retrieves the client-side encryption materials for the given bucket and encryption algorithm.

Specified by:

getClientSideEncryptionMaterials in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

bucket - The name of the S3 bucket.

algorithm - The client-side encryption algorithm to use.

Returns:

null.

getS3ClientFactory

public S3ClientFactory getS3ClientFactory(org.apache.hadoop.conf.Configuration conf)

Retrieves the S3 client factory for the specified class and configuration.

Specified by:

getS3ClientFactory in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

Returns:

The S3 client factory instance.

getUnencryptedS3ClientFactory

public S3ClientFactory getUnencryptedS3ClientFactory(org.apache.hadoop.conf.Configuration conf)

Retrieves the S3 client factory for the specified class and configuration.

Specified by:

getUnencryptedS3ClientFactory in interface S3AFileSystemOperations

Parameters:

conf - The Hadoop configuration object.

Returns:

null.

getS3ObjectSize

public long getS3ObjectSize(String key,
                            long length,
                            S3AStore store,
                            software.amazon.awssdk.services.s3.model.HeadObjectResponse response)
                     throws IOException

Return the size of S3 object.

Specified by:

getS3ObjectSize in interface S3AFileSystemOperations

Parameters:

key - The key (path) of the object in the S3 bucket.

length - The expected length of the object.

store - The S3AStore object representing the S3 bucket.

response - The HeadObjectResponse containing the metadata of the object.

Returns:

The size of the object in bytes.

Throws:

IOException - If an error occurs while retrieving the object size.