org.apache.hadoop.fs.s3a.impl

Class ClientManagerImpl

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.fs.s3a.impl.ClientManagerImpl

All Implemented Interfaces:

Closeable, AutoCloseable, ClientManager, org.apache.hadoop.service.Service

public class ClientManagerImpl
extends org.apache.hadoop.service.AbstractService
implements ClientManager

Client manager for on-demand creation of S3 clients, with parallelized close of them in serviceStop(). Updates Statistic.STORE_CLIENT_CREATION to track count and duration of client creation.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
ClientManagerImpl(S3ClientFactory clientFactory, S3ClientFactory unencryptedClientFactory, S3ClientFactory.S3ClientCreationParameters clientCreationParameters, org.apache.hadoop.fs.statistics.DurationTrackerFactory durationTrackerFactory) Constructor.

Method Summary

Modifier and Type	Method and Description
software.amazon.awssdk.services.s3.S3AsyncClient	getOrCreateAsyncClient() Get the Async S3Client,raising a failure to create as an IOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateAsyncS3ClientUnchecked() Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateS3Client() Get the S3Client, raising a failure to create as an IOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateS3ClientUnchecked() Get the S3Client, raising a failure to create as an UncheckedIOException.
software.amazon.awssdk.transfer.s3.S3TransferManager	getOrCreateTransferManager() Get the transfer manager, creating it and any dependencies if needed.
software.amazon.awssdk.services.s3.S3Client	getOrCreateUnencryptedS3Client() Get or create an unencrypted S3 client.
URI	getUri() Get the URI of the filesystem.
protected void	serviceStop()
String	toString()

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.service.Service

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, registerServiceListener, start, stop, unregisterServiceListener, waitForServiceToStop

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

ClientManagerImpl

public ClientManagerImpl(S3ClientFactory clientFactory,
                         S3ClientFactory unencryptedClientFactory,
                         S3ClientFactory.S3ClientCreationParameters clientCreationParameters,
                         org.apache.hadoop.fs.statistics.DurationTrackerFactory durationTrackerFactory)

Constructor.

This does not create any clients.

It does disable noisy logging from the S3 Transfer Manager.

Parameters:

clientFactory - client factory to invoke

unencryptedClientFactory - client factory to invoke

clientCreationParameters - creation parameters.

durationTrackerFactory - duration tracker.

Method Detail

getOrCreateS3Client

public software.amazon.awssdk.services.s3.S3Client getOrCreateS3Client()
                                                                throws IOException

Description copied from interface: ClientManager

Get the S3Client, raising a failure to create as an IOException.

Specified by:

getOrCreateS3Client in interface ClientManager

Returns:

the S3 client

Throws:

IOException - failure to create the client.

getOrCreateS3ClientUnchecked

public software.amazon.awssdk.services.s3.S3Client getOrCreateS3ClientUnchecked()
                                                                         throws UncheckedIOException

Get the S3Client, raising a failure to create as an UncheckedIOException.

Specified by:

getOrCreateS3ClientUnchecked in interface ClientManager

Returns:

the S3 client

Throws:

UncheckedIOException - failure to create the client.

getOrCreateAsyncClient

public software.amazon.awssdk.services.s3.S3AsyncClient getOrCreateAsyncClient()
                                                                        throws IOException

Description copied from interface: ClientManager

Get the Async S3Client,raising a failure to create as an IOException.

Specified by:

getOrCreateAsyncClient in interface ClientManager

Returns:

the Async S3 client

Throws:

IOException - failure to create the client.

getOrCreateAsyncS3ClientUnchecked

public software.amazon.awssdk.services.s3.S3Client getOrCreateAsyncS3ClientUnchecked()
                                                                              throws UncheckedIOException

Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.

Specified by:

getOrCreateAsyncS3ClientUnchecked in interface ClientManager

Returns:

the S3 client

Throws:

UncheckedIOException - failure to create the client.

getOrCreateUnencryptedS3Client

public software.amazon.awssdk.services.s3.S3Client getOrCreateUnencryptedS3Client()
                                                                           throws IOException

Get or create an unencrypted S3 client. This is used for unencrypted operations when CSE is enabled with V1 compatibility.

Specified by:

getOrCreateUnencryptedS3Client in interface ClientManager

Returns:

unencrypted S3 client

Throws:

IOException - on any failure

getOrCreateTransferManager

public software.amazon.awssdk.transfer.s3.S3TransferManager getOrCreateTransferManager()
                                                                                throws IOException

Description copied from interface: ClientManager

Get the transfer manager, creating it and any dependencies if needed.

Specified by:

getOrCreateTransferManager in interface ClientManager

Returns:

a transfer manager

Throws:

IOException - on any failure to create the manager

serviceStop

protected void serviceStop()
                    throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

getUri

public URI getUri()

Get the URI of the filesystem.

Returns:

URI to use when creating clients.

toString

public String toString()

Overrides:

toString in class org.apache.hadoop.service.AbstractService