org.apache.hadoop.fs.s3a.impl

Class ClientManagerImpl

java.lang.Object

org.apache.hadoop.fs.s3a.impl.ClientManagerImpl

All Implemented Interfaces:

Closeable, AutoCloseable, ClientManager

public class ClientManagerImpl
extends Object
implements ClientManager

Client manager for on-demand creation of S3 clients, with parallelized close of them in close(). Updates Statistic.STORE_CLIENT_CREATION to track count and duration of client creation.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
ClientManagerImpl(S3ClientFactory clientFactory, S3ClientFactory.S3ClientCreationParameters clientCreationParameters, org.apache.hadoop.fs.statistics.DurationTrackerFactory durationTrackerFactory) Constructor.

Method Summary

Modifier and Type	Method and Description
void	close() Close() is synchronized to avoid race conditions between slow client creation and this close operation.
software.amazon.awssdk.services.s3.S3AsyncClient	getOrCreateAsyncClient() Get the Async S3Client,raising a failure to create as an IOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateAsyncS3ClientUnchecked() Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateS3Client() Get the S3Client, raising a failure to create as an IOException.
software.amazon.awssdk.services.s3.S3Client	getOrCreateS3ClientUnchecked() Get the S3Client, raising a failure to create as an UncheckedIOException.
software.amazon.awssdk.transfer.s3.S3TransferManager	getOrCreateTransferManager() Get the transfer manager, creating it and any dependencies if needed.
URI	getUri() Get the URI of the filesystem.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

ClientManagerImpl

public ClientManagerImpl(S3ClientFactory clientFactory,
                         S3ClientFactory.S3ClientCreationParameters clientCreationParameters,
                         org.apache.hadoop.fs.statistics.DurationTrackerFactory durationTrackerFactory)

Constructor. This does not create any clients.

Parameters:

clientFactory - client factory to invoke

clientCreationParameters - creation parameters.

durationTrackerFactory - duration tracker.

Method Detail

getOrCreateS3Client

public software.amazon.awssdk.services.s3.S3Client getOrCreateS3Client()
                                                                throws IOException

Description copied from interface: ClientManager

Get the S3Client, raising a failure to create as an IOException.

Specified by:

getOrCreateS3Client in interface ClientManager

Returns:

the S3 client

Throws:

IOException - failure to create the client.

getOrCreateS3ClientUnchecked

public software.amazon.awssdk.services.s3.S3Client getOrCreateS3ClientUnchecked()
                                                                         throws UncheckedIOException

Get the S3Client, raising a failure to create as an UncheckedIOException.

Specified by:

getOrCreateS3ClientUnchecked in interface ClientManager

Returns:

the S3 client

Throws:

UncheckedIOException - failure to create the client.

getOrCreateAsyncClient

public software.amazon.awssdk.services.s3.S3AsyncClient getOrCreateAsyncClient()
                                                                        throws IOException

Description copied from interface: ClientManager

Get the Async S3Client,raising a failure to create as an IOException.

Specified by:

getOrCreateAsyncClient in interface ClientManager

Returns:

the Async S3 client

Throws:

IOException - failure to create the client.

getOrCreateAsyncS3ClientUnchecked

public software.amazon.awssdk.services.s3.S3Client getOrCreateAsyncS3ClientUnchecked()
                                                                              throws UncheckedIOException

Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.

Specified by:

getOrCreateAsyncS3ClientUnchecked in interface ClientManager

Returns:

the S3 client

Throws:

UncheckedIOException - failure to create the client.

getOrCreateTransferManager

public software.amazon.awssdk.transfer.s3.S3TransferManager getOrCreateTransferManager()
                                                                                throws IOException

Description copied from interface: ClientManager

Get the transfer manager, creating it and any dependencies if needed.

Specified by:

getOrCreateTransferManager in interface ClientManager

Returns:

a transfer manager

Throws:

IOException - on any failure to create the manager

close

public void close()

Close() is synchronized to avoid race conditions between slow client creation and this close operation.

The objects are all deleted in parallel

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface ClientManager

getUri

public URI getUri()

Get the URI of the filesystem.

Returns:

URI to use when creating clients.

toString

public String toString()

Overrides:

toString in class Object