org.apache.hadoop.fs.s3a

Class InconsistentS3ClientFactory

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.s3a.DefaultS3ClientFactory

org.apache.hadoop.fs.s3a.InconsistentS3ClientFactory

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, S3ClientFactory

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public class InconsistentS3ClientFactory
extends DefaultS3ClientFactory

S3 Client factory used for testing with eventual consistency fault injection. This client is for testing only; it is in the production hadoop-aws module to enable integration tests to use this just by editing the Hadoop configuration used to bring up the client. The factory uses the older constructor-based instantiation/configuration of the client, so does not wire up metrics, handlers etc.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.s3a.S3ClientFactory

S3ClientFactory.S3ClientCreationParameters

Field Summary

Fields inherited from class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory

LOG

Constructor Summary

Constructors

Constructor and Description
InconsistentS3ClientFactory()

Method Summary

Modifier and Type	Method and Description
protected com.amazonaws.services.s3.AmazonS3	buildAmazonS3Client(com.amazonaws.ClientConfiguration awsConf, S3ClientFactory.S3ClientCreationParameters parameters) Use the Builder API to create an AWS S3 client.

Methods inherited from class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory

buildAmazonS3EncryptionClient, configureAmazonS3Client, createEndpointConfiguration, createS3Client

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InconsistentS3ClientFactory

public InconsistentS3ClientFactory()

Method Detail

buildAmazonS3Client

protected com.amazonaws.services.s3.AmazonS3 buildAmazonS3Client(com.amazonaws.ClientConfiguration awsConf,
                                                                 S3ClientFactory.S3ClientCreationParameters parameters)

Description copied from class: DefaultS3ClientFactory

Use the Builder API to create an AWS S3 client.

This has a more complex endpoint configuration mechanism which initially caused problems; the withForceGlobalBucketAccessEnabled(true) command is critical here.

Overrides:

buildAmazonS3Client in class DefaultS3ClientFactory

Parameters:

awsConf - AWS configuration

parameters - parameters

Returns:

new AmazonS3 client