Uses of Class
org.apache.hadoop.fs.s3a.S3AFileSystem

Packages that use S3AFileSystem

Package	Description
org.apache.hadoop.fs.s3a	S3A Filesystem.
org.apache.hadoop.fs.s3a.commit	Support for committing the output of analytics jobs directly to S3.
org.apache.hadoop.fs.s3a.commit.magic	This is the "Magic" committer and support.
org.apache.hadoop.fs.s3a.commit.staging	The staging committers.
org.apache.hadoop.fs.s3a.impl	Implementation classes private to the S3A store.
org.apache.hadoop.fs.s3a.s3guard	This package contains classes related to S3Guard: a feature of S3A to mask the eventual consistency behavior of S3 and optimize access patterns by coordinating with a strongly consistent external store for file system metadata.

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a

Methods in org.apache.hadoop.fs.s3a with parameters of type S3AFileSystem

Modifier and Type	Method and Description
static void	InconsistentAmazonS3Client.clearInconsistency(S3AFileSystem fs) Clear any accumulated inconsistency state.
static void	InconsistentAmazonS3Client.setFailureInjectionPolicy(S3AFileSystem fs, FailureInjectionPolicy policy) A way for tests to patch in a different fault injection policy at runtime.

Constructors in org.apache.hadoop.fs.s3a with parameters of type S3AFileSystem

Constructor and Description
ProgressableProgressListener(S3AFileSystem fs, String key, com.amazonaws.services.s3.transfer.Upload upload, org.apache.hadoop.util.Progressable progress) Instantiate.
WriteOperationHelper(S3AFileSystem owner, org.apache.hadoop.conf.Configuration conf, S3AStatisticsContext statisticsContext, org.apache.hadoop.fs.store.audit.AuditSpanSource auditSpanSource, org.apache.hadoop.fs.store.audit.AuditSpan auditSpan) Constructor.

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a.commit

Methods in org.apache.hadoop.fs.s3a.commit that return S3AFileSystem

Modifier and Type	Method and Description
S3AFileSystem	AbstractS3ACommitter.getDestS3AFS() Get the destination as an S3A Filesystem; casting it.
static S3AFileSystem	CommitUtils.getS3AFileSystem(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, boolean magicCommitRequired) Get the S3A FS of a path.
static S3AFileSystem	CommitUtils.verifyIsS3AFS(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) Verify that an FS is an S3A FS.

Methods in org.apache.hadoop.fs.s3a.commit with parameters of type S3AFileSystem

Modifier and Type	Method and Description
abstract org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	AbstractS3ACommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context) Implementation point: create a task committer for a specific filesystem.
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	S3ACommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context) Create a task committer.
static void	CommitUtils.verifyIsMagicCommitFS(S3AFileSystem fs) Verify that an S3A FS instance is a magic commit FS.
static void	CommitUtils.verifyIsMagicCommitPath(S3AFileSystem fs, org.apache.hadoop.fs.Path path) Verify that the path is a magic one.

Constructors in org.apache.hadoop.fs.s3a.commit with parameters of type S3AFileSystem

Constructor and Description
CommitOperations(S3AFileSystem fs) Instantiate.
CommitOperations(S3AFileSystem fs, CommitterStatistics committerStatistics) Instantiate.
MagicCommitIntegration(S3AFileSystem owner, boolean magicCommitEnabled) Instantiate.

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a.commit.magic

Methods in org.apache.hadoop.fs.s3a.commit.magic with parameters of type S3AFileSystem

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	MagicS3GuardCommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a.commit.staging

Methods in org.apache.hadoop.fs.s3a.commit.staging with parameters of type S3AFileSystem

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	PartitionedStagingCommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	DirectoryStagingCommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	StagingCommitterFactory.createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a.impl

Constructors in org.apache.hadoop.fs.s3a.impl with parameters of type S3AFileSystem

Constructor and Description
S3AMultipartUploaderBuilder(S3AFileSystem fileSystem, WriteOperations writeOperations, StoreContext context, org.apache.hadoop.fs.Path p, S3AMultipartUploaderStatistics statistics)

Uses of S3AFileSystem in org.apache.hadoop.fs.s3a.s3guard

Methods in org.apache.hadoop.fs.s3a.s3guard that return S3AFileSystem

Modifier and Type	Method and Description
protected S3AFileSystem	S3GuardTool.bindFilesystem(org.apache.hadoop.fs.FileSystem bindingFS) Sets the filesystem; it must be an S3A FS instance, or a FilterFS around an S3A Filesystem.
protected S3AFileSystem	S3GuardTool.getFilesystem()
S3AFileSystem	AbstractS3GuardDynamoDBDiagnostic.getFilesystem()

Methods in org.apache.hadoop.fs.s3a.s3guard with parameters of type S3AFileSystem

Modifier and Type	Method and Description
static boolean	S3Guard.allowAuthoritative(org.apache.hadoop.fs.Path p, S3AFileSystem fs, boolean authMetadataStore, Collection<String> authPaths) Is the path for the given FS instance authoritative?
static DumpS3GuardDynamoTable	DumpS3GuardDynamoTable.dumpStore(S3AFileSystem fs, DynamoDBMetadataStore store, org.apache.hadoop.conf.Configuration conf, File destFile, URI uri) Entry point to dump the metastore and s3 store world views
String	S3GuardFsckViolationHandler.ViolationHandler.fixViolation(S3AFileSystem fs, DynamoDBMetadataStore ddbms)
String	S3GuardFsckViolationHandler.OrphanDDBEntry.fixViolation(S3AFileSystem fs, DynamoDBMetadataStore ddbms)
static Collection<String>	S3Guard.getAuthoritativePaths(S3AFileSystem fs)
static org.apache.commons.lang3.tuple.Pair<Long,Long>	PurgeS3GuardDynamoTable.purgeStore(S3AFileSystem fs, DynamoDBMetadataStore store, org.apache.hadoop.conf.Configuration conf, URI uri, boolean force) API Entry point to dump the metastore and S3 store world views

Constructors in org.apache.hadoop.fs.s3a.s3guard with parameters of type S3AFileSystem

Constructor and Description
AbstractS3GuardDynamoDBDiagnostic(String name, S3AFileSystem filesystem, DynamoDBMetadataStore store, URI uri) Constructor.
DumpS3GuardDynamoTable(S3AFileSystem fs, DynamoDBMetadataStore store, File destFile, URI uri) Bind to a specific FS + store.
PurgeS3GuardDynamoTable(S3AFileSystem filesystem, DynamoDBMetadataStore store, URI uri, boolean force)
S3GuardFsck(S3AFileSystem fs, MetadataStore ms) Creates an S3GuardFsck.
S3GuardFsckViolationHandler(S3AFileSystem fs, DynamoDBMetadataStore ddbms)