| Package | Description |
|---|---|
| org.apache.hadoop.fs.s3a.commit |
Support for committing the output of analytics jobs directly to S3.
|
| org.apache.hadoop.fs.s3a.commit.files |
Persistent data formats for the committers.
|
| Modifier and Type | Method and Description |
|---|---|
SinglePendingCommit |
CommitOperations.uploadFileToPendingCommit(File localFile,
org.apache.hadoop.fs.Path destPath,
String partition,
long uploadPartSize,
org.apache.hadoop.util.Progressable progress)
Upload all the data in the local file, returning the information
needed to commit the work.
|
| Modifier and Type | Method and Description |
|---|---|
void |
CommitOperations.CommitContext.abortSingleCommit(SinglePendingCommit commit)
|
CommitOperations.MaybeIOE |
CommitOperations.CommitContext.commit(SinglePendingCommit commit,
String origin)
Commit a single pending commit; exceptions are caught
and converted to an outcome.
|
void |
CommitOperations.CommitContext.commitOrFail(SinglePendingCommit commit)
Commit the operation, throwing an exception on any failure.
|
void |
CommitOperations.CommitContext.revertCommit(SinglePendingCommit commit)
|
void |
CommitOperations.revertCommit(SinglePendingCommit commit,
BulkOperationState operationState)
Revert a pending commit by deleting the destination.
|
| Modifier and Type | Method and Description |
|---|---|
protected void |
AbstractS3ACommitter.abortPendingUploads(org.apache.hadoop.mapreduce.JobContext context,
List<SinglePendingCommit> pending,
boolean suppressExceptions)
Abort all pending uploads in the list.
|
| Modifier and Type | Method and Description |
|---|---|
static SinglePendingCommit |
SinglePendingCommit.load(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Load an instance from a file, then validate it.
|
| Modifier and Type | Method and Description |
|---|---|
List<SinglePendingCommit> |
PendingSet.getCommits() |
static org.apache.hadoop.util.JsonSerialization<SinglePendingCommit> |
SinglePendingCommit.serializer()
Get a JSON serializer for this class.
|
| Modifier and Type | Method and Description |
|---|---|
void |
PendingSet.add(SinglePendingCommit commit)
Add a commit.
|
| Modifier and Type | Method and Description |
|---|---|
void |
PendingSet.setCommits(List<SinglePendingCommit> commits) |
Copyright © 2008–2022 Apache Software Foundation. All rights reserved.