org.apache.hadoop.fs.s3a.commit.magic

Class S3MagicCommitTracker

java.lang.Object

org.apache.hadoop.fs.s3a.commit.PutTracker

org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker

org.apache.hadoop.fs.s3a.commit.magic.S3MagicCommitTracker

public class S3MagicCommitTracker
extends MagicCommitTracker

Stores the commit data under the magic path.

Field Summary

Fields inherited from class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker

EMPTY, LOG

Constructor Summary

Constructors

Constructor and Description
S3MagicCommitTracker(org.apache.hadoop.fs.Path path, String bucket, String originalDestKey, String destKey, String pendingsetKey, WriteOperationHelper writer, PutTrackerStatistics trackerStatistics)

Method Summary

Modifier and Type	Method and Description
boolean	aboutToComplete(String uploadId, List<software.amazon.awssdk.services.s3.model.CompletedPart> parts, long bytesWritten, org.apache.hadoop.fs.statistics.IOStatistics iostatistics) Complete operation: generate the final commit data, put it.

Methods inherited from class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker

getBucket, getOriginalDestKey, getPath, getPendingPartKey, getTrackerStatistics, getWriter, initialize, outputImmediatelyVisible, toString

Methods inherited from class org.apache.hadoop.fs.s3a.commit.PutTracker

getDestKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

S3MagicCommitTracker

public S3MagicCommitTracker(org.apache.hadoop.fs.Path path,
                            String bucket,
                            String originalDestKey,
                            String destKey,
                            String pendingsetKey,
                            WriteOperationHelper writer,
                            PutTrackerStatistics trackerStatistics)

Method Detail

aboutToComplete

public boolean aboutToComplete(String uploadId,
                               List<software.amazon.awssdk.services.s3.model.CompletedPart> parts,
                               long bytesWritten,
                               org.apache.hadoop.fs.statistics.IOStatistics iostatistics)
                        throws IOException

Description copied from class: MagicCommitTracker

Complete operation: generate the final commit data, put it.

Specified by:

aboutToComplete in class MagicCommitTracker

Parameters:

uploadId - Upload ID

parts - list of parts

bytesWritten - bytes written

iostatistics - nullable IO statistics

Returns:

false, indicating that the commit must fail.

Throws:

IOException - any IO problem.