org.apache.hadoop.fs.s3a.s3guard

Class DelayedUpdateRenameTracker

java.lang.Object

org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation

org.apache.hadoop.fs.s3a.s3guard.RenameTracker

org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker

public class DelayedUpdateRenameTracker
extends RenameTracker

This is the rename updating strategy originally used: a collection of source paths and a list of destinations are created, then updated at the end (possibly slow).

It is not currently instantiated by any of the active trackers, but is preserved to show that the original rename strategy can be implemented via the tracker model.

Field Summary

Fields inherited from class org.apache.hadoop.fs.s3a.s3guard.RenameTracker

LOG

Constructor Summary

Constructors

Constructor and Description
DelayedUpdateRenameTracker(StoreContext storeContext, MetadataStore metadataStore, org.apache.hadoop.fs.Path sourceRoot, org.apache.hadoop.fs.Path dest, BulkOperationState operationState)

Method Summary

Modifier and Type	Method and Description
void	completeRename() Complete the operation.
void	directoryMarkerCopied(org.apache.hadoop.fs.Path sourcePath, org.apache.hadoop.fs.Path destPath, boolean addAncestors) A directory marker has been copied.
void	fileCopied(org.apache.hadoop.fs.Path sourcePath, S3ObjectAttributes sourceAttributes, S3ObjectAttributes destAttributes, org.apache.hadoop.fs.Path destPath, long blockSize, boolean addAncestors) A file has been copied.
void	moveSourceDirectory() Top level directory move.
IOException	renameFailed(Exception ex) Rename has failed.
void	sourceObjectsDeleted(Collection<org.apache.hadoop.fs.Path> paths) Note that source objects have been deleted.

Methods inherited from class org.apache.hadoop.fs.s3a.s3guard.RenameTracker

convertToIOException, deleteFailed, getDest, getMetadataStore, getOperationState, getOwner, getSourceRoot, noteRenameFinished, toString

Methods inherited from class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation

activateAuditSpan, getAuditSpan, getStoreContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

DelayedUpdateRenameTracker

public DelayedUpdateRenameTracker(StoreContext storeContext,
                                  MetadataStore metadataStore,
                                  org.apache.hadoop.fs.Path sourceRoot,
                                  org.apache.hadoop.fs.Path dest,
                                  BulkOperationState operationState)

Method Detail

fileCopied

public void fileCopied(org.apache.hadoop.fs.Path sourcePath,
                       S3ObjectAttributes sourceAttributes,
                       S3ObjectAttributes destAttributes,
                       org.apache.hadoop.fs.Path destPath,
                       long blockSize,
                       boolean addAncestors)
                throws IOException

Description copied from class: RenameTracker

A file has been copied.

Specified by:

fileCopied in class RenameTracker

Parameters:

sourcePath - source of the file. This may actually be different from the path of the sourceAttributes. (HOW?)

sourceAttributes - status of source.

destAttributes - destination attributes

destPath - destination path.

blockSize - block size.

addAncestors - should ancestors be added?

Throws:

IOException - failure.

directoryMarkerCopied

public void directoryMarkerCopied(org.apache.hadoop.fs.Path sourcePath,
                                  org.apache.hadoop.fs.Path destPath,
                                  boolean addAncestors)
                           throws IOException

Description copied from class: RenameTracker

A directory marker has been copied.

Overrides:

directoryMarkerCopied in class RenameTracker

Parameters:

sourcePath - source path.

destPath - destination path.

addAncestors - should ancestors be added?

Throws:

IOException - failure.

moveSourceDirectory

public void moveSourceDirectory()
                         throws IOException

Description copied from class: RenameTracker

Top level directory move. This is invoked after all child entries have been copied

Overrides:

moveSourceDirectory in class RenameTracker

Throws:

IOException - on failure

sourceObjectsDeleted

public void sourceObjectsDeleted(Collection<org.apache.hadoop.fs.Path> paths)
                          throws IOException

Description copied from class: RenameTracker

Note that source objects have been deleted. The metastore will already have been updated.

Overrides:

sourceObjectsDeleted in class RenameTracker

Parameters:

paths - path of objects deleted.

Throws:

IOException

completeRename

public void completeRename()
                    throws IOException

Description copied from class: RenameTracker

Complete the operation.

Overrides:

completeRename in class RenameTracker

Throws:

IOException - failure.

renameFailed

public IOException renameFailed(Exception ex)

Description copied from class: RenameTracker

Rename has failed.

The metastore now needs to be updated with its current state even though the operation is incomplete. Implementations MUST NOT throw exceptions here, as this is going to be invoked in an exception handler. catch and log or catch and return/wrap.

The base implementation returns the IOE passed in and translates any AWS exception into an IOE.

Overrides:

renameFailed in class RenameTracker

Parameters:

ex - the exception which caused the failure. This is either an IOException or and AWS exception

Returns:

an IOException to throw in an exception.