org.apache.hadoop.fs.s3a.tools

Interface MarkerToolOperations

All Known Implementing Classes:

MarkerToolOperationsImpl

public interface MarkerToolOperations

Operations which must be offered by the store for MarkerTool. These are a proper subset of OperationCallbacks; this interface strips down those provided to the tool.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.RemoteIterator<S3AFileStatus>	listObjects(org.apache.hadoop.fs.Path path, String key) Create an iterator over objects in S3 only; S3Guard is not involved.
com.amazonaws.services.s3.model.DeleteObjectsResult	removeKeys(List<com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion> keysToDelete, boolean deleteFakeDir, List<org.apache.hadoop.fs.Path> undeletedObjectsOnFailure, BulkOperationState operationState, boolean quiet) Remove keys from the store, updating the metastore on a partial delete represented as a MultiObjectDeleteException failure by deleting all those entries successfully deleted and then rethrowing the MultiObjectDeleteException.

Method Detail

listObjects

@Retries.RetryTranslated
org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> listObjects(org.apache.hadoop.fs.Path path,
                                                                                        String key)
                                                                                 throws IOException

Create an iterator over objects in S3 only; S3Guard is not involved. The listing includes the key itself, if found.

Parameters:

path - path of the listing.

key - object key

Returns:

iterator with the first listing completed.

Throws:

IOException - failure.

removeKeys

@Retries.RetryMixed
com.amazonaws.services.s3.model.DeleteObjectsResult removeKeys(List<com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion> keysToDelete,
                                                                                   boolean deleteFakeDir,
                                                                                   List<org.apache.hadoop.fs.Path> undeletedObjectsOnFailure,
                                                                                   BulkOperationState operationState,
                                                                                   boolean quiet)
                                                                            throws com.amazonaws.services.s3.model.MultiObjectDeleteException,
                                                                                   com.amazonaws.AmazonClientException,
                                                                                   IOException

Remove keys from the store, updating the metastore on a partial delete represented as a MultiObjectDeleteException failure by deleting all those entries successfully deleted and then rethrowing the MultiObjectDeleteException.

Parameters:

keysToDelete - collection of keys to delete on the s3-backend. if empty, no request is made of the object store.

deleteFakeDir - indicates whether this is for deleting fake dirs.

undeletedObjectsOnFailure - List which will be built up of all files that were not deleted. This happens even as an exception is raised.

operationState - bulk operation state

quiet - should a bulk query be quiet, or should its result list all deleted keys

Returns:

the deletion result if a multi object delete was invoked and it returned without a failure, else null.

Throws:

org.apache.hadoop.fs.InvalidRequestException - if the request was rejected due to a mistaken attempt to delete the root directory.

com.amazonaws.services.s3.model.MultiObjectDeleteException - one or more of the keys could not be deleted in a multiple object delete operation.

com.amazonaws.AmazonClientException - amazon-layer failure.

IOException - other IO Exception.