org.apache.hadoop.fs.s3a

Class S3AFileSystem.ListingOperationCallbacksImpl

java.lang.Object

org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl

All Implemented Interfaces:

ListingOperationCallbacks

Enclosing class:

S3AFileSystem

protected class S3AFileSystem.ListingOperationCallbacksImpl
extends Object
implements ListingOperationCallbacks

Callbacks from Listing. Auditing: the listing object is long-lived; the audit span for a single listing is passed in from the listing method calls and then down to the callbacks.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ListingOperationCallbacksImpl()

Method Summary

Modifier and Type	Method and Description
CompletableFuture<S3ListResult>	continueListObjectsAsync(S3ListRequest request, S3ListResult prevResult, org.apache.hadoop.fs.statistics.DurationTrackerFactory trackerFactory, org.apache.hadoop.fs.store.audit.AuditSpan span) List the next set of objects.
S3ListRequest	createListObjectsRequest(String key, String delimiter, org.apache.hadoop.fs.store.audit.AuditSpan span) Create a ListObjectsRequest request against this bucket, with the maximum keys returned in a query set in the FS config.
long	getDefaultBlockSize(org.apache.hadoop.fs.Path path) Return the number of bytes that large input files should be optimally be split into to minimize I/O time.
int	getMaxKeys() Get the maximum key count.
CompletableFuture<S3ListResult>	listObjectsAsync(S3ListRequest request, org.apache.hadoop.fs.statistics.DurationTrackerFactory trackerFactory, org.apache.hadoop.fs.store.audit.AuditSpan span) Initiate a listObjectsAsync operation, incrementing metrics in the process.
S3ALocatedFileStatus	toLocatedFileStatus(S3AFileStatus status) Build a S3ALocatedFileStatus from a FileStatus instance.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ListingOperationCallbacksImpl

protected ListingOperationCallbacksImpl()

Method Detail

listObjectsAsync

public CompletableFuture<S3ListResult> listObjectsAsync(S3ListRequest request,
                                                        org.apache.hadoop.fs.statistics.DurationTrackerFactory trackerFactory,
                                                        org.apache.hadoop.fs.store.audit.AuditSpan span)

Description copied from interface: ListingOperationCallbacks

Initiate a listObjectsAsync operation, incrementing metrics in the process. Retry policy: failures will come from the future.

Specified by:

listObjectsAsync in interface ListingOperationCallbacks

Parameters:

request - request to initiate

trackerFactory - tracker with statistics to update

span - audit span for this operation

Returns:

the results

continueListObjectsAsync

@Retries.RetryRaw
public CompletableFuture<S3ListResult> continueListObjectsAsync(S3ListRequest request,
                                                                                  S3ListResult prevResult,
                                                                                  org.apache.hadoop.fs.statistics.DurationTrackerFactory trackerFactory,
                                                                                  org.apache.hadoop.fs.store.audit.AuditSpan span)

Description copied from interface: ListingOperationCallbacks

List the next set of objects. Retry policy: failures will come from the future.

Specified by:

continueListObjectsAsync in interface ListingOperationCallbacks

Parameters:

request - last list objects request to continue

prevResult - last paged result to continue from

trackerFactory - tracker with statistics to update

span - audit span for the IO

Returns:

the next result object

toLocatedFileStatus

public S3ALocatedFileStatus toLocatedFileStatus(S3AFileStatus status)
                                         throws IOException

Description copied from interface: ListingOperationCallbacks

Build a S3ALocatedFileStatus from a FileStatus instance.

Specified by:

toLocatedFileStatus in interface ListingOperationCallbacks

Parameters:

status - file status

Returns:

a located status with block locations set up from this FS.

Throws:

IOException - IO Problems.

createListObjectsRequest

public S3ListRequest createListObjectsRequest(String key,
                                              String delimiter,
                                              org.apache.hadoop.fs.store.audit.AuditSpan span)

Description copied from interface: ListingOperationCallbacks

Create a ListObjectsRequest request against this bucket, with the maximum keys returned in a query set in the FS config. The active span for the FS is handed the request to prepare it before this method returns. ListingOperationCallbacks.getMaxKeys().

Specified by:

createListObjectsRequest in interface ListingOperationCallbacks

Parameters:

key - key for request

delimiter - any delimiter

span - span within which the request takes place.

Returns:

the request

getDefaultBlockSize

public long getDefaultBlockSize(org.apache.hadoop.fs.Path path)

Description copied from interface: ListingOperationCallbacks

Return the number of bytes that large input files should be optimally be split into to minimize I/O time. The given path will be used to locate the actual filesystem. The full path does not have to exist.

Specified by:

getDefaultBlockSize in interface ListingOperationCallbacks

Parameters:

path - path of file

Returns:

the default block size for the path's filesystem

getMaxKeys

public int getMaxKeys()

Description copied from interface: ListingOperationCallbacks

Get the maximum key count.

Specified by:

getMaxKeys in interface ListingOperationCallbacks

Returns:

a value, valid after initialization