org.apache.hadoop.fs.s3a

Class Listing

java.lang.Object

org.apache.hadoop.fs.s3a.Listing

@InterfaceAudience.Private
public class Listing
extends Object

Place for the S3A listing classes; keeps all the small classes under control.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Listing.AcceptAllButSelfAndS3nDirs Accept all entries except the base path and those which map to S3N pseudo directory markers.

Constructor Summary

Constructors

Constructor and Description
Listing(S3AFileSystem owner)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.s3a.Listing.FileStatusListingIterator	createFileStatusListingIterator(org.apache.hadoop.fs.Path listPath, S3ListRequest request, org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.s3a.Listing.FileStatusAcceptor acceptor) Create a FileStatus iterator against a path, with a given list object request.
org.apache.hadoop.fs.s3a.Listing.FileStatusListingIterator	createFileStatusListingIterator(org.apache.hadoop.fs.Path listPath, S3ListRequest request, org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.s3a.Listing.FileStatusAcceptor acceptor, org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> providedStatus) Create a FileStatus iterator against a path, with a given list object request.
org.apache.hadoop.fs.s3a.Listing.LocatedFileStatusIterator	createLocatedFileStatusIterator(org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> statusIterator) Create a located status iterator over a file status iterator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Listing

public Listing(S3AFileSystem owner)

Method Detail

createFileStatusListingIterator

public org.apache.hadoop.fs.s3a.Listing.FileStatusListingIterator createFileStatusListingIterator(org.apache.hadoop.fs.Path listPath,
                                                                                                  S3ListRequest request,
                                                                                                  org.apache.hadoop.fs.PathFilter filter,
                                                                                                  org.apache.hadoop.fs.s3a.Listing.FileStatusAcceptor acceptor)
                                                                                           throws IOException

Create a FileStatus iterator against a path, with a given list object request.

Parameters:

listPath - path of the listing

request - initial request to make

filter - the filter on which paths to accept

acceptor - the class/predicate to decide which entries to accept in the listing based on the full file status.

Returns:

the iterator

Throws:

IOException - IO Problems

createFileStatusListingIterator

@Retries.RetryRaw
public org.apache.hadoop.fs.s3a.Listing.FileStatusListingIterator createFileStatusListingIterator(org.apache.hadoop.fs.Path listPath,
                                                                                                                    S3ListRequest request,
                                                                                                                    org.apache.hadoop.fs.PathFilter filter,
                                                                                                                    org.apache.hadoop.fs.s3a.Listing.FileStatusAcceptor acceptor,
                                                                                                                    org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> providedStatus)
                                                                                                             throws IOException

Create a FileStatus iterator against a path, with a given list object request.

Parameters:

listPath - path of the listing

request - initial request to make

filter - the filter on which paths to accept

acceptor - the class/predicate to decide which entries to accept in the listing based on the full file status.

providedStatus - the provided list of file status, which may contain items that are not listed from source.

Returns:

the iterator

Throws:

IOException - IO Problems

createLocatedFileStatusIterator

public org.apache.hadoop.fs.s3a.Listing.LocatedFileStatusIterator createLocatedFileStatusIterator(org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> statusIterator)

Create a located status iterator over a file status iterator.

Parameters:

statusIterator - an iterator over the remote status entries

Returns:

a new remote iterator