| Package | Description |
|---|---|
| org.apache.hadoop.fs.s3a |
S3A Filesystem.
|
| org.apache.hadoop.fs.s3a.s3guard |
This package contains classes related to S3Guard: a feature of S3A to mask
the eventual consistency behavior of S3 and optimize access patterns by
coordinating with a strongly consistent external store for file system
metadata.
|
| Modifier and Type | Method and Description |
|---|---|
org.apache.commons.lang3.tuple.Triple<org.apache.hadoop.fs.RemoteIterator<S3AFileStatus>,DirListingMetadata,Boolean> |
Listing.getFileStatusesAssumingNonEmptyDir(org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.store.audit.AuditSpan span)
Calculate list of file statuses assuming path
to be a non-empty directory.
|
| Modifier and Type | Method and Description |
|---|---|
DirListingMetadata |
LocalMetadataEntry.getDirListingMeta() |
DirListingMetadata |
S3GuardFsckViolationHandler.ViolationHandler.getMsDirListing() |
DirListingMetadata |
S3GuardFsck.ComparePair.getMsDirListing() |
DirListingMetadata |
DynamoDBMetadataStore.listChildren(org.apache.hadoop.fs.Path path) |
DirListingMetadata |
LocalMetadataStore.listChildren(org.apache.hadoop.fs.Path p) |
DirListingMetadata |
MetadataStore.listChildren(org.apache.hadoop.fs.Path path)
Lists metadata for all direct children of a path.
|
DirListingMetadata |
NullMetadataStore.listChildren(org.apache.hadoop.fs.Path path) |
static DirListingMetadata |
S3Guard.listChildrenWithTtl(MetadataStore ms,
org.apache.hadoop.fs.Path path,
ITtlTimeProvider timeProvider,
boolean allowAuthoritative)
List children; mark the result as non-auth if the TTL has expired.
|
DirListingMetadata |
DirListingMetadata.withoutTombstones()
Get the directory listing excluding tombstones.
|
| Modifier and Type | Method and Description |
|---|---|
static org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> |
S3Guard.dirListingUnion(MetadataStore ms,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> backingStatuses,
DirListingMetadata dirMeta,
boolean isAuthoritative,
ITtlTimeProvider timeProvider,
Function<S3AFileStatus[],org.apache.hadoop.fs.RemoteIterator<S3AFileStatus>> toStatusItr)
Given directory listing metadata from both the backing store and the
MetadataStore, merge the two sources of truth to create a consistent
view of the current directory contents, which can be returned to clients.
|
static S3AFileStatus[] |
S3Guard.dirMetaToStatuses(DirListingMetadata dirMeta)
Convert the data of a directory listing to an array of
FileStatus
entries. |
void |
DynamoDBMetadataStore.put(DirListingMetadata meta,
List<org.apache.hadoop.fs.Path> unchangedEntries,
BulkOperationState operationState)
Save directory listing metadata.
|
void |
LocalMetadataStore.put(DirListingMetadata meta,
List<org.apache.hadoop.fs.Path> unchangedEntries,
BulkOperationState operationState) |
void |
MetadataStore.put(DirListingMetadata meta,
List<org.apache.hadoop.fs.Path> unchangedEntries,
BulkOperationState operationState)
Save directory listing metadata.
|
void |
NullMetadataStore.put(DirListingMetadata meta,
List<org.apache.hadoop.fs.Path> unchangedEntries,
BulkOperationState operationState) |
static void |
S3Guard.putWithTtl(MetadataStore ms,
DirListingMetadata dirMeta,
List<org.apache.hadoop.fs.Path> unchangedEntries,
ITtlTimeProvider timeProvider,
BulkOperationState operationState)
Put a directory entry, setting the updated timestamp of the
directory and its children.
|
void |
LocalMetadataEntry.setDirListingMetadata(DirListingMetadata dirListingMetadata) |
| Constructor and Description |
|---|
DirListingMetadata(DirListingMetadata d)
Copy constructor.
|
Copyright © 2008–2022 Apache Software Foundation. All rights reserved.