Uses of Interface
org.apache.hadoop.fs.s3a.s3guard.MetadataStore

Packages that use MetadataStore

Package	Description
org.apache.hadoop.fs.s3a	S3A Filesystem.
org.apache.hadoop.fs.s3a.impl	Implementation classes private to the S3A store.
org.apache.hadoop.fs.s3a.s3guard	This package contains classes related to S3Guard: a feature of S3A to mask the eventual consistency behavior of S3 and optimize access patterns by coordinating with a strongly consistent external store for file system metadata.

Uses of MetadataStore in org.apache.hadoop.fs.s3a

Methods in org.apache.hadoop.fs.s3a that return MetadataStore

Modifier and Type	Method and Description
MetadataStore	S3AFileSystem.getMetadataStore() Get the metadata store.

Uses of MetadataStore in org.apache.hadoop.fs.s3a.impl

Methods in org.apache.hadoop.fs.s3a.impl that return MetadataStore

Modifier and Type	Method and Description
MetadataStore	StoreContext.getMetadataStore()

Methods in org.apache.hadoop.fs.s3a.impl with parameters of type MetadataStore

Modifier and Type	Method and Description
StoreContextBuilder	StoreContextBuilder.setMetadataStore(MetadataStore store)

Uses of MetadataStore in org.apache.hadoop.fs.s3a.s3guard

Classes in org.apache.hadoop.fs.s3a.s3guard that implement MetadataStore

Modifier and Type	Class and Description
class	DynamoDBMetadataStore DynamoDBMetadataStore is a MetadataStore that persists file system metadata to DynamoDB.
class	LocalMetadataStore This is a local, in-memory implementation of MetadataStore.
class	NullMetadataStore A no-op implementation of MetadataStore.

Methods in org.apache.hadoop.fs.s3a.s3guard that return MetadataStore

Modifier and Type	Method and Description
protected MetadataStore	RenameTracker.getMetadataStore() Get the metadata store.
static MetadataStore	S3Guard.getMetadataStore(org.apache.hadoop.fs.FileSystem fs, ITtlTimeProvider ttlTimeProvider) Create a new instance of the configured MetadataStore.
MetadataStore	S3GuardTool.getStore()
protected MetadataStore	S3GuardTool.initMetadataStore(boolean forceCreate) Parse metadata store from command line option or HDFS configuration.

Methods in org.apache.hadoop.fs.s3a.s3guard with parameters of type MetadataStore

Modifier and Type	Method and Description
static void	S3Guard.addAncestors(MetadataStore metadataStore, org.apache.hadoop.fs.Path qualifiedPath, ITtlTimeProvider timeProvider, BulkOperationState operationState) This adds all new ancestors of a path as directories.
static void	S3Guard.addMoveAncestors(MetadataStore ms, Collection<org.apache.hadoop.fs.Path> srcPaths, Collection<PathMetadata> dstMetas, org.apache.hadoop.fs.Path srcRoot, org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path dstPath, String owner) Helper method that records the move of all ancestors of a path.
static void	S3Guard.addMoveDir(MetadataStore ms, Collection<org.apache.hadoop.fs.Path> srcPaths, Collection<PathMetadata> dstMetas, org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path dstPath, String owner) Helper function that records the move of directory paths, adding resulting metadata to the supplied lists.
static void	S3Guard.addMoveFile(MetadataStore ms, Collection<org.apache.hadoop.fs.Path> srcPaths, Collection<PathMetadata> dstMetas, org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path dstPath, long size, long blockSize, String owner, String eTag, String versionId) Like S3Guard.addMoveDir(MetadataStore, Collection, Collection, Path, Path, String) (), but for files.
static org.apache.hadoop.fs.RemoteIterator<S3AFileStatus>	S3Guard.dirListingUnion(MetadataStore ms, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.RemoteIterator<S3AFileStatus> backingStatuses, DirListingMetadata dirMeta, boolean isAuthoritative, ITtlTimeProvider timeProvider, Function<S3AFileStatus[],org.apache.hadoop.fs.RemoteIterator<S3AFileStatus>> toStatusItr) Given directory listing metadata from both the backing store and the MetadataStore, merge the two sources of truth to create a consistent view of the current directory contents, which can be returned to clients.
static PathMetadata	S3Guard.getWithTtl(MetadataStore ms, org.apache.hadoop.fs.Path path, ITtlTimeProvider timeProvider, boolean needEmptyDirectoryFlag, boolean allowAuthoritative) Get a path entry provided it is not considered expired.
static BulkOperationState	S3Guard.initiateBulkWrite(MetadataStore metastore, BulkOperationState.OperationType operation, org.apache.hadoop.fs.Path path) Initiate a bulk write and create an operation state for it.
static boolean	S3Guard.isNullMetadataStore(MetadataStore ms) Although NullMetadataStore does nothing, callers may wish to avoid work (fast path) when the NullMetadataStore is in use.
static DirListingMetadata	S3Guard.listChildrenWithTtl(MetadataStore ms, org.apache.hadoop.fs.Path path, ITtlTimeProvider timeProvider, boolean allowAuthoritative) List children; mark the result as non-auth if the TTL has expired.
static void	S3Guard.makeDirsOrdered(MetadataStore ms, List<org.apache.hadoop.fs.Path> dirs, String owner, boolean authoritative, ITtlTimeProvider timeProvider) Deprecated. this is no longer called by S3AFilesystem.innerMkDirs. See: HADOOP-15079 (January 2018). It is currently retained because of its discussion in the method on atomicity and in case we need to reinstate it or adapt the current process of directory marker creation. But it is not being tested and so may age with time...consider deleting it in future if it's clear there's no need for it.
protected static void	S3GuardTool.printStoreDiagnostics(PrintStream out, MetadataStore store) Retrieve and Print store diagnostics.
static S3AFileStatus	S3Guard.putAndReturn(MetadataStore ms, S3AFileStatus status, ITtlTimeProvider timeProvider) Helper function which puts a given S3AFileStatus into the MetadataStore and returns the same S3AFileStatus.
static S3AFileStatus	S3Guard.putAndReturn(MetadataStore ms, S3AFileStatus status, ITtlTimeProvider timeProvider, BulkOperationState operationState) Helper function which puts a given S3AFileStatus into the MetadataStore and returns the same S3AFileStatus.
static void	S3Guard.putAuthDirectoryMarker(MetadataStore ms, S3AFileStatus status, ITtlTimeProvider timeProvider, BulkOperationState operationState) Creates an authoritative directory marker for the store.
static void	S3Guard.putWithTtl(MetadataStore ms, Collection<? extends PathMetadata> fileMetas, ITtlTimeProvider timeProvider, BulkOperationState operationState) Put entries, using the time provider to set their timestamp.
static void	S3Guard.putWithTtl(MetadataStore ms, DirListingMetadata dirMeta, List<org.apache.hadoop.fs.Path> unchangedEntries, ITtlTimeProvider timeProvider, BulkOperationState operationState) Put a directory entry, setting the updated timestamp of the directory and its children.
static void	S3Guard.putWithTtl(MetadataStore ms, PathMetadata fileMeta, ITtlTimeProvider timeProvider, BulkOperationState operationState) Put an entry, using the time provider to set its timestamp.
static boolean	S3Guard.refreshEntry(MetadataStore metadataStore, PathMetadata pm, S3AFileStatus s3AFileStatus, ITtlTimeProvider timeProvider) We update the metastore for the specific case of S3 value == S3Guard value so as to place a more recent modtime in the store.
protected void	S3GuardTool.setStore(MetadataStore store)

Constructors in org.apache.hadoop.fs.s3a.s3guard with parameters of type MetadataStore

Constructor and Description
DelayedUpdateRenameTracker(StoreContext storeContext, MetadataStore metadataStore, org.apache.hadoop.fs.Path sourceRoot, org.apache.hadoop.fs.Path dest, BulkOperationState operationState)
DescendantsIterator(MetadataStore ms, PathMetadata meta) Creates a new DescendantsIterator.
MetadataStoreListFilesIterator(MetadataStore ms, PathMetadata meta, boolean allowAuthoritative)
ProgressiveRenameTracker(StoreContext storeContext, MetadataStore metadataStore, org.apache.hadoop.fs.Path sourceRoot, org.apache.hadoop.fs.Path dest, BulkOperationState operationState)
RenameTracker(String name, StoreContext storeContext, MetadataStore metadataStore, org.apache.hadoop.fs.Path sourceRoot, org.apache.hadoop.fs.Path dest, BulkOperationState operationState) Constructor.
S3GuardFsck(S3AFileSystem fs, MetadataStore ms) Creates an S3GuardFsck.