Index

A

acquire(Path) - Method in class io.delta.storage.internal.PathLock: Acquire a lock for the path before writing.
AzureLogStore - Class in io.delta.storage: LogStore implementation for Azure.
AzureLogStore(Configuration) - Constructor for class io.delta.storage.AzureLogStore

close() - Method in class io.delta.storage.LineCloseableIterator
CloseableIterator<T> - Interface in io.delta.storage: :: DeveloperApi :: An iterator that may contain resources which should be released after use.

deltaFile(Path, long) - Static method in class io.delta.storage.internal.FileNameUtils: Returns the delta (json format) path for a given delta file.
deltaVersion(Path) - Static method in class io.delta.storage.internal.FileNameUtils: Returns the version for the given delta path.

FileNameUtils - Class in io.delta.storage.internal: Helper for misc functions relating to file names for delta commits.
FileNameUtils() - Constructor for class io.delta.storage.internal.FileNameUtils

GCSLogStore - Class in io.delta.storage: The LogStore implementation for GCS, which uses gcs-connector to provide the necessary atomic and durability guarantees: Atomic Visibility: Read/read-after-metadata-update/delete are strongly consistent for GCS. Consistent Listing: GCS guarantees strong consistency for both object and bucket listing operations.
GCSLogStore(Configuration) - Constructor for class io.delta.storage.GCSLogStore

HadoopFileSystemLogStore - Class in io.delta.storage: Default implementation of LogStore for Hadoop FileSystem implementations.
HadoopFileSystemLogStore(Configuration) - Constructor for class io.delta.storage.HadoopFileSystemLogStore
hasNext() - Method in class io.delta.storage.LineCloseableIterator
HDFSLogStore - Class in io.delta.storage: The LogStore implementation for HDFS, which uses Hadoop FileContext API's to provide the necessary atomic and durability guarantees: Atomic visibility of files: `FileContext.rename` is used write files which is atomic for HDFS. Consistent file listing: HDFS file listing is consistent.
HDFSLogStore(Configuration) - Constructor for class io.delta.storage.HDFSLogStore

incorrectLogStoreImplementationException(Throwable) - Static method in class io.delta.storage.internal.LogStoreErrors
initHadoopConf() - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Hadoop configuration that should only be used during initialization of LogStore.
io.delta.storage - package io.delta.storage
io.delta.storage.internal - package io.delta.storage.internal
isDeltaFile(Path) - Static method in class io.delta.storage.internal.FileNameUtils: Returns true if the given path is a delta file, else false.
isNonFatal(Throwable) - Static method in class io.delta.storage.internal.LogStoreErrors: Returns true if the provided Throwable is to be considered non-fatal, or false if it is to be considered fatal
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.AzureLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.GCSLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.HDFSLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.LocalLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Whether a partial write is visible for the underlying file system of `path`.
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore

LineCloseableIterator - Class in io.delta.storage: Turn a Reader to CloseableIterator which can be read on demand.
LineCloseableIterator(Reader) - Constructor for class io.delta.storage.LineCloseableIterator
listFrom(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
listFrom(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given `path`.
listFrom(Path, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore
LocalLogStore - Class in io.delta.storage: Default LogStore implementation (should be used for testing only!).
LocalLogStore(Configuration) - Constructor for class io.delta.storage.LocalLogStore
LogStore - Class in io.delta.storage: :: DeveloperApi ::
LogStore(Configuration) - Constructor for class io.delta.storage.LogStore
LogStoreErrors - Class in io.delta.storage.internal
LogStoreErrors() - Constructor for class io.delta.storage.internal.LogStoreErrors

next() - Method in class io.delta.storage.LineCloseableIterator
NO_ABSTRACT_FILE_SYSTEM_EXCEPTION_MESSAGE - Static variable in class io.delta.storage.HDFSLogStore

PathLock - Class in io.delta.storage.internal: A lock that provides per-file-path `acquire` and `release` semantics.
PathLock() - Constructor for class io.delta.storage.internal.PathLock

read(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
read(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Load the given file and return an `Iterator` of lines, with line breaks removed from each line.
release(Path) - Method in class io.delta.storage.internal.PathLock: Release the lock for the path after writing.
resolvePathOnPhysicalStorage(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
resolvePathOnPhysicalStorage(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Resolve the fully qualified path for the given `path`.
runInNewThread(String, boolean, Callable<T>) - Static method in class io.delta.storage.internal.ThreadUtils: Based on Apache Spark's ThreadUtils.runInNewThread Run a piece of code in a new thread and return the result.

s3ListFromArray(FileSystem, Path, Path) - Static method in class io.delta.storage.internal.S3LogStoreUtil: Uses the S3ListRequest.v2 interface with the startAfter parameter to only list files which are lexicographically greater than resolvedPath.
S3LogStoreUtil - Class in io.delta.storage.internal: Static utility methods for the S3SingleDriverLogStore.
S3SingleDriverLogStore - Class in io.delta.storage: Single Spark-driver/JVM LogStore implementation for S3.
S3SingleDriverLogStore(Configuration) - Constructor for class io.delta.storage.S3SingleDriverLogStore

write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.AzureLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.GCSLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.HDFSLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.LocalLogStore: This write implementation needs to wrap `writeWithRename` with `synchronized` as rename() for RawLocalFileSystem doesn't throw an exception when the target file exists.
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Write the given `actions` to the given `path` with or without overwrite as indicated.
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore