Packages

trait LogStore extends AnyRef

General interface for all critical file system operations required to read and write the DeltaLog. The correctness of the DeltaLog is predicated on the atomicity and durability guarantees of the implementation of this interface. Specifically,

1. Atomic visibility of files: Any file written through this store must be made visible atomically. In other words, this should not generate partial files.

2. Mutual exclusion: Only one writer must be able to create (or rename) a file at the final destination.

3. Consistent listing: Once a file has been written in a directory, all future listings for that directory must return that file.

Linear Supertypes
AnyRef, Any
Known Subclasses
AzureLogStore, HDFSLogStore, HadoopFileSystemLogStore, LocalLogStore, S3SingleDriverLogStore
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LogStore
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def invalidateCache(): Unit

    Invalidate any caching that the implementation may be using

  2. abstract def listFrom(path: Path): Iterator[FileStatus]

    List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given path.

    List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given path. The result should also be sorted by the file name.

  3. abstract def read(path: Path): Seq[String]

    Read the given path

  4. abstract def write(path: Path, actions: Iterator[String], overwrite: Boolean = false): Unit

    Write the given actions to the given path with or without overwrite as indicated.

    Write the given actions to the given path with or without overwrite as indicated. Implementation must throw java.nio.file.FileAlreadyExistsException exception if the file already exists and overwrite = false. Furthermore, implementation must ensure that the entire file is made visible atomically, that is, it should not generate partial files.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. def isPartialWriteVisible(path: Path): Boolean

    Whether a partial write is visible when writing to path.

    Whether a partial write is visible when writing to path.

    As this depends on the underlying file system implementations, we require the input of path here in order to identify the underlying file system, even though in most cases a log store only deals with one file system.

    The default value is only provided here for legacy reasons, which will be removed. Any LogStore implementation should override this instead of relying on the default.

  13. final def listFrom(path: String): Iterator[FileStatus]

    List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given path.

    List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given path. The result should also be sorted by the file name.

  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def read(path: String): Seq[String]

    Read the given path

  18. def resolvePathOnPhysicalStorage(path: Path): Path

    Resolve the fully qualified path for the given path.

  19. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  24. final def write(path: String, actions: Iterator[String]): Unit

    Write the given actions to the given path without overwriting any existing file.

    Write the given actions to the given path without overwriting any existing file. Implementation must throw java.nio.file.FileAlreadyExistsException exception if the file already exists. Furthermore, implementation must ensure that the entire file is made visible atomically, that is, it should not generate partial files.

Inherited from AnyRef

Inherited from Any

Ungrouped