Packages

t

org.apache.spark.sql.delta.storage.dv

DeletionVectorStoreUtils

trait DeletionVectorStoreUtils extends AnyRef

Trait containing the utility and constants needed for DeletionVectorStore

Linear Supertypes
AnyRef, Any
Known Subclasses
DeletionVectorStore
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeletionVectorStoreUtils
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class DVRangeDescriptor(offset: Int, length: Int, checksum: Int) extends Product with Serializable

    Descriptor for a serialized Deletion Vector in a file.

  2. trait Writer extends Closeable

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final val CHECKSUM_LEN: Int(4)

    The length of a DV checksum.

    The length of a DV checksum. See calculateChecksum().

  5. final val DATA_SIZE_LEN: Int(4)

    The size of the stored length of a DV.

  6. final val DV_FILE_FORMAT_VERSION_ID_V1: Byte
  7. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  8. def assembleDeletionVectorPathWithFileSystem(targetParentPathWithFileSystem: PathWithFileSystem, id: UUID, prefix: String = ""): PathWithFileSystem

    Same as assembleDeletionVectorPath, but keeps the new path bundled with the fs.

  9. def calculateChecksum(data: Array[Byte]): Int

    Calculate checksum of a serialized deletion vector.

    Calculate checksum of a serialized deletion vector. We are using CRC32 which has 4bytes size, but CRC32 implementation conforms to Java Checksum interface which requires a long. However, the high-order bytes are zero, so here is safe to cast to Int. This will result in negative checksums, but this is not a problem because we only care about equality.

  10. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def escapedStringToPath(path: String): Path

    Convert the given String path to a Hadoop Path, Please make sure the path is escaped.

  14. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. def getTotalSizeOfDVFieldsInFile(bitmapDataSize: Int): Int
  17. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. def pathToEscapedString(path: Path): String

    Convert the given Hadoop path to a String Path, handing special characters properly.

  23. def readRangeFromStream(reader: DataInputStream, size: Int): Array[Byte]

    Read a serialized deletion vector from a data stream.

  24. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. def unescapedStringToPath(path: String): Path

    Convert the given String path to a Hadoop Path.

    Convert the given String path to a Hadoop Path. Please make sure the path is not escaped.

  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped