Type Members

1.  trait AnalysisHelper extends AnyRef
2.  class DatasetRefCache[T] extends AnyRef

   A Dataset reference cache to automatically create new Dataset objects when the active SparkSession changes.

   A Dataset reference cache to automatically create new Dataset objects when the active SparkSession changes. This is useful when sharing objects holding Dataset references cross multiple sessions. Without this, using a Dataset that holds a stale session may change the active session and cause multiple issues (e.g., if we switch to a stale session coming from a notebook that has been detached, we may not be able to use built-in functions because those are cleaned up).

   The creator function will be called to create a new Dataset object when the old one has a different session than the current active session.

   Unlike StateCache, this class only caches the Dataset reference and doesn't cache the underlying RDD.

3.  sealed trait DateFormatter extends Serializable

   Forked from org.apache.spark.sql.catalyst.util.DateFormatter

4.  trait DateTimeFormatterHelper extends AnyRef

   Forked from org.apache.spark.sql.catalyst.util.DateTimeFormatterHelper

5.  trait DeltaProgressReporter extends Logging
6.  class FractionTimestampFormatter extends Iso8601TimestampFormatter

   The formatter parses/formats timestamps according to the pattern yyyy-MM-dd HH:mm:ss.[..fff..] where [..fff..] is a fraction of second up to microsecond resolution.

   The formatter parses/formats timestamps according to the pattern yyyy-MM-dd HH:mm:ss.[..fff..] where [..fff..] is a fraction of second up to microsecond resolution. The formatter does not output trailing zeros in the fraction. For example, the timestamp 2019-03-05 15:00:01.123400 is formatted as the string 2019-03-05 15:00:01.1234.

7.  class Iso8601DateFormatter extends DateFormatter with DateTimeFormatterHelper
8.  class Iso8601TimestampFormatter extends TimestampFormatter with DateTimeFormatterHelper
9.  class PartitionPath extends AnyRef

   Holds a directory in a partitioned collection of files as well as the partition values in the form of a Row.

   Holds a directory in a partitioned collection of files as well as the partition values in the form of a Row. Before scanning, the files at path need to be enumerated.

10.  case class PartitionSpec(partitionColumns: StructType, partitions: Seq[PartitionPath]) extends Product with Serializable
11.  case class SerializableFileStatus(path: String, length: Long, isDir: Boolean, modificationTime: Long) extends Product with Serializable

    A serializable variant of HDFS's FileStatus.

12.  class SetAccumulator[T] extends AccumulatorV2[T, Set[T]]

    Accumulator to collect distinct elements as a set.

13.  trait StateCache extends AnyRef

    Machinary that caches the reconstructed state of a Delta table using the RDD cache.

    Machinary that caches the reconstructed state of a Delta table using the RDD cache. The cache is designed so that the first access will materialize the results. However once uncache is called, all data will be flushed and will not be cached again.

14.  sealed trait TimestampFormatter extends Serializable

    Forked from org.apache.spark.sql.catalyst.util.TimestampFormatter