object ParquetPartitioningFlow
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- ParquetPartitioningFlow
- AnyRef
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Type Members
- trait Builder[T, W, Self] extends AnyRef
Builds an instance of ParquetPartitioningFlow
Builds an instance of ParquetPartitioningFlow
- T
Type of message that flow accepts
- W
Schema of Parquet file that flow writes
- trait GenericBuilder extends Builder[RowParquetRecord, RowParquetRecord, GenericBuilder]
- case class PostWriteState[T](processedData: T, modifiedPartitions: Map[Path, Long], flush: (Path) => Unit) extends Product with Serializable
Represent the state of writer after processing of
processedData.Represent the state of writer after processing of
processedData.- T
type of input data
- processedData
Processed input element
- modifiedPartitions
State of partitions that has been written in effect of processing the element T. More than one partition can be modified due to preWriteTransformation. The map contains values representing total number of writes to a single file (number of writes to the partition after last rotation).
- flush
Flushes all writes to given partition and rotates the file.
- trait TypedBuilder[T, W] extends Builder[T, W, TypedBuilder[T, W]]
- trait ViaParquet extends AnyRef
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val DefaultMaxCount: Long
- val DefaultMaxDuration: FiniteDuration
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()