c

ai.chronon.aggregator.windowing

SawtoothMutationAggregator

class SawtoothMutationAggregator extends SawtoothAggregator

Mutations processing starts with an end of the day snapshot FinalBatchIR. On top of this FinalBatchIR mutations are processed.

update/merge/finalize are related to snapshot data. As such they follow the snapshot Schema and aggregators. However mutations come into play later in the group by and a finalized version of the snapshot data is created to be processed with the mutations rows. Since the dataframe inputs are aligned between mutations and snapshot (input) no additional schema is needed.

Linear Supertypes
SawtoothAggregator, Serializable, Serializable, AnyRef, Any
Known Subclasses
SawtoothOnlineAggregator
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SawtoothMutationAggregator
  2. SawtoothAggregator
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SawtoothMutationAggregator(aggregations: Seq[Aggregation], inputSchema: Seq[(String, DataType)], resolution: Resolution = FiveMinuteResolution, tailBufferMillis: Long = new Window(2, TimeUnit.DAYS).millis)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. lazy val baseAggregator: RowAggregator
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  6. lazy val baseIrIndices: Array[Int]
    Attributes
    protected
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  7. def batchIrSchema: Array[(String, DataType)]
  8. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  9. def computeWindows(hops: OutputArrayType, endTimes: Array[Long]): Array[Array[Any]]
    Definition Classes
    SawtoothAggregator
  10. def cumulate(inputs: Iterator[Row], sortedEndTimes: Array[Long], baseIR: Array[Any]): Array[Array[Any]]
    Definition Classes
    SawtoothAggregator
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. def finalizeSnapshot(batchIr: BatchIr): FinalBatchIr
  15. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. val hopSizes: Array[Long]
    Attributes
    protected
    Definition Classes
    SawtoothAggregator
  18. val hopsAggregator: HopsAggregatorBase
  19. def init: BatchIr
  20. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  21. def lambdaAggregateIrMany(batchEndTs: Long, finalBatchIr: FinalBatchIr, sortedInputs: Array[Row], sortedEndTimes: Array[Long]): Array[Array[Any]]

    Given aggregations FinalBatchIRs at the end of the Snapshot (batchEndTs) and mutation and query times, determine the values at the query times for the aggregations.

    Given aggregations FinalBatchIRs at the end of the Snapshot (batchEndTs) and mutation and query times, determine the values at the query times for the aggregations. This is pretty much a mix of online with extra work for multiple queries ts support.

  22. def merge(batchIr1: BatchIr, batchIr2: BatchIr): BatchIr
  23. def mergeTailHops(ir: Array[Any], queryTs: Long, batchEndTs: Long, batchIr: FinalBatchIr): Array[Any]

    Update the intermediate results with tail hops data from a FinalBatchIr.

  24. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  25. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  26. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  27. lazy val perWindowAggs: Array[AggregationPart]
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  28. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  29. lazy val tailHopIndices: Array[Int]
    Attributes
    protected
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  30. def tailTs(batchEndTs: Long): Array[Option[Long]]
  31. def toString(): String
    Definition Classes
    AnyRef → Any
  32. lazy val unpackedAggs: UnpackedAggregations
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  33. def update(batchEndTs: Long, batchIr: BatchIr, row: Row, batchTails: Array[Option[Long]]): BatchIr
  34. def update(batchEndTs: Long, batchIr: BatchIr, row: Row): BatchIr
  35. def updateIr(ir: Array[Any], row: Row, queryTs: Long, hasReversal: Boolean = false): Unit

    Go through the aggregators and update or delete the intermediate with the information of the row if relevant.

    Go through the aggregators and update or delete the intermediate with the information of the row if relevant. Useful for both online and mutations

  36. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  37. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  39. lazy val windowMappings: Array[WindowMapping]
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()
  40. lazy val windowedAggregator: RowAggregator
    Definition Classes
    SawtoothAggregator
    Annotations
    @transient()

Inherited from SawtoothAggregator

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped