Class

ai.chronon.aggregator.windowing

SawtoothMutationAggregator

Related Doc: package windowing

Permalink

class SawtoothMutationAggregator extends SawtoothAggregator

Mutations processing starts with an end of the day snapshot FinalBatchIR. On top of this FinalBatchIR mutations are processed.

update/merge/finalize are related to snapshot data. As such they follow the snapshot Schema and aggregators. However mutations come into play later in the group by and a finalized version of the snapshot data is created to be processed with the mutations rows. Since the dataframe inputs are aligned between mutations and snapshot (input) no additional schema is needed.

Linear Supertypes
SawtoothAggregator, Serializable, Serializable, AnyRef, Any
Known Subclasses
SawtoothOnlineAggregator
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SawtoothMutationAggregator
  2. SawtoothAggregator
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SawtoothMutationAggregator(aggregations: Seq[Aggregation], inputSchema: Seq[(String, DataType)], resolution: Resolution = FiveMinuteResolution, tailBufferMillis: Long = new Window(2, TimeUnit.DAYS).millis)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. lazy val baseAggregator: RowAggregator

    Permalink
    Definition Classes
    SawtoothAggregator

  6. lazy val baseIrIndices: Array[Int]

    Permalink
    Attributes
    protected
    Definition Classes
    SawtoothAggregator

  7. def batchIrSchema: Array[(String, DataType)]

    Permalink

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. def computeWindows(hops: OutputArrayType, endTimes: Array[Long]): Array[Array[Any]]

    Permalink
    Definition Classes
    SawtoothAggregator

  10. def cumulate(inputs: Iterator[Row], sortedEndTimes: Array[Long], baseIR: Array[Any]): Array[Array[Any]]

    Permalink
    Definition Classes
    SawtoothAggregator

  11. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  12. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  14. def finalizeSnapshot(batchIr: BatchIr): FinalBatchIr

    Permalink

  15. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  16. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  17. val hopSizes: Array[Long]

    Permalink
    Attributes
    protected
    Definition Classes
    SawtoothAggregator

  18. val hopsAggregator: HopsAggregatorBase

    Permalink

  19. def init: BatchIr

    Permalink

  20. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  21. def lambdaAggregateIrMany(batchEndTs: Long, finalBatchIr: FinalBatchIr, sortedInputs: Array[Row], sortedEndTimes: Array[Long]): Array[Array[Any]]

    Permalink

    Given aggregations FinalBatchIRs at the end of the Snapshot (batchEndTs) and mutation and query times, determine the values at the query times for the aggregations.

    Given aggregations FinalBatchIRs at the end of the Snapshot (batchEndTs) and mutation and query times, determine the values at the query times for the aggregations. This is pretty much a mix of online with extra work for multiple queries ts support.

  22. def merge(batchIr1: BatchIr, batchIr2: BatchIr): BatchIr

    Permalink

  23. def mergeTailHops(ir: Array[Any], queryTs: Long, batchEndTs: Long, batchIr: FinalBatchIr): Array[Any]

    Permalink

    Update the intermediate results with tail hops data from a FinalBatchIr.

  24. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  25. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  26. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  27. lazy val perWindowAggs: Array[AggregationPart]

    Permalink
    Definition Classes
    SawtoothAggregator

  28. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  29. lazy val tailHopIndices: Array[Int]

    Permalink
    Attributes
    protected
    Definition Classes
    SawtoothAggregator

  30. def tailTs(batchEndTs: Long): Array[Option[Long]]

    Permalink

  31. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  32. lazy val unpackedAggs: UnpackedAggregations

    Permalink
    Definition Classes
    SawtoothAggregator

  33. def update(batchEndTs: Long, batchIr: BatchIr, row: Row, batchTails: Array[Option[Long]]): BatchIr

    Permalink

  34. def update(batchEndTs: Long, batchIr: BatchIr, row: Row): BatchIr

    Permalink

  35. def updateIr(ir: Array[Any], row: Row, queryTs: Long, hasReversal: Boolean = false): Unit

    Permalink

    Go through the aggregators and update or delete the intermediate with the information of the row if relevant.

    Go through the aggregators and update or delete the intermediate with the information of the row if relevant. Useful for both online and mutations

  36. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  37. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  38. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  39. lazy val windowMappings: Array[WindowMapping]

    Permalink
    Definition Classes
    SawtoothAggregator

  40. lazy val windowedAggregator: RowAggregator

    Permalink
    Definition Classes
    SawtoothAggregator

Inherited from SawtoothAggregator

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped