Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark
    Definition Classes
    apache
  • package sql

    Allows the execution of relational queries, including those expressed in SQL using Spark.

    Allows the execution of relational queries, including those expressed in SQL using Spark.

    Definition Classes
    spark
  • package execution

    The physical execution component of Spark SQL.

    The physical execution component of Spark SQL. Note that this is a private package. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

    Definition Classes
    sql
  • package streaming
    Definition Classes
    execution
  • trait FlatMapGroupsWithStateExecBase extends SparkPlan with StateStoreWriter with WatermarkSupport

    Physical operator for executing FlatMapGroupsWithState

    Physical operator for executing FlatMapGroupsWithState

    Definition Classes
    streaming
  • InputProcessor

abstract class InputProcessor extends AnyRef

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. InputProcessor
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new InputProcessor(store: StateStore)

Abstract Value Members

  1. abstract def callFunctionAndUpdateState(stateData: StateData, valueRowIter: Iterator[InternalRow], hasTimedOut: Boolean): Iterator[InternalRow]

    Call the user function on a key's data, update the state store, and return the return data iterator.

    Call the user function on a key's data, update the state store, and return the return data iterator. Note that the store updating is lazy, that is, the store will be updated only after the returned iterator is fully consumed.

    stateData

    All the data related to the state to be updated

    valueRowIter

    Iterator of values as rows, cannot be null, but can be empty

    hasTimedOut

    Whether this function is being called for a key timeout

    Attributes
    protected

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  9. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. val numOutputRows: SQLMetric
    Attributes
    protected
  16. val numRemovedStateRows: SQLMetric
    Attributes
    protected
  17. val numUpdatedStateRows: SQLMetric
    Attributes
    protected
  18. def processNewData(dataIter: Iterator[InternalRow]): Iterator[InternalRow]

    For every group, get the key, values and corresponding state and call the function, and return an iterator of rows

  19. def processNewDataWithInitialState(childDataIter: Iterator[InternalRow], initStateIter: Iterator[InternalRow]): Iterator[InternalRow]

    Process the new data iterator along with the initial state.

    Process the new data iterator along with the initial state. The initial state is applied before processing the new data for every key. The user defined function is called only once for every key that has either initial state or data or both.

  20. def processTimedOutState(): Iterator[InternalRow]

    Find the groups that have timeout set and are timing out right now, and call the function

  21. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  22. def toString(): String
    Definition Classes
    AnyRef → Any
  23. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  24. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  25. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped