Packages

c

com.nvidia.spark.rapids.shims

OffsetWindowFunctionMeta

abstract class OffsetWindowFunctionMeta[INPUT <: OffsetWindowFunction] extends ExprMeta[INPUT]

Spark 3.2-specific replacement for com.nvidia.spark.rapids.OffsetWindowFunctionMeta. This is required primarily for two reasons:

  1. com.nvidia.spark.rapids.OffsetWindowFunctionMeta (compiled against Spark 3.0.x) fails class load in Spark 3.1.x. (expr.input is not recognized as an Expression.) 2. The semantics of offsets in LAG() are reversed/negated in Spark 3.1.1. E.g. The expression LAG(col, 5) causes Lag.offset to be set to -5, as opposed to 5, in prior versions of Spark. 3. IGNORE NULL and RESPECT NULL was added in 3.2.0 and we need to fall back until this can be supported. This class adjusts the LAG offset to use similar semantics to Spark 3.0.x.
Linear Supertypes
ExprMeta[INPUT], BaseExprMeta[INPUT], RapidsMeta[INPUT, Expression, Expression], AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OffsetWindowFunctionMeta
  2. ExprMeta
  3. BaseExprMeta
  4. RapidsMeta
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new OffsetWindowFunctionMeta(expr: INPUT, conf: RapidsConf, parent: Option[RapidsMeta[_, _, _]], rule: DataFromReplacementRule)

Abstract Value Members

  1. abstract def convertToGpu(): GpuExpression

    Convert what this wraps to a GPU enabled version.

    Convert what this wraps to a GPU enabled version.

    Definition Classes
    ExprMeta → RapidsMeta

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def addConvertedDataType(expression: Expression, typeMeta: DataTypeMeta): Unit

    Call this method to record information about type conversions via DataTypeMeta.

    Call this method to record information about type conversions via DataTypeMeta.

    Definition Classes
    RapidsMeta
  5. lazy val adjustedOffset: Expression
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def canDataWriteCmdsBeReplaced: Boolean

    Returns true iff all of the data writing commands can be replaced.

    Returns true iff all of the data writing commands can be replaced.

    Definition Classes
    RapidsMeta
  8. def canExprTreeBeReplaced: Boolean

    Returns true iff all of the expressions and their children could be replaced.

    Returns true iff all of the expressions and their children could be replaced.

    Definition Classes
    BaseExprMeta → RapidsMeta
  9. def canPartsBeReplaced: Boolean

    Returns true iff all of the partitioning can be replaced.

    Returns true iff all of the partitioning can be replaced.

    Definition Classes
    RapidsMeta
  10. def canScansBeReplaced: Boolean

    Returns true iff all of the scans can be replaced.

    Returns true iff all of the scans can be replaced.

    Definition Classes
    RapidsMeta
  11. lazy val canSelfBeAst: Boolean

    Check whether this node itself can be converted to AST.

    Check whether this node itself can be converted to AST. It will not recursively check its children. It's used to check join condition AST-ability in top-down fashion.

    Definition Classes
    BaseExprMeta
  12. final def canThisBeAst: Boolean
    Definition Classes
    BaseExprMeta
  13. final def canThisBeReplaced: Boolean

    Returns true iff this could be replaced.

    Returns true iff this could be replaced.

    Definition Classes
    RapidsMeta
  14. var cannotBeReplacedReasons: Option[Set[String]]
    Attributes
    protected
    Definition Classes
    RapidsMeta
  15. var cannotRunOnGpuBecauseOfCost: Boolean
    Attributes
    protected
    Definition Classes
    RapidsMeta
  16. var cannotRunOnGpuBecauseOfSparkPlan: Boolean
    Attributes
    protected
    Definition Classes
    RapidsMeta
  17. def checkExprForTimezone(): Unit
    Definition Classes
    BaseExprMeta
  18. def checkTimeZoneId(sessionZoneId: ZoneId): Unit
    Definition Classes
    RapidsMeta
  19. def checkUTCTimezone(meta: RapidsMeta[_, _, _], zoneId: ZoneId): Unit

    Timezone check which only allows UTC timezone.

    Timezone check which only allows UTC timezone. This is consistent with previous behavior.

    meta

    to check whether it's UTC

    Definition Classes
    BaseExprMeta
  20. val childDataWriteCmds: Seq[DataWritingCommandMeta[_]]

    The wrapped data writing commands that should be examined

    The wrapped data writing commands that should be examined

    Definition Classes
    BaseExprMeta → RapidsMeta
  21. val childExprs: Seq[BaseExprMeta[_]]

    The wrapped expressions that should be examined

    The wrapped expressions that should be examined

    Definition Classes
    OffsetWindowFunctionMeta → BaseExprMeta → RapidsMeta
  22. val childParts: Seq[PartMeta[_]]

    The wrapped partitioning that should be examined

    The wrapped partitioning that should be examined

    Definition Classes
    BaseExprMeta → RapidsMeta
  23. val childPlans: Seq[SparkPlanMeta[_]]

    The wrapped plans that should be examined

    The wrapped plans that should be examined

    Definition Classes
    BaseExprMeta → RapidsMeta
  24. val childRunnableCmds: Seq[RunnableCommandMeta[_]]

    The wrapped runnable commands that should be examined.

    The wrapped runnable commands that should be examined.

    Definition Classes
    RapidsMeta
  25. val childScans: Seq[ScanMeta[_]]

    The wrapped scans that should be examined

    The wrapped scans that should be examined

    Definition Classes
    BaseExprMeta → RapidsMeta
  26. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  27. val conf: RapidsConf
    Definition Classes
    RapidsMeta
  28. def confKey: String
    Definition Classes
    RapidsMeta
  29. lazy val context: ExpressionContext
    Definition Classes
    BaseExprMeta
  30. def convertToCpu(): Expression

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled.

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled. By default this just returns what is wrapped by this. For some types of operators/stages, like SparkPlan, each part of the query can be converted independent of other parts. As such in a subclass this should be overridden to do the correct thing.

    Definition Classes
    RapidsMeta
  31. final def costPreventsRunningOnGpu(): Unit
    Definition Classes
    RapidsMeta
  32. def dataType: DataType
    Definition Classes
    BaseExprMeta
  33. lazy val default: BaseExprMeta[_]
  34. final val disabledMsg: Option[String]
    Definition Classes
    RapidsMeta
  35. final def entirePlanExcludedReasons: Set[String]

    Returns the list of reasons the entire plan can't be replaced.

    Returns the list of reasons the entire plan can't be replaced. An empty set means the entire plan is ok to be replaced, do the normal checking per exec and children.

    Definition Classes
    RapidsMeta
  36. final def entirePlanWillNotWork(because: String): Unit

    Call this if there is a condition found that the entire plan is not allowed to run on the GPU.

    Call this if there is a condition found that the entire plan is not allowed to run on the GPU.

    Definition Classes
    RapidsMeta
  37. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  38. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  39. def explain(all: Boolean): String
    Definition Classes
    RapidsMeta
  40. def explainAst(all: Boolean): String
    Definition Classes
    BaseExprMeta
  41. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  42. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  43. def getZoneId(): ZoneId
    Attributes
    protected
    Definition Classes
    BaseExprMeta
  44. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  45. val ignoreUnsetDataTypes: Boolean
    Definition Classes
    BaseExprMeta
  46. final val incompatDoc: Option[String]
    Definition Classes
    RapidsMeta
  47. def indent(append: StringBuilder, depth: Int): Unit
    Attributes
    protected
    Definition Classes
    RapidsMeta
  48. def initReasons(): Unit
    Definition Classes
    RapidsMeta
  49. lazy val input: BaseExprMeta[_]
  50. def isDisabledByDefault: Boolean
    Definition Classes
    RapidsMeta
  51. val isFoldableNonLitAllowed: Boolean
    Definition Classes
    BaseExprMeta
  52. def isIncompat: Boolean
    Definition Classes
    RapidsMeta
  53. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  54. def isTimeZoneSupported: Boolean
    Definition Classes
    BaseExprMeta
  55. final def mustBeReplaced(because: String): Unit
    Definition Classes
    RapidsMeta
  56. final def mustThisBeReplaced: Boolean

    Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.

    Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.

    Definition Classes
    RapidsMeta
  57. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  58. lazy val needTimeZoneCheck: Boolean
    Definition Classes
    BaseExprMeta
  59. def noReplacementPossibleMessage(reasons: String): String
    Definition Classes
    RapidsMeta
  60. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  61. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  62. lazy val offset: BaseExprMeta[_]
  63. final val operationName: String
    Definition Classes
    RapidsMeta
  64. def overrideDataType(dt: DataType): Unit

    Overrides the data type of the wrapped expression during type checking.

    Overrides the data type of the wrapped expression during type checking.

    NOTICE: This method will NOT modify the wrapped expression itself. Therefore, the actual transition on data type is still necessary when converting this expression to GPU.

    Definition Classes
    BaseExprMeta
  65. val parent: Option[RapidsMeta[_, _, _]]
    Definition Classes
    RapidsMeta
  66. def print(strBuilder: StringBuilder, depth: Int, all: Boolean): Unit

    Create a string representation of this in append.

    Create a string representation of this in append.

    strBuilder

    where to place the string representation.

    depth

    how far down the tree this is.

    all

    should all the data be printed or just what does not work on the GPU?

    Definition Classes
    RapidsMeta
  67. def printAst(strBuilder: StringBuilder, depth: Int, all: Boolean): Unit

    Create a string explanation for whether this expression tree can be converted to an AST

    Create a string explanation for whether this expression tree can be converted to an AST

    strBuilder

    where to place the string representation.

    depth

    how far down the tree this is.

    all

    should all the data be printed or just what does not work in the AST?

    Attributes
    protected
    Definition Classes
    BaseExprMeta
  68. val printWrapped: Boolean
    Definition Classes
    BaseExprMeta → RapidsMeta
  69. final def recursiveCostPreventsRunningOnGpu(): Unit

    Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.

    Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.

    Definition Classes
    RapidsMeta
  70. final def recursiveSparkPlanPreventsRunningOnGpu(): Unit
    Definition Classes
    RapidsMeta
  71. final def recursiveSparkPlanRemoved(): Unit
    Definition Classes
    RapidsMeta
  72. def replaceMessage: String
    Definition Classes
    RapidsMeta
  73. final def requireAstForGpu(): Unit
    Definition Classes
    BaseExprMeta
  74. final def shouldBeRemoved(because: String): Unit
    Definition Classes
    RapidsMeta
  75. final def shouldThisBeRemoved: Boolean

    Returns true if this node should be removed.

    Returns true if this node should be removed.

    Definition Classes
    RapidsMeta
  76. def supportsColumnar: Boolean

    Return true if the resulting node in the plan will support columnar execution

    Return true if the resulting node in the plan will support columnar execution

    Definition Classes
    RapidsMeta
  77. def suppressWillWorkOnGpuInfo: Boolean
    Definition Classes
    RapidsMeta
  78. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  79. def tagExprForGpu(): Unit

    Called to verify that this expression will work on the GPU.

    Called to verify that this expression will work on the GPU. For most expressions without extra checks all of the checks should have already been done.

    Definition Classes
    OffsetWindowFunctionMeta → BaseExprMeta
  80. final def tagForGpu(): Unit

    Tag all of the children to see if they are GPU compatible first.

    Tag all of the children to see if they are GPU compatible first. Do basic common verification for the operators, and then call tagSelfForGpu

    Definition Classes
    RapidsMeta
  81. def tagSelfForAst(): Unit

    Called to verify that this expression will work as a GPU AST expression.

    Called to verify that this expression will work as a GPU AST expression.

    Attributes
    protected
    Definition Classes
    BaseExprMeta
  82. final def tagSelfForGpu(): Unit

    Do any extra checks and tag yourself if you are compatible or not.

    Do any extra checks and tag yourself if you are compatible or not. Be aware that this may already have been marked as incompatible for a number of reasons.

    All of your children should have already been tagged so if there are situations where you may need to disqualify your children for various reasons you may do it here too.

    Definition Classes
    BaseExprMeta → RapidsMeta
  83. def toString(): String
    Definition Classes
    RapidsMeta → AnyRef → Any
  84. def typeMeta: DataTypeMeta

    Gets the DataTypeMeta of current BaseExprMeta, which is supposed to be called in the tag methods of expression-level type checks.

    Gets the DataTypeMeta of current BaseExprMeta, which is supposed to be called in the tag methods of expression-level type checks.

    By default, it simply returns the data type of wrapped expression. But for specific expressions, they can easily override data type for type checking through calling the method overrideDataType.

    Definition Classes
    BaseExprMeta
  85. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  86. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  87. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  88. final def willNotWorkInAst(because: String): Unit
    Definition Classes
    BaseExprMeta
  89. final def willNotWorkOnGpu(because: String): Unit

    Call this to indicate that this should not be replaced with a GPU enabled version

    Call this to indicate that this should not be replaced with a GPU enabled version

    because

    why it should not be replaced.

    Definition Classes
    RapidsMeta
  90. def willWorkInAstInfo: String
    Attributes
    protected
    Definition Classes
    BaseExprMeta
  91. val wrapped: INPUT
    Definition Classes
    RapidsMeta

Inherited from ExprMeta[INPUT]

Inherited from BaseExprMeta[INPUT]

Inherited from RapidsMeta[INPUT, Expression, Expression]

Inherited from AnyRef

Inherited from Any

Ungrouped