Packages

package rapids

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. trait BasePad extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with NullIntolerant
  2. class BasicColumnarWriteJobStatsTracker extends ColumnarWriteJobStatsTracker

    Simple ColumnarWriteJobStatsTracker implementation that's serializable, capable ofinstantiating BasicColumnarWriteTaskStatsTracker on executors and processing the BasicColumnarWriteTaskStats they produce by aggregating the metrics and posting them as DriverMetricUpdates.

  3. case class BasicColumnarWriteTaskStats(numPartitions: Int, numFiles: Int, numBytes: Long, numRows: Long) extends WriteTaskStats with Product with Serializable

    Simple metrics collected during an instance of GpuFileFormatDataWriter.

    Simple metrics collected during an instance of GpuFileFormatDataWriter. These were first introduced in https://github.com/apache/spark/pull/18159 (SPARK-20703).

  4. class BasicColumnarWriteTaskStatsTracker extends ColumnarWriteTaskStatsTracker with Logging

    Simple metrics collected during an instance of GpuFileFormatDataWriter.

    Simple metrics collected during an instance of GpuFileFormatDataWriter. This is the columnar version of org.apache.spark.sql.execution.datasources.BasicWriteTaskStatsTracker.

  5. trait ColumnarWriteJobStatsTracker extends Serializable

    A class implementing this trait is basically a collection of parameters that are necessary for instantiating a (derived type of) ColumnarWriteTaskStatsTracker on all executors and then process the statistics produced by them (e.g.

    A class implementing this trait is basically a collection of parameters that are necessary for instantiating a (derived type of) ColumnarWriteTaskStatsTracker on all executors and then process the statistics produced by them (e.g. save them to memory/disk, issue warnings, etc). It is therefore important that such an objects is Serializable, as it will be sent from the driver to all executors.

  6. trait ColumnarWriteTaskStatsTracker extends AnyRef

    A trait for classes that are capable of collecting statistics on columnar data that's being processed by a single write task in GpuFileFormatDataWriter - i.e.

    A trait for classes that are capable of collecting statistics on columnar data that's being processed by a single write task in GpuFileFormatDataWriter - i.e. there should be one instance per executor.

    This trait is coupled with the way GpuFileFormatWriter works, in the sense that its methods will be called according to how column batches are being written out to disk, namely in sorted order according to partitionValue(s), then bucketId.

    As such, a typical call scenario is:

    newPartition -> newBucket -> newFile -> newRow -. ^ |^_ ^| | | || | || ||

    newPartition and newBucket events are only triggered if the relation to be written out is partitioned and/or bucketed, respectively.

  7. abstract case class CudfAggregate(ref: Expression) extends Expression with GpuUnevaluable with Product with Serializable
  8. abstract class CudfBinaryArithmetic extends CudfBinaryOperator with NullIntolerant
  9. abstract class CudfBinaryComparison extends CudfBinaryOperator with Predicate
  10. abstract class CudfBinaryMathExpression extends BinaryExpression with CudfBinaryExpression with Serializable with ImplicitCastInputTypes
  11. class CudfCount extends CudfAggregate
  12. class CudfFirstExcludeNulls extends CudfFirstLastBase
  13. class CudfFirstIncludeNulls extends CudfFirstLastBase
  14. abstract class CudfFirstLastBase extends CudfAggregate
  15. class CudfLastExcludeNulls extends CudfFirstLastBase
  16. class CudfLastIncludeNulls extends CudfFirstLastBase
  17. class CudfMax extends CudfAggregate
  18. class CudfMin extends CudfAggregate
  19. class CudfSum extends CudfAggregate
  20. abstract class CudfUnaryMathExpression extends GpuUnaryMathExpression with CudfUnaryExpression
  21. case class GpuAbs(child: Expression) extends GpuUnaryExpression with CudfUnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
  22. case class GpuAcos(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  23. case class GpuAcoshCompat(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  24. case class GpuAcoshImproved(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  25. case class GpuAdd(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  26. case class GpuAggregateExpression(origAggregateFunction: GpuAggregateFunction, mode: AggregateMode, isDistinct: Boolean, filter: Option[Expression], resultId: ExprId) extends Expression with GpuExpression with GpuUnevaluable with Product with Serializable
  27. trait GpuAggregateFunction extends Expression with GpuExpression with GpuUnevaluable
  28. case class GpuAnd(left: Expression, right: Expression) extends CudfBinaryOperator with Predicate with Product with Serializable
  29. case class GpuArrayContains(left: Expression, right: Expression) extends BinaryExpression with GpuBinaryExpression with NullIntolerant with Product with Serializable

    Checks if the array (left) has the element (right)

  30. case class GpuAsin(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  31. case class GpuAsinhCompat(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  32. case class GpuAsinhImproved(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  33. case class GpuAtan(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  34. case class GpuAtanh(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  35. case class GpuAverage(child: Expression) extends Expression with GpuAggregateFunction with GpuAggregateWindowFunction[MeanAggregation] with Product with Serializable
  36. case class GpuBRound(child: Expression, scale: Expression) extends GpuRoundBase with Product with Serializable
  37. case class GpuBitwiseAnd(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  38. case class GpuBitwiseNot(child: Expression) extends GpuUnaryExpression with CudfUnaryExpression with ExpectsInputTypes with Product with Serializable
  39. case class GpuBitwiseOr(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  40. case class GpuBitwiseXor(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  41. class GpuCartesianPartition extends Partition
  42. case class GpuCartesianProductExec(left: SparkPlan, right: SparkPlan, condition: Option[Expression], targetSizeBytes: Long) extends SparkPlan with BinaryExecNode with GpuExec with Product with Serializable
  43. class GpuCartesianRDD extends RDD[ColumnarBatch] with Serializable with Arm
  44. case class GpuCbrt(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  45. case class GpuCeil(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  46. trait GpuCollectBase[T <: Aggregation with RollingAggregation[T]] extends Expression with GpuAggregateFunction with GpuAggregateWindowFunction[T]
  47. case class GpuCollectList(childExpression: Expression, mutableAggBufferOffset: Int = 0, inputAggBufferOffset: Int = 0) extends Expression with GpuCollectBase[CollectListAggregation] with Product with Serializable

    Collects and returns a list of non-unique elements.

    Collects and returns a list of non-unique elements.

    The two 'offset' parameters are not used by GPU version, but are here for the compatibility with the CPU version and automated checks.

  48. case class GpuCollectSet(childExpression: Expression, mutableAggBufferOffset: Int = 0, inputAggBufferOffset: Int = 0) extends Expression with GpuCollectBase[CollectSetAggregation] with Product with Serializable

    Collects and returns a set of unique elements.

    Collects and returns a set of unique elements.

    The two 'offset' parameters are not used by GPU version, but are here for the compatibility with the CPU version and automated checks.

  49. case class GpuConcat(children: Seq[Expression]) extends Expression with GpuComplexTypeMergingExpression with Product with Serializable
  50. case class GpuConcatWs(children: Seq[Expression]) extends Expression with GpuExpression with ImplicitCastInputTypes with Product with Serializable
  51. case class GpuContains(left: Expression, right: Expression) extends BinaryExpression with GpuBinaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  52. case class GpuCos(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  53. case class GpuCosh(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  54. case class GpuCot(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  55. case class GpuCount(children: Seq[Expression]) extends Expression with GpuAggregateFunction with GpuBatchedRunningWindowWithFixer with GpuAggregateWindowFunction[CountAggregation] with GpuRunningWindowFunction with Product with Serializable
  56. case class GpuCreateArray(children: Seq[Expression], useStringTypeWhenEmpty: Boolean) extends Expression with GpuExpression with Product with Serializable
  57. case class GpuCreateDataSourceTableAsSelectCommand(table: CatalogTable, mode: SaveMode, query: LogicalPlan, outputColumnNames: Seq[String], origProvider: Class[_], gpuFileFormat: ColumnarFileFormat) extends LogicalPlan with GpuDataWritingCommand with Product with Serializable
  58. case class GpuCreateNamedStruct(children: Seq[Expression]) extends Expression with GpuExpression with Product with Serializable
  59. case class GpuDataSource(sparkSession: SparkSession, className: String, paths: Seq[String] = Nil, userSpecifiedSchema: Option[StructType] = None, partitionColumns: Seq[String] = Seq.empty, bucketSpec: Option[BucketSpec] = None, options: Map[String, String] = Map.empty, catalogTable: Option[CatalogTable] = None, origProvider: Class[_], gpuFileFormat: ColumnarFileFormat) extends Logging with Product with Serializable

    A truncated version of Spark DataSource that converts to use the GPU version of InsertIntoHadoopFsRelationCommand for FileFormats we support.

    A truncated version of Spark DataSource that converts to use the GPU version of InsertIntoHadoopFsRelationCommand for FileFormats we support. This does not support DataSource V2 writing at this point because at the time of copying, it did not.

  60. trait GpuDataSourceScanExec extends SparkPlan with LeafExecNode

    GPU implementation of Spark's DataSourceScanExec

  61. case class GpuDateAdd(startDate: Expression, days: Expression) extends BinaryExpression with GpuDateMathBase with Product with Serializable
  62. case class GpuDateAddInterval(start: Expression, interval: Expression, timeZoneId: Option[String] = None) extends GpuTimeMath with Product with Serializable
  63. case class GpuDateDiff(endDate: Expression, startDate: Expression) extends BinaryExpression with GpuBinaryExpression with ImplicitCastInputTypes with Product with Serializable
  64. case class GpuDateFormatClass(timestamp: Expression, format: Expression, strfFormat: String, timeZoneId: Option[String] = None) extends BinaryExpression with GpuBinaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with Product with Serializable
  65. trait GpuDateMathBase extends BinaryExpression with GpuBinaryExpression with ExpectsInputTypes
  66. case class GpuDateSub(startDate: Expression, days: Expression) extends BinaryExpression with GpuDateMathBase with Product with Serializable
  67. trait GpuDateUnaryExpression extends GpuUnaryExpression with ImplicitCastInputTypes
  68. case class GpuDayOfMonth(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  69. case class GpuDayOfWeek(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  70. case class GpuDayOfYear(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  71. trait GpuDivModLike extends CudfBinaryArithmetic
  72. case class GpuDivide(left: Expression, right: Expression, failOnErrorOverride: Boolean = ...) extends CudfBinaryArithmetic with GpuDivModLike with Product with Serializable
  73. class GpuDynamicPartitionDataWriter extends GpuFileFormatDataWriter

    Writes data to using dynamic partition writes, meaning this single function can write to multiple directories (partitions) or files (bucketing).

  74. case class GpuElementAt(left: Expression, right: Expression, failOnError: Boolean) extends BinaryExpression with GpuBinaryExpression with ExpectsInputTypes with Product with Serializable
  75. class GpuEmptyDirectoryDataWriter extends GpuFileFormatDataWriter

    GPU data writer for empty partitions

  76. case class GpuEndsWith(left: Expression, right: Expression) extends BinaryExpression with GpuBinaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  77. case class GpuEqualNullSafe(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable
  78. case class GpuEqualTo(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable

    The table below shows how the result is calculated for Equal-to.

    The table below shows how the result is calculated for Equal-to. To make calculation easier we are leveraging the fact that the cudf-result(r) always returns false. So that result is used in place of false when needed.

    Return (lhs.nan && rhs.nan) || result[i]

    +-------------+------------+------------------+---------------+----+ | lhs.isNan()| rhs.isNan | cudf-result(r) | final-result | eq | +-------------+------------+------------------+---------------+----+ | t | f | f | r | f | | f | t | f | r | f | | t | t | f | t | t | | f | f | r | r | na | +-------------+------------+------------------+---------------+----+

  79. case class GpuExp(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  80. case class GpuExpm1(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  81. abstract class GpuFileFormatDataWriter extends DataWriter[ColumnarBatch]

    Abstract class for writing out data in a single Spark task using the GPU.

    Abstract class for writing out data in a single Spark task using the GPU. This is the GPU version of org.apache.spark.sql.execution.datasources.FileFormatDataWriter.

  82. case class GpuFileSourceScanExec(relation: HadoopFsRelation, output: Seq[Attribute], requiredSchema: StructType, partitionFilters: Seq[Expression], optionalBucketSet: Option[BitSet], optionalNumCoalescedBuckets: Option[Int], dataFilters: Seq[Expression], tableIdentifier: Option[TableIdentifier], queryUsesInputFile: Boolean = false)(rapidsConf: RapidsConf) extends SparkPlan with GpuDataSourceScanExec with GpuExec with Product with Serializable

    GPU version of Spark's FileSourceScanExec

    GPU version of Spark's FileSourceScanExec

    relation

    The file-based relation to scan.

    output

    Output attributes of the scan, including data attributes and partition attributes.

    requiredSchema

    Required schema of the underlying relation, excluding partition columns.

    partitionFilters

    Predicates to use for partition pruning.

    optionalBucketSet

    Bucket ids for bucket pruning.

    optionalNumCoalescedBuckets

    Number of coalesced buckets.

    dataFilters

    Filters on non-partition columns.

    tableIdentifier

    identifier for the table in the metastore.

    queryUsesInputFile

    This is a parameter to easily allow turning it off in GpuTransitionOverrides if InputFileName, InputFileBlockStart, or InputFileBlockLength are used

    rapidsConf

    Rapids conf

  83. case class GpuFirst(child: Expression, ignoreNulls: Boolean) extends Expression with GpuAggregateFunction with ImplicitCastInputTypes with Serializable with Product
  84. case class GpuFloor(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  85. case class GpuFromUnixTime(sec: Expression, format: Expression, strfFormat: String, timeZoneId: Option[String] = None) extends BinaryExpression with GpuBinaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with Product with Serializable
  86. case class GpuGetArrayItem(child: Expression, ordinal: Expression, failOnError: Boolean) extends BinaryExpression with GpuBinaryExpression with ExpectsInputTypes with ExtractValue with Product with Serializable

    Returns the field at ordinal in the Array child.

    Returns the field at ordinal in the Array child.

    We need to do type checking here as ordinal expression maybe unresolved.

  87. class GpuGetArrayItemMeta extends BinaryExprMeta[GetArrayItem]
  88. case class GpuGetMapValue(child: Expression, key: Expression, failOnError: Boolean) extends BinaryExpression with GpuBinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  89. class GpuGetMapValueMeta extends BinaryExprMeta[GetMapValue]
  90. case class GpuGetStructField(child: Expression, ordinal: Int, name: Option[String] = None) extends UnaryExpression with GpuExpression with ExtractValue with NullIntolerant with Product with Serializable
  91. case class GpuGetTimestamp(strTs: Expression, format: Expression, sparkFormat: String, strf: String, timeZoneId: Option[String] = None) extends GpuToTimestamp with Product with Serializable
  92. case class GpuGreaterThan(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable

    The table below shows how the result is calculated for greater-than.

    The table below shows how the result is calculated for greater-than. To make calculation easier we are leveraging the fact that the cudf-result(r) always returns false. So that result is used in place of false when needed.

    In this case return (lhs.nan && !lhs.nan) || result[i]

    +-------------+------------+-----------------+---------------+----+ | lhs.isNan()| rhs.isNan | cudf-result(r) | final-result | gt | +-------------+------------+-----------------+---------------+----+ | t | f | f | t | t | | f | t | f | r | f | | t | t | f | r | f | | f | f | r | r | na | +-------------+------------+-----------------+---------------+----+

  93. case class GpuGreaterThanOrEqual(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable

    The table below shows how the result is calculated for Greater-than-Eq.

    The table below shows how the result is calculated for Greater-than-Eq. To make calculation easier we are leveraging the fact that the cudf-result(r) always returns false. So that result is used in place of false when needed.

    In this case return lhs.isNan || result[i]

    +-------------+------------+-----------------+---------------+-----+ | lhs.isNan()| rhs.isNan | cudf-result(r) | final-result | gte | +-------------+------------+-----------------+---------------+-----+ | t | f | f | t | t | | f | t | f | r | f | | t | t | f | t | t | | f | f | r | r | NA | +-------------+------------+-----------------+---------------+-----+

  94. case class GpuGreatest(children: Seq[Expression]) extends Expression with GpuGreatestLeastBase with Product with Serializable
  95. trait GpuGreatestLeastBase extends Expression with ComplexTypeMergingExpression with GpuExpression
  96. case class GpuHour(child: Expression, timeZoneId: Option[String] = None) extends GpuUnaryExpression with GpuTimeUnaryExpression with Product with Serializable
  97. case class GpuInitCap(child: Expression) extends GpuUnaryExpression with ImplicitCastInputTypes with Product with Serializable
  98. case class GpuInputFileBlockLength() extends GpuLeafExpression with Product with Serializable

    Returns the length of the block being read, or -1 if not available.

    Returns the length of the block being read, or -1 if not available. This is extra difficult because we cannot coalesce batches in between when this is used and the input file or else we could run into problems with returning the wrong thing.

  99. case class GpuInputFileBlockStart() extends GpuLeafExpression with Product with Serializable

    Returns the start offset of the block being read, or -1 if not available.

    Returns the start offset of the block being read, or -1 if not available. This is extra difficult because we cannot coalesce batches in between when this is used and the input file or else we could run into problems with returning the wrong thing.

  100. case class GpuInputFileName() extends GpuLeafExpression with Product with Serializable

    Returns the name of the file being read, or empty string if not available.

    Returns the name of the file being read, or empty string if not available. This is extra difficult because we cannot coalesce batches in between when this is used and the input file or else we could run into problems with returning the wrong thing.

  101. case class GpuInsertIntoHadoopFsRelationCommand(outputPath: Path, staticPartitions: TablePartitionSpec, ifPartitionNotExists: Boolean, partitionColumns: Seq[Attribute], bucketSpec: Option[BucketSpec], fileFormat: ColumnarFileFormat, options: Map[String, String], query: LogicalPlan, mode: SaveMode, catalogTable: Option[CatalogTable], fileIndex: Option[FileIndex], outputColumnNames: Seq[String]) extends LogicalPlan with GpuDataWritingCommand with Product with Serializable
  102. case class GpuIntegralDivide(left: Expression, right: Expression) extends CudfBinaryArithmetic with GpuDivModLike with Product with Serializable
  103. case class GpuLast(child: Expression, ignoreNulls: Boolean) extends Expression with GpuAggregateFunction with ImplicitCastInputTypes with Serializable with Product
  104. case class GpuLastDay(startDate: Expression) extends GpuUnaryExpression with ImplicitCastInputTypes with Product with Serializable
  105. case class GpuLeast(children: Seq[Expression]) extends Expression with GpuGreatestLeastBase with Product with Serializable
  106. case class GpuLength(child: Expression) extends GpuUnaryExpression with ExpectsInputTypes with Product with Serializable
  107. case class GpuLessThan(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable

    The table below shows how the result is calculated for Less-than.

    The table below shows how the result is calculated for Less-than. To make calculation easier we are leveraging the fact that the cudf-result(r) always returns false. So that result is used in place of false when needed.

    In this case return !lhs.nan && rhs.nan || result[i]

    +-------------+------------+-----------------+---------------+-----+ | lhs.isNan()| rhs.isNan | cudf-result(r) | final-result | lt | +-------------+------------+-----------------+---------------+-----+ | t | f | f | r | f | | f | t | f | t | t | | t | t | f | r | f | | f | f | r | r | NA | +-------------+------------+-----------------+---------------+-----+

  108. case class GpuLessThanOrEqual(left: Expression, right: Expression) extends CudfBinaryComparison with NullIntolerant with Product with Serializable

    The table below shows how the result is calculated for Less-than-Eq.

    The table below shows how the result is calculated for Less-than-Eq. To make calculation easier we are leveraging the fact that the cudf-result(r) always returns false. So that result is used in place of false when needed.

    In this case, return rhs.nan || result[i]

    +-------------+------------+------------------+---------------+-----+ | lhs.isNan()| rhs.isNan | cudf-result(r) | final-result | lte | +-------------+------------+------------------+---------------+-----+ | t | f | f | r | f | | f | t | f | t | t | | t | t | f | t | t | | f | f | r | r | NA | +-------------+------------+------------------+---------------+-----+

  109. case class GpuLike(left: Expression, right: Expression, escapeChar: Char) extends BinaryExpression with GpuBinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  110. case class GpuLog(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  111. case class GpuLogarithm(left: Expression, right: Expression) extends CudfBinaryMathExpression with Product with Serializable
  112. case class GpuLower(child: Expression) extends GpuUnaryString2StringExpression with Product with Serializable
  113. case class GpuMax(child: Expression) extends Expression with GpuAggregateFunction with GpuBatchedRunningWindowWithFixer with GpuAggregateWindowFunction[MaxAggregation] with GpuRunningWindowFunction with Product with Serializable
  114. case class GpuMd5(child: Expression) extends GpuUnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  115. case class GpuMin(child: Expression) extends Expression with GpuAggregateFunction with GpuBatchedRunningWindowWithFixer with GpuAggregateWindowFunction[MinAggregation] with GpuRunningWindowFunction with Product with Serializable
  116. case class GpuMinute(child: Expression, timeZoneId: Option[String] = None) extends GpuUnaryExpression with GpuTimeUnaryExpression with Product with Serializable
  117. case class GpuMonth(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  118. case class GpuMultiply(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  119. case class GpuMurmur3Hash(children: Seq[Expression], seed: Int) extends Expression with GpuExpression with Product with Serializable
  120. case class GpuNormalizeNaNAndZero(child: Expression) extends GpuUnaryExpression with ExpectsInputTypes with Product with Serializable
  121. case class GpuNot(child: Expression) extends GpuUnaryExpression with CudfUnaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  122. case class GpuOr(left: Expression, right: Expression) extends CudfBinaryOperator with Predicate with Product with Serializable
  123. class GpuOrcFileFormat extends ColumnarFileFormat with Logging
  124. class GpuOrcWriter extends ColumnarOutputWriter
  125. case class GpuPivotFirst(pivotColumn: Expression, valueColumn: Expression, pivotColumnValues: Seq[Any]) extends Expression with GpuAggregateFunction with Product with Serializable
  126. case class GpuPmod(left: Expression, right: Expression) extends CudfBinaryArithmetic with GpuDivModLike with Product with Serializable
  127. case class GpuPow(left: Expression, right: Expression) extends CudfBinaryMathExpression with Product with Serializable
  128. trait GpuPredicateHelper extends AnyRef
  129. case class GpuQuarter(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  130. case class GpuRemainder(left: Expression, right: Expression) extends CudfBinaryArithmetic with GpuDivModLike with Product with Serializable
  131. case class GpuRint(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  132. case class GpuRound(child: Expression, scale: Expression) extends GpuRoundBase with Product with Serializable
  133. abstract class GpuRoundBase extends BinaryExpression with GpuBinaryExpression with Serializable with ImplicitCastInputTypes
  134. case class GpuScalaUDF(function: RapidsUDF, dataType: DataType, children: Seq[Expression], udfName: Option[String], nullable: Boolean, udfDeterministic: Boolean) extends Expression with GpuUserDefinedFunction with Product with Serializable
  135. case class GpuScalarSubquery(plan: BaseSubqueryExec, exprId: ExprId) extends ExecSubqueryExpression with GpuExpression with Product with Serializable

    GPU placeholder of ScalarSubquery, which returns the scalar result with columnarEval method.

    GPU placeholder of ScalarSubquery, which returns the scalar result with columnarEval method. This placeholder is to make ScalarSubquery working as a GPUExpression to cooperate other GPU overrides.

  136. case class GpuSecond(child: Expression, timeZoneId: Option[String] = None) extends GpuUnaryExpression with GpuTimeUnaryExpression with Product with Serializable
  137. class GpuSerializableBatch extends Serializable with AutoCloseable with Arm
    Annotations
    @SerialVersionUID()
  138. trait GpuShiftBase extends BinaryExpression with GpuBinaryExpression with ImplicitCastInputTypes
  139. case class GpuShiftLeft(left: Expression, right: Expression) extends BinaryExpression with GpuShiftBase with Product with Serializable
  140. case class GpuShiftRight(left: Expression, right: Expression) extends BinaryExpression with GpuShiftBase with Product with Serializable
  141. case class GpuShiftRightUnsigned(left: Expression, right: Expression) extends BinaryExpression with GpuShiftBase with Product with Serializable
  142. class GpuShuffleBlockResolver extends ShuffleBlockResolver with Logging
  143. class GpuShuffleDependency[K, V, C] extends ShuffleDependency[K, V, C]
  144. class GpuShuffleEnv extends Logging
  145. class GpuShuffleHandle[K, V] extends BaseShuffleHandle[K, V, V]
  146. case class GpuSignum(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  147. case class GpuSin(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  148. class GpuSingleDirectoryDataWriter extends GpuFileFormatDataWriter

    Writes data to a single directory (used for non-dynamic-partition writes).

  149. case class GpuSinh(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  150. case class GpuSize(child: Expression, legacySizeOfNull: Boolean) extends GpuUnaryExpression with Product with Serializable
  151. case class GpuSortArray(base: Expression, ascendingOrder: Expression) extends BinaryExpression with GpuBinaryExpression with ExpectsInputTypes with Product with Serializable
  152. case class GpuSqrt(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  153. case class GpuStartsWith(left: Expression, right: Expression) extends BinaryExpression with GpuBinaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  154. case class GpuStringLPad(str: Expression, len: Expression, pad: Expression) extends TernaryExpression with BasePad with Product with Serializable
  155. case class GpuStringLocate(substr: Expression, col: Expression, start: Expression) extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with Product with Serializable
  156. case class GpuStringRPad(str: Expression, len: Expression, pad: Expression) extends TernaryExpression with BasePad with Product with Serializable
  157. case class GpuStringReplace(srcExpr: Expression, searchExpr: Expression, replaceExpr: Expression) extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with Product with Serializable
  158. case class GpuStringSplit(str: Expression, regex: Expression, limit: Expression) extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with Product with Serializable
  159. class GpuStringSplitMeta extends TernaryExprMeta[StringSplit]
  160. case class GpuStringTrim(column: Expression, trimParameters: Option[Expression] = None) extends Expression with GpuString2TrimExpression with ImplicitCastInputTypes with Product with Serializable
  161. case class GpuStringTrimLeft(column: Expression, trimParameters: Option[Expression] = None) extends Expression with GpuString2TrimExpression with ImplicitCastInputTypes with Product with Serializable
  162. case class GpuStringTrimRight(column: Expression, trimParameters: Option[Expression] = None) extends Expression with GpuString2TrimExpression with ImplicitCastInputTypes with Product with Serializable
  163. case class GpuSubstring(str: Expression, pos: Expression, len: Expression) extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
  164. case class GpuSubstringIndex(strExpr: Expression, regexp: String, ignoredDelimExpr: Expression, ignoredCountExpr: Expression) extends TernaryExpression with GpuTernaryExpression with ImplicitCastInputTypes with Product with Serializable
  165. case class GpuSubtract(left: Expression, right: Expression) extends CudfBinaryArithmetic with Product with Serializable
  166. case class GpuSum(child: Expression, resultType: DataType) extends Expression with GpuAggregateFunction with ImplicitCastInputTypes with GpuBatchedRunningWindowWithFixer with GpuAggregateWindowFunction[SumAggregation] with GpuRunningWindowFunction with Product with Serializable
  167. case class GpuTan(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  168. case class GpuTanh(child: Expression) extends CudfUnaryMathExpression with Product with Serializable
  169. case class GpuTimeAdd(start: Expression, interval: Expression, timeZoneId: Option[String] = None) extends GpuTimeMath with Product with Serializable
  170. abstract class GpuTimeMath extends BinaryExpression with GpuExpression with TimeZoneAwareExpression with ExpectsInputTypes with Serializable
  171. case class GpuTimeSub(start: Expression, interval: Expression, timeZoneId: Option[String] = None) extends GpuTimeMath with Product with Serializable
  172. trait GpuTimeUnaryExpression extends GpuUnaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with NullIntolerant
  173. case class GpuToDegrees(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  174. case class GpuToRadians(child: Expression) extends GpuUnaryMathExpression with Product with Serializable
  175. abstract class GpuToTimestamp extends BinaryExpression with GpuBinaryExpression with TimeZoneAwareExpression with ExpectsInputTypes

    A direct conversion of Spark's ToTimestamp class which converts time to UNIX timestamp by first converting to microseconds and then dividing by the downScaleFactor

  176. abstract class GpuToTimestampImproved extends GpuToTimestamp

    An improved version of GpuToTimestamp conversion which converts time to UNIX timestamp without first converting to microseconds

  177. case class GpuToUnixTimestamp(strTs: Expression, format: Expression, sparkFormat: String, strf: String, timeZoneId: Option[String] = None) extends GpuToTimestamp with Product with Serializable
  178. case class GpuToUnixTimestampImproved(strTs: Expression, format: Expression, sparkFormat: String, strf: String, timeZoneId: Option[String] = None) extends GpuToTimestampImproved with Product with Serializable
  179. abstract class GpuUnaryMathExpression extends GpuUnaryExpression with Serializable with ImplicitCastInputTypes
  180. case class GpuUnaryMinus(child: Expression) extends GpuUnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
  181. case class GpuUnaryPositive(child: Expression) extends GpuUnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
  182. abstract class GpuUnaryString2StringExpression extends GpuUnaryExpression with ExpectsInputTypes
  183. case class GpuUnixTimestamp(strTs: Expression, format: Expression, sparkFormat: String, strf: String, timeZoneId: Option[String] = None) extends GpuToTimestamp with Product with Serializable
  184. case class GpuUnixTimestampImproved(strTs: Expression, format: Expression, sparkFormat: String, strf: String, timeZoneId: Option[String] = None) extends GpuToTimestampImproved with Product with Serializable
  185. case class GpuUpper(child: Expression) extends GpuUnaryString2StringExpression with Product with Serializable
  186. case class GpuWeekDay(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  187. class GpuWriteJobDescription extends Serializable

    A shared job description for all the GPU write tasks.

    A shared job description for all the GPU write tasks. This is the GPU version of org.apache.spark.sql.execution.datasources.WriteJobDescription.

  188. class GpuWriteJobStatsTracker extends BasicColumnarWriteJobStatsTracker

    Simple ColumnarWriteJobStatsTracker implementation that's serializable, capable of instantiating GpuWriteTaskStatsTracker on executors and processing the WriteTaskStats they produce by aggregating the metrics and posting them as DriverMetricUpdates.

  189. class GpuWriteTaskStatsTracker extends BasicColumnarWriteTaskStatsTracker

    ColumnarWriteTaskStatsTracker implementation that produces WriteTaskStats and tracks writing times per task.

  190. case class GpuYear(child: Expression) extends GpuUnaryExpression with GpuDateUnaryExpression with Product with Serializable
  191. case class LegacyParseFormat(separator: Char, isTimestamp: Boolean, validRegex: String) extends Product with Serializable
  192. trait OrcFiltersBase extends AnyRef

    Methods that can be shared when upgrading the built-in Hive.

    Methods that can be shared when upgrading the built-in Hive.

    Derived from Apache Spark to avoid depending upon it directly, since its API has changed between Spark versions.

  193. class RapidsCachingReader[K, C] extends ShuffleReader[K, C] with Arm with Logging
  194. class RapidsCachingWriter[K, V] extends ShuffleWriter[K, V] with Logging
  195. class RapidsDiskBlockManager extends AnyRef

    Maps logical blocks to local disk locations.

  196. abstract class RapidsShuffleInternalManagerBase extends ShuffleManager with RapidsShuffleHeartbeatHandler with Logging

    A shuffle manager optimized for the RAPIDS Plugin For Apache Spark.

    A shuffle manager optimized for the RAPIDS Plugin For Apache Spark.

    Note

    This is an internal class to obtain access to the private ShuffleManager and SortShuffleManager classes. When configuring Apache Spark to use the RAPIDS shuffle manager,

  197. case class RegexReplace(search: String, replace: String) extends Product with Serializable
  198. trait ShuffleManagerShimBase extends AnyRef
  199. trait ShuffleMetricsUpdater extends AnyRef
  200. class SubstringIndexMeta extends TernaryExprMeta[SubstringIndex]
  201. case class TempSpillBufferId extends RapidsBufferId with Product with Serializable
  202. sealed trait TimeParserPolicy extends Serializable
  203. abstract class UnixTimeExprMeta[A <: BinaryExpression with TimeZoneAwareExpression] extends BinaryExprMeta[A]
  204. case class WrappedAggFunction(aggregateFunction: GpuAggregateFunction, filter: Expression) extends Expression with GpuAggregateFunction with Product with Serializable

Value Members

  1. object BasicColumnarWriteJobStatsTracker extends Serializable
  2. object CorrectedTimeParserPolicy extends TimeParserPolicy
  3. object CudfRegexp
  4. object ExceptionTimeParserPolicy extends TimeParserPolicy
  5. object GpuDataSource extends Logging with Serializable
  6. object GpuDataSourceScanExec extends Serializable
  7. object GpuDivModLike
  8. object GpuDivideUtil
  9. object GpuFileFormatWriter extends Logging

    A helper object for writing columnar data out to a location.

  10. object GpuFileSourceScanExec extends Serializable
  11. object GpuLogarithm extends Arm with Serializable
  12. object GpuLogicHelper
  13. object GpuMultiplyUtil
  14. object GpuMurmur3Hash extends Arm with Serializable
  15. object GpuOrcFileFormat extends Logging
  16. object GpuScalaUDF extends Serializable
  17. object GpuShuffleEnv extends Logging
  18. object GpuSubstringIndex extends Serializable
  19. object GpuToTimestamp extends Arm
  20. object GpuWriteJobStatsTracker extends Serializable
  21. object InputFileUtils
  22. object LegacyTimeParserPolicy extends TimeParserPolicy
  23. object OrcFilters extends OrcFiltersBase

    Helper object for building ORC SearchArguments, which are used for ORC predicate push-down.

    Helper object for building ORC SearchArguments, which are used for ORC predicate push-down.

    Due to limitation of ORC SearchArgument builder, we had to implement separate checking and conversion passes through the Filter to make sure we only convert predicates that are known to be convertible.

    An ORC SearchArgument must be built in one pass using a single builder. For example, you can't build a = 1 and b = 2 first, and then combine them into a = 1 AND b = 2. This is quite different from the cases in Spark SQL or Parquet, where complex filters can be easily built using existing simpler ones.

    The annoying part is that, SearchArgument builder methods like startAnd(), startOr(), and startNot() mutate internal state of the builder instance. This forces us to translate all convertible filters with a single builder instance. However, if we try to translate a filter before checking whether it can be converted or not, we may end up with a builder whose internal state is inconsistent in the case of an inconvertible filter.

    For example, to convert an And filter with builder b, we call b.startAnd() first, and then try to convert its children. Say we convert left child successfully, but find that right child is inconvertible. Alas, b.startAnd() call can't be rolled back, and b is inconsistent now.

    The workaround employed here is to trim the Spark filters before trying to convert them. This way, we can only do the actual conversion on the part of the Filter that is known to be convertible.

    P.S.: Hive seems to use SearchArgument together with ExprNodeGenericFuncDesc only. Usage of builder methods mentioned above can only be found in test code, where all tested filters are known to be convertible.

  24. object RapidsShuffleInternalManagerBase extends Logging
  25. object ShiftHelper extends Arm
  26. object TempSpillBufferId extends Serializable

Ungrouped