Packages

c

com.nvidia.spark.rapids.RapidsPluginImplicits

AutoCloseableFromBatchColumns

implicit class AutoCloseableFromBatchColumns extends MapsSafely[Int, Seq[Int]]

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AutoCloseableFromBatchColumns
  2. MapsSafely
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AutoCloseableFromBatchColumns(in: ColumnarBatch)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. val in: ColumnarBatch
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. def safeMap[B <: AutoCloseable](fn: (GpuColumnVector) ⇒ B): Seq[B]

    safeMap: Is an implicit on ColumnarBatch, that lets you map over the columns of a batch as if the batch was a Seq[GpuColumnVector], iff safeMap's body is producing AutoCloseable (otherwise, it is not defined).

    safeMap: Is an implicit on ColumnarBatch, that lets you map over the columns of a batch as if the batch was a Seq[GpuColumnVector], iff safeMap's body is producing AutoCloseable (otherwise, it is not defined).

    See MapsSafely.safeMap for a more detailed explanation.

    B

    the type of the elements produced in the safeMap (should be subclasses of AutoCloseable)

    fn

    a function that takes GpuColumnVector, and returns a subclass of AutoCloseable

    returns

    a sequence of B, in the success case

  17. def safeMap[B <: AutoCloseable, That](in: SeqLike[Int, Seq[Int]], fn: (Int) ⇒ B)(implicit bf: CanBuildFrom[Seq[Int], B, That]): That

    safeMap: safeMap implementation that is leveraged by other type-specific implicits.

    safeMap: safeMap implementation that is leveraged by other type-specific implicits.

    safeMap has the added safety net that as you produce AutoCloseable values they are tracked, and if an exception were to occur within the maps's body, it will make every attempt to close each produced value.

    Note: safeMap will close in case of errors, without any knowledge of whether it should or not. Use safeMap only in these circumstances if fn increases the reference count, producing an AutoCloseable, and nothing else is tracking these references: a) seq.safeMap(x => {...; x.incRefCount; x}) b) seq.safeMap(x => GpuColumnVector.from(...))

    Usage of safeMap chained with other maps is a bit confusing:

    seq.map(GpuColumnVector.from).safeMap(couldThrow)

    Will close the column vectors produced from couldThrow up until the time where safeMap throws.

    The correct pattern of usage in cases like this is:

    val closeTheseLater = seq.safeMap(GpuColumnVector.from) closeTheseLater.safeMap{ x => var success = false try { val res = couldThrow(x.incRefCount()) success = true res // return a ref count of 2 } finally { if (!success) { // in case of an error, we close x as part of normal error handling // the exception will be caught by the safeMap, and it will close all // AutoCloseables produced before x // - Sequence looks like: [2, 2, 2, ..., 2] + x, which has also has a refcount of 2 x.close() // x now has a ref count of 1, the rest of the sequence has 2s } } } // safeMap cleaned, and now everything has 1s for ref counts (as they were before)

    closeTheseLater.safeClose() // go from 1 to 0 in all things inside closeTheseLater

    B

    the type of the elements produced in the safeMap (should be subclasses of AutoCloseable)

    That

    the type of the output collection (needed by builder)

    in

    the Seq[A] to map on

    fn

    a function that takes A, and produces B (a subclass of AutoCloseable)

    returns

    a sequence of B, in the success case

    Attributes
    protected
    Definition Classes
    MapsSafely
  18. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from MapsSafely[Int, Seq[Int]]

Inherited from AnyRef

Inherited from Any

Ungrouped