Type Members

1.  abstract class CurrentBatchIterator extends Iterator[ColumnarBatch] with AutoCloseable
2.  class ExclusiveModeGpuDiscoveryPlugin extends ResourceDiscoveryPlugin with Proxy

   A Spark Resource Discovery Plugin that relies on the NVIDIA GPUs being in PROCESS_EXCLUSIVE mode so that it can discover free GPUs.

   A Spark Resource Discovery Plugin that relies on the NVIDIA GPUs being in PROCESS_EXCLUSIVE mode so that it can discover free GPUs. This plugin iterates through all the GPUs on the node and tries to initialize a CUDA context on each one. When the GPUs are in process exclusive mode this will result in that GPU being assigned to the specific process running this plugin and other executors will not be able to use it.

   This plugin can be activated in spark with the configuration: --conf spark.resources.discoveryPlugin=com.nvidia.spark.ExclusiveModeGpuDiscoveryPlugin

3.  trait GpuCachedBatchSerializer extends CachedBatchSerializer
4.  class ParquetCachedBatchSerializer extends GpuCachedBatchSerializer with Proxy

   User facing wrapper class that calls into the internal version.

5.  trait RapidsUDF extends AnyRef

   A RAPIDS accelerated version of a user-defined function (UDF).

6.  trait Retryable extends AnyRef

   An interface that can be used to retry the processing on non-deterministic expressions on the GPU.

   An interface that can be used to retry the processing on non-deterministic expressions on the GPU.

   GPU memory is a limited resource. When it runs out the RAPIDS Accelerator for Apache Spark will use several different strategies to try and free more GPU memory to let the query complete. One of these strategies is to roll back the processioning for one task, pause the task thread, then retry the task when more memory is available. This works transparently for any stateless deterministic processing. But technically an expression/UDF can be non-deterministic and/or keep state in between calls. This interface provides a checkpoint method to save any needed state, and a restore method to reset the state in the case of a retry.

   Please note that a retry is not isolated to a single expression, so a restore can be called even after the expression returned one or more batches of results. And each time checkpoint it called any previously saved state can be overwritten.

7.  class SQLPlugin extends SparkPlugin with Logging

   The RAPIDS plugin for Spark.

   The RAPIDS plugin for Spark. To enable this plugin, set the config "spark.plugins" to com.nvidia.spark.SQLPlugin