Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  def addReference(): Boolean

   Try to add a reference to this buffer to acquire it.

   Try to add a reference to this buffer to acquire it.

   returns

   true if the reference was added or false if this buffer is no longer valid

   Definition Classes

   RapidsBufferBase → RapidsBuffer

   Note

   The close method must be called for every successfully obtained reference.

5.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

6.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

7.  def close(): Unit

   close() is called by client code to decrease the ref count of this RapidsBufferBase.

   close() is called by client code to decrease the ref count of this RapidsBufferBase. In the off chance that by the time close is invoked, the buffer was freed (not valid) then this close call winds up freeing the resources of the rapids buffer.

   Definition Classes

   RapidsBufferBase → AutoCloseable

8.  def columnarBatchFromDeviceBuffer(devBuffer: DeviceMemoryBuffer, sparkTypes: Array[DataType]): ColumnarBatch

   Attributes

   protected

   Definition Classes

   RapidsBufferBase

9.  def copyToMemoryBuffer(srcOffset: Long, dst: MemoryBuffer, dstOffset: Long, length: Long, stream: Stream): Unit

   Copy the content of this buffer into the specified memory buffer, starting from the given offset.

   Copy the content of this buffer into the specified memory buffer, starting from the given offset.

   srcOffset

   offset to start copying from.

   dst

   the memory buffer to copy into.

   dstOffset

   offset to copy into.

   length

   number of bytes to copy.

   stream

   CUDA stream to use

   Definition Classes

   RapidsBufferBase → RapidsBuffer

10.  final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

11.  def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

12.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

13.  def free(): Unit

    We overwrite free to make sure we don't have a handler for the underlying buffer, since this RapidsBuffer is no longer tracked.

    We overwrite free to make sure we don't have a handler for the underlying buffer, since this RapidsBuffer is no longer tracked.

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBufferBase → RapidsBuffer

14.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

15.  def getColumnarBatch(sparkTypes: Array[DataType]): ColumnarBatch

    Get the columnar batch within this buffer.

    Get the columnar batch within this buffer. The caller must have successfully acquired the buffer beforehand.

    sparkTypes

    the spark data types the batch should have

    Definition Classes

    RapidsBufferBase → RapidsBuffer

    Note

    It is the responsibility of the caller to close the batch.

    ,

    If the buffer is compressed data then the resulting batch will be built using GpuCompressedColumnVector, and it is the responsibility of the caller to deal with decompressing the data if necessary.

    See also

    addReference

16.  def getCopyIterator: RapidsBufferCopyIterator

    At spill time, obtain an iterator used to copy this buffer to a different tier.

    At spill time, obtain an iterator used to copy this buffer to a different tier.

    Definition Classes

    RapidsBuffer

17.  def getDeviceMemoryBuffer: DeviceMemoryBuffer

    TODO: we want to remove this method from the buffer, instead we want the catalog to be responsible for producing the DeviceMemoryBuffer by asking the buffer.

    TODO: we want to remove this method from the buffer, instead we want the catalog to be responsible for producing the DeviceMemoryBuffer by asking the buffer. This hides the RapidsBuffer from clients and simplifies locking.

    Definition Classes

    RapidsBufferBase → RapidsBuffer

18.  def getHostColumnarBatch(sparkTypes: Array[DataType]): ColumnarBatch

    Get the host-backed columnar batch from this buffer.

    Get the host-backed columnar batch from this buffer. The caller must have successfully acquired the buffer beforehand.

    If this RapidsBuffer was added originally to the device tier, or if this is a just a buffer (not a batch), this function will throw.

    sparkTypes

    the spark data types the batch should have

    Definition Classes

    RapidsBuffer

    Note

    It is the responsibility of the caller to close the batch.

    See also

    addReference

19.  def getHostMemoryBuffer: HostMemoryBuffer

    Get the host memory buffer from the underlying storage.

    Get the host memory buffer from the underlying storage. If the buffer currently resides outside of host memory, a new HostMemoryBuffer is created with the data copied over. The caller must have successfully acquired the buffer beforehand.

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBufferBase → RapidsBuffer

    Note

    It is the responsibility of the caller to close the buffer.

    See also

    addReference

20.  def getMemoryBuffer: MemoryBuffer

    Get the underlying memory buffer.

    Get the underlying memory buffer. This may be either a HostMemoryBuffer or a DeviceMemoryBuffer depending on where the buffer currently resides. The caller must have successfully acquired the buffer beforehand.

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBuffer

    Note

    It is the responsibility of the caller to close the buffer.

    See also

    addReference

21.  def getPackedSizeBytes: Long

    The size of this buffer if it has already gone through contiguous_split.

    The size of this buffer if it has already gone through contiguous_split.

    Definition Classes

    RapidsBuffer

    Note

    Use this function when allocating a target buffer for spill or shuffle purposes.

22.  def getSpillPriority: Long

    Get the spill priority value for this buffer.

    Get the spill priority value for this buffer. Lower values are higher priority for spilling, meaning buffers with lower values will be preferred for spilling over buffers with a higher value.

    Definition Classes

    RapidsBufferBase → RapidsBuffer

23.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

24.  val id: RapidsBufferId

    The buffer identifier for this buffer.

    The buffer identifier for this buffer.

    Definition Classes

    RapidsBufferBase → RapidsBuffer

25.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

26.  var isValid: Boolean

    Attributes

    protected[this]

    Definition Classes

    RapidsBufferBase

27.  def makeChunkedPacker: ChunkedPacker

    Makes a new chunked packer.

    Makes a new chunked packer. It is the responsibility of the caller to close this.

    Definition Classes

    RapidsBuffer

28.  def materializeMemoryBuffer: MemoryBuffer

    Materialize the memory buffer from the underlying storage.

    Materialize the memory buffer from the underlying storage.

    If the buffer resides in device or host memory, only reference count is incremented. If the buffer resides in secondary storage, a new host or device memory buffer is created, with the data copied to the new buffer. The caller must have successfully acquired the buffer beforehand.

    Attributes

    protected

    Definition Classes

    RapidsBufferBase

    Note

    It is the responsibility of the caller to close the buffer.

    ,

    This is an internal API only used by Rapids buffer stores.

    See also

    addReference

29.  val memoryUsedBytes: Long

    The size of this buffer in bytes.

    The size of this buffer in bytes.

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBuffer

30.  def meta: TableMeta

    Descriptor for how the memory buffer is formatted

    Descriptor for how the memory buffer is formatted

    Definition Classes

    RapidsBufferBase → RapidsBuffer

31.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

32.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

33.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

34.  def onClosed(refCount: Int): Unit

    Override from the MemoryBuffer.EventHandler interface.

    Override from the MemoryBuffer.EventHandler interface.

    If we are being invoked we have the buffer lock, as this callback is being invoked from MemoryBuffer.close

    refCount

    - buffer's current refCount

    Definition Classes

    RapidsHostMemoryBuffer → EventHandler

35.  var refcount: Int

    Attributes

    protected[this]

    Definition Classes

    RapidsBufferBase

36.  def releaseResources(): Unit

    Release the underlying resources for this buffer.

    Release the underlying resources for this buffer.

    Attributes

    protected

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBufferBase

37.  def setSpillPriority(priority: Long): Unit

    Set the spill priority for this buffer.

    Set the spill priority for this buffer. Lower values are higher priority for spilling, meaning buffers with lower values will be preferred for spilling over buffers with a higher value.

    priority

    new priority value for this buffer

    Definition Classes

    RapidsBufferBase → RapidsBuffer

    Note

    should only be called from the buffer catalog

38.  val storageTier: StorageTier

    The storage tier for this buffer

    The storage tier for this buffer

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBuffer

39.  val supportsChunkedPacker: Boolean

    Definition Classes

    RapidsBuffer

40.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

41.  def toString(): String

    Definition Classes

    RapidsBufferBase → AnyRef → Any

42.  def updateSpillability(): Unit

    Function invoked by the RapidsBufferStore.addBuffer method that prompts the specific RapidsBuffer to check its reference counting to make itself spillable or not.

    Function invoked by the RapidsBufferStore.addBuffer method that prompts the specific RapidsBuffer to check its reference counting to make itself spillable or not. Only RapidsTable and RapidsHostMemoryBuffer implement this method.

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBuffer

43.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

44.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

45.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()

46.  def withMemoryBufferReadLock[K](body: (MemoryBuffer) ⇒ K): K

    Obtains a read lock on this instance of RapidsBuffer and calls the function in body while holding the lock.

    Obtains a read lock on this instance of RapidsBuffer and calls the function in body while holding the lock.

    K

    any return type specified by body

    body

    function that takes a MemoryBuffer and produces K

    returns

    the result of body(memoryBuffer)

    Definition Classes

    RapidsBufferBase → RapidsBuffer

47.  def withMemoryBufferWriteLock[K](body: (MemoryBuffer) ⇒ K): K

    Obtains a write lock on this instance of RapidsBuffer and calls the function in body while holding the lock.

    Obtains a write lock on this instance of RapidsBuffer and calls the function in body while holding the lock.

    K

    any return type specified by body

    body

    function that takes a MemoryBuffer and produces K

    returns

    the result of body(memoryBuffer)

    Definition Classes

    RapidsBufferBase → RapidsBuffer

48.  def writeToChannel(outputChannel: WritableByteChannel, ignored: Stream): Long

    At spill time, write this buffer to an nio WritableByteChannel.

    At spill time, write this buffer to an nio WritableByteChannel.

    returns

    the amount of bytes written to the channel

    Definition Classes

    RapidsHostMemoryBuffer → RapidsBufferChannelWritable