trait RapidsBuffer extends AutoCloseable
Interface provided by all types of RAPIDS buffers
- Alphabetic
- By Inheritance
- RapidsBuffer
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Abstract Value Members
-
abstract
def
addReference(): Boolean
Try to add a reference to this buffer to acquire it.
Try to add a reference to this buffer to acquire it.
- returns
true if the reference was added or false if this buffer is no longer valid
- Note
The close method must be called for every successfully obtained reference.
-
abstract
def
close(): Unit
- Definition Classes
- AutoCloseable
- Annotations
- @throws( classOf[java.lang.Exception] )
-
abstract
def
copyToMemoryBuffer(srcOffset: Long, dst: MemoryBuffer, dstOffset: Long, length: Long, stream: Stream): Unit
Copy the content of this buffer into the specified memory buffer, starting from the given offset.
Copy the content of this buffer into the specified memory buffer, starting from the given offset.
- srcOffset
offset to start copying from.
- dst
the memory buffer to copy into.
- dstOffset
offset to copy into.
- length
number of bytes to copy.
- stream
CUDA stream to use
-
abstract
def
free(): Unit
Schedule the release of the buffer's underlying resources.
Schedule the release of the buffer's underlying resources. Subsequent attempts to acquire the buffer will fail. As soon as the buffer has no outstanding references, the resources will be released.
This is separate from the close method which does not normally release resources. close will only release resources if called as the last outstanding reference and the buffer was previously marked as freed.
-
abstract
def
getColumnarBatch(sparkTypes: Array[DataType]): ColumnarBatch
Get the columnar batch within this buffer.
Get the columnar batch within this buffer. The caller must have successfully acquired the buffer beforehand.
- sparkTypes
the spark data types the batch should have
- Note
It is the responsibility of the caller to close the batch.
,If the buffer is compressed data then the resulting batch will be built using
GpuCompressedColumnVector, and it is the responsibility of the caller to deal with decompressing the data if necessary.- See also
-
abstract
def
getDeviceMemoryBuffer: DeviceMemoryBuffer
Get the device memory buffer from the underlying storage.
Get the device memory buffer from the underlying storage. If the buffer currently resides outside of device memory, a new DeviceMemoryBuffer is created with the data copied over. The caller must have successfully acquired the buffer beforehand.
- Note
It is the responsibility of the caller to close the buffer.
- See also
-
abstract
def
getHostMemoryBuffer: HostMemoryBuffer
Get the host memory buffer from the underlying storage.
Get the host memory buffer from the underlying storage. If the buffer currently resides outside of host memory, a new HostMemoryBuffer is created with the data copied over. The caller must have successfully acquired the buffer beforehand.
- Note
It is the responsibility of the caller to close the buffer.
- See also
-
abstract
def
getMemoryBuffer: MemoryBuffer
Get the underlying memory buffer.
Get the underlying memory buffer. This may be either a HostMemoryBuffer or a DeviceMemoryBuffer depending on where the buffer currently resides. The caller must have successfully acquired the buffer beforehand.
- Note
It is the responsibility of the caller to close the buffer.
- See also
-
abstract
def
getSpillPriority: Long
Get the spill priority value for this buffer.
Get the spill priority value for this buffer. Lower values are higher priority for spilling, meaning buffers with lower values will be preferred for spilling over buffers with a higher value.
-
abstract
val
id: RapidsBufferId
The buffer identifier for this buffer.
-
abstract
val
memoryUsedBytes: Long
The size of this buffer in bytes in its _current_ store.
The size of this buffer in bytes in its _current_ store. As the buffer goes through contiguous split (either added as a contiguous table already, or spilled to host), its size changes because contiguous_split adds its own alignment padding.
- Note
Do not use this size to allocate a target buffer to copy, always use
getPackedSize.
-
abstract
def
meta: TableMeta
Descriptor for how the memory buffer is formatted
-
abstract
def
setSpillPriority(priority: Long): Unit
Set the spill priority for this buffer.
Set the spill priority for this buffer. Lower values are higher priority for spilling, meaning buffers with lower values will be preferred for spilling over buffers with a higher value.
- priority
new priority value for this buffer
- Note
should only be called from the buffer catalog
-
abstract
val
storageTier: StorageTier
The storage tier for this buffer
-
abstract
def
withMemoryBufferReadLock[K](body: (MemoryBuffer) ⇒ K): K
Obtains a read lock on this instance of
RapidsBufferand calls the function inbodywhile holding the lock.Obtains a read lock on this instance of
RapidsBufferand calls the function inbodywhile holding the lock.- K
any return type specified by
body- body
function that takes a
MemoryBufferand producesK- returns
the result of body(memoryBuffer)
-
abstract
def
withMemoryBufferWriteLock[K](body: (MemoryBuffer) ⇒ K): K
Obtains a write lock on this instance of
RapidsBufferand calls the function inbodywhile holding the lock.Obtains a write lock on this instance of
RapidsBufferand calls the function inbodywhile holding the lock.- K
any return type specified by
body- body
function that takes a
MemoryBufferand producesK- returns
the result of body(memoryBuffer)
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getCopyIterator: RapidsBufferCopyIterator
At spill time, obtain an iterator used to copy this buffer to a different tier.
-
def
getHostColumnarBatch(sparkTypes: Array[DataType]): ColumnarBatch
Get the host-backed columnar batch from this buffer.
Get the host-backed columnar batch from this buffer. The caller must have successfully acquired the buffer beforehand.
If this
RapidsBufferwas added originally to the device tier, or if this is a just a buffer (not a batch), this function will throw.- sparkTypes
the spark data types the batch should have
- Note
It is the responsibility of the caller to close the batch.
- See also
-
def
getPackedSizeBytes: Long
The size of this buffer if it has already gone through contiguous_split.
The size of this buffer if it has already gone through contiguous_split.
- Note
Use this function when allocating a target buffer for spill or shuffle purposes.
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
makeChunkedPacker: ChunkedPacker
Makes a new chunked packer.
Makes a new chunked packer. It is the responsibility of the caller to close this.
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val supportsChunkedPacker: Boolean
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
updateSpillability(): Unit
Function invoked by the
RapidsBufferStore.addBuffermethod that prompts the specificRapidsBufferto check its reference counting to make itself spillable or not.Function invoked by the
RapidsBufferStore.addBuffermethod that prompts the specificRapidsBufferto check its reference counting to make itself spillable or not. OnlyRapidsTableandRapidsHostMemoryBufferimplement this method. -
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()