Package io.trino.execution.buffer

Class ArbitraryOutputBuffer

java.lang.Object

io.trino.execution.buffer.ArbitraryOutputBuffer

All Implemented Interfaces:

OutputBuffer

public class ArbitraryOutputBuffer
extends Object
implements OutputBuffer

A buffer that assigns pages to queues based on a first come, first served basis.

Constructor Summary

Constructors

Constructor	Description
ArbitraryOutputBuffer(String taskInstanceId, StateMachine<BufferState> state, io.airlift.units.DataSize maxBufferSize, Supplier<LocalMemoryContext> systemMemoryContextSupplier, Executor notificationExecutor)

Method Summary

Modifier and Type	Method	Description
void	abort(OutputBuffers.OutputBufferId bufferId)	Closes the specified output buffer.
void	acknowledge(OutputBuffers.OutputBufferId bufferId, long sequenceId)	Acknowledges the previously received pages from the output buffer.
void	addStateChangeListener(StateMachine.StateChangeListener<BufferState> stateChangeListener)	Add a listener which fires anytime the buffer state changes.
void	destroy()	Destroys the buffer, discarding all pages.
void	enqueue(int partition, List<SerializedPage> pages)	Adds a split-up page to a specific partition.
void	enqueue(List<SerializedPage> pages)	Adds a split-up page to an unpartitioned buffer.
void	fail()	Fail the buffer, discarding all pages, but blocking readers.
com.google.common.util.concurrent.ListenableFuture<BufferResult>	get(OutputBuffers.OutputBufferId bufferId, long startingSequenceId, io.airlift.units.DataSize maxSize)	Gets pages from the output buffer, and acknowledges all pages received from the last request.
OutputBufferInfo	getInfo()	Gets the current state of this buffer.
long	getPeakMemoryUsage()
double	getUtilization()	Get the memory utilization percentage.
boolean	isFinished()	A buffer is finished once no-more-pages has been set and all buffers have been closed with an abort call.
com.google.common.util.concurrent.ListenableFuture<Void>	isFull()	Get a future that will be completed when the buffer is not full.
boolean	isOverutilized()	Check if the buffer is blocking producers.
void	setNoMorePages()	Notify buffer that no more pages will be added.
void	setOutputBuffers(OutputBuffers newOutputBuffers)	Updates the buffer configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArbitraryOutputBuffer

public ArbitraryOutputBuffer(String taskInstanceId,
                             StateMachine<BufferState> state,
                             io.airlift.units.DataSize maxBufferSize,
                             Supplier<LocalMemoryContext> systemMemoryContextSupplier,
                             Executor notificationExecutor)

Method Detail

addStateChangeListener

public void addStateChangeListener(StateMachine.StateChangeListener<BufferState> stateChangeListener)

Description copied from interface: OutputBuffer

Add a listener which fires anytime the buffer state changes. Listener is always notified asynchronously using a dedicated notification thread pool so, care should be taken to avoid leaking this when adding a listener in a constructor. Additionally, it is possible notifications are observed out of order due to the asynchronous execution.

Specified by:

addStateChangeListener in interface OutputBuffer

isFinished

public boolean isFinished()

Description copied from interface: OutputBuffer

A buffer is finished once no-more-pages has been set and all buffers have been closed with an abort call.

Specified by:

isFinished in interface OutputBuffer

getUtilization

public double getUtilization()

Description copied from interface: OutputBuffer

Get the memory utilization percentage.

Specified by:

getUtilization in interface OutputBuffer

isOverutilized

public boolean isOverutilized()

Description copied from interface: OutputBuffer

Check if the buffer is blocking producers.

Specified by:

isOverutilized in interface OutputBuffer

getInfo

public OutputBufferInfo getInfo()

Description copied from interface: OutputBuffer

Gets the current state of this buffer. This method is guaranteed to not block or acquire contended locks, but the stats in the info object may be internally inconsistent.

Specified by:

getInfo in interface OutputBuffer

setOutputBuffers

public void setOutputBuffers(OutputBuffers newOutputBuffers)

Description copied from interface: OutputBuffer

Updates the buffer configuration.

Specified by:

setOutputBuffers in interface OutputBuffer

isFull

public com.google.common.util.concurrent.ListenableFuture<Void> isFull()

Description copied from interface: OutputBuffer

Get a future that will be completed when the buffer is not full.

Specified by:

isFull in interface OutputBuffer

enqueue

public void enqueue(List<SerializedPage> pages)

Description copied from interface: OutputBuffer

Adds a split-up page to an unpartitioned buffer. If no-more-pages has been set, the enqueue page call is ignored. This can happen with limit queries.

Specified by:

enqueue in interface OutputBuffer

enqueue

public void enqueue(int partition,
                    List<SerializedPage> pages)

Description copied from interface: OutputBuffer

Adds a split-up page to a specific partition. If no-more-pages has been set, the enqueue page call is ignored. This can happen with limit queries.

Specified by:

enqueue in interface OutputBuffer

get

public com.google.common.util.concurrent.ListenableFuture<BufferResult> get(OutputBuffers.OutputBufferId bufferId,
                                                                            long startingSequenceId,
                                                                            io.airlift.units.DataSize maxSize)

Description copied from interface: OutputBuffer

Gets pages from the output buffer, and acknowledges all pages received from the last request. The initial token is zero. Subsequent tokens are acquired from the next token field in the BufferResult returned from the previous request. If the buffer result is marked as complete, the client must call abort to acknowledge receipt of the final state.

Specified by:

get in interface OutputBuffer

acknowledge

public void acknowledge(OutputBuffers.OutputBufferId bufferId,
                        long sequenceId)

Description copied from interface: OutputBuffer

Acknowledges the previously received pages from the output buffer.

Specified by:

acknowledge in interface OutputBuffer

abort

public void abort(OutputBuffers.OutputBufferId bufferId)

Description copied from interface: OutputBuffer

Closes the specified output buffer.

Specified by:

abort in interface OutputBuffer

setNoMorePages

public void setNoMorePages()

Description copied from interface: OutputBuffer

Notify buffer that no more pages will be added. Any future calls to enqueue a page are ignored.

Specified by:

setNoMorePages in interface OutputBuffer

destroy

public void destroy()

Description copied from interface: OutputBuffer

Destroys the buffer, discarding all pages.

Specified by:

destroy in interface OutputBuffer

fail

public void fail()

Description copied from interface: OutputBuffer

Fail the buffer, discarding all pages, but blocking readers. It is expected that readers will be unblocked when the failed query is cleaned up.

Specified by:

fail in interface OutputBuffer

getPeakMemoryUsage

public long getPeakMemoryUsage()

Specified by:

getPeakMemoryUsage in interface OutputBuffer

Returns:

the peak memory usage of this output buffer.