org.apache.hudi.common.util.queue

Class BoundedInMemoryExecutor<I,O,E>

java.lang.Object

org.apache.hudi.common.util.queue.BoundedInMemoryExecutor<I,O,E>

public class BoundedInMemoryExecutor<I,O,E>
extends Object

Executor which orchestrates concurrent producers and consumers communicating through a bounded in-memory queue. This class takes as input the size limit, queue producer(s), consumer and transformer and exposes API to orchestrate concurrent execution of these actors communicating through a central bounded queue

Constructor Summary

Constructors

Constructor and Description
BoundedInMemoryExecutor(long bufferLimitInBytes, BoundedInMemoryQueueProducer<I> producer, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction)
BoundedInMemoryExecutor(long bufferLimitInBytes, BoundedInMemoryQueueProducer<I> producer, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction, Runnable preExecuteRunnable)
BoundedInMemoryExecutor(long bufferLimitInBytes, Iterator<I> inputItr, BoundedInMemoryQueueConsumer<O,E> consumer, Function<I,O> transformFunction, Runnable preExecuteRunnable)
BoundedInMemoryExecutor(long bufferLimitInBytes, List<BoundedInMemoryQueueProducer<I>> producers, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction, SizeEstimator<O> sizeEstimator, Runnable preExecuteRunnable)

Method Summary

Modifier and Type	Method and Description
boolean	awaitTermination()
E	execute() Main API to run both production and consumption.
BoundedInMemoryQueue<I,O>	getQueue()
boolean	isRemaining()
void	shutdownNow()
ExecutorCompletionService<Boolean>	startProducers() Start all Producers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BoundedInMemoryExecutor

public BoundedInMemoryExecutor(long bufferLimitInBytes,
                               Iterator<I> inputItr,
                               BoundedInMemoryQueueConsumer<O,E> consumer,
                               Function<I,O> transformFunction,
                               Runnable preExecuteRunnable)

BoundedInMemoryExecutor

public BoundedInMemoryExecutor(long bufferLimitInBytes,
                               BoundedInMemoryQueueProducer<I> producer,
                               Option<BoundedInMemoryQueueConsumer<O,E>> consumer,
                               Function<I,O> transformFunction)

BoundedInMemoryExecutor

public BoundedInMemoryExecutor(long bufferLimitInBytes,
                               BoundedInMemoryQueueProducer<I> producer,
                               Option<BoundedInMemoryQueueConsumer<O,E>> consumer,
                               Function<I,O> transformFunction,
                               Runnable preExecuteRunnable)

BoundedInMemoryExecutor

public BoundedInMemoryExecutor(long bufferLimitInBytes,
                               List<BoundedInMemoryQueueProducer<I>> producers,
                               Option<BoundedInMemoryQueueConsumer<O,E>> consumer,
                               Function<I,O> transformFunction,
                               SizeEstimator<O> sizeEstimator,
                               Runnable preExecuteRunnable)

Method Detail

startProducers

public ExecutorCompletionService<Boolean> startProducers()

Start all Producers.

execute

public E execute()

Main API to run both production and consumption.

isRemaining

public boolean isRemaining()

shutdownNow

public void shutdownNow()

awaitTermination

public boolean awaitTermination()

getQueue

public BoundedInMemoryQueue<I,O> getQueue()