org.apache.hudi.common.util.queue

Class SimpleExecutor<I,O,E>

java.lang.Object

org.apache.hudi.common.util.queue.SimpleExecutor<I,O,E>

All Implemented Interfaces:

HoodieExecutor<E>

public class SimpleExecutor<I,O,E>
extends Object
implements HoodieExecutor<E>

Simple implementation of the HoodieExecutor interface assuming single-writer/single-reader mode allowing it to consume from the input Iterator directly avoiding the need for any internal materialization (ie queueing).

Such executor is aimed primarily at allowing the production-consumption chain to run w/ as little overhead as possible, at the expense of limited parallelism and therefore throughput, which is not an issue for execution environments such as Spark, where it's used primarily in a parallelism constraint environment (on executors)

Constructor Summary

Constructors

Constructor and Description
SimpleExecutor(Iterator<I> inputItr, HoodieConsumer<O,E> consumer, Function<I,O> transformFunction)

Method Summary

Modifier and Type	Method and Description
boolean	awaitTermination() Allows to gracefully await the termination of the executor
E	execute() Consuming records from input iterator directly without any producers and inner message queue.
void	shutdownNow() Shuts executor down immediately, cleaning up any allocated resources

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleExecutor

public SimpleExecutor(Iterator<I> inputItr,
                      HoodieConsumer<O,E> consumer,
                      Function<I,O> transformFunction)

Method Detail

execute

public E execute()

Consuming records from input iterator directly without any producers and inner message queue.

Specified by:

execute in interface HoodieExecutor<E>

shutdownNow

public void shutdownNow()

Description copied from interface: HoodieExecutor

Shuts executor down immediately, cleaning up any allocated resources

Specified by:

shutdownNow in interface HoodieExecutor<E>

awaitTermination

public boolean awaitTermination()

Description copied from interface: HoodieExecutor

Allows to gracefully await the termination of the executor

Specified by:

awaitTermination in interface HoodieExecutor<E>