org.apache.hadoop.tools.util

Class ProducerConsumer<T,R>

java.lang.Object

org.apache.hadoop.tools.util.ProducerConsumer<T,R>

public class ProducerConsumer<T,R>
extends Object

ProducerConsumer class encapsulates input and output queues and a thread-pool of Workers that loop on WorkRequest<T> inputQueue and for each consumed WorkRequest Workers invoke WorkRequestProcessor.processItem() and output resulting WorkReport<R> to the outputQueue.

Constructor Summary

Constructors

Constructor and Description
ProducerConsumer(int numThreads) ProducerConsumer maintains input and output queues and a thread-pool of workers.

Method Summary

Methods

Modifier and Type	Method and Description
void	addWorker(WorkRequestProcessor<T,R> processor) Add another worker that will consume WorkRequest<T> items from input queue, process each item using supplied processor, and for every processed item output WorkReport<R> to output queue.
WorkReport<R>	blockingTake() Blocking take from ProducerConsumer output queue (catches exceptions and retries forever).
int	getWorkCnt() Returns number of pending ProducerConsumer items (submitted to input queue for processing via put() method but not yet consumed by take() or blockingTake().
boolean	hasWork() Returns true if there are items in ProducerConsumer that are either pending for processing or waiting to be consumed.
void	put(WorkRequest<T> workRequest) Blocking put workRequest to ProducerConsumer input queue.
void	shutdown() Shutdown ProducerConsumer worker thread-pool without waiting for completion of any pending work.
WorkReport<R>	take() Blocking take from ProducerConsumer output queue that can be interrupted.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ProducerConsumer

public ProducerConsumer(int numThreads)

ProducerConsumer maintains input and output queues and a thread-pool of workers.

Parameters:

numThreads - Size of thread-pool to execute Workers.

Method Detail

addWorker

public void addWorker(WorkRequestProcessor<T,R> processor)

Add another worker that will consume WorkRequest<T> items from input queue, process each item using supplied processor, and for every processed item output WorkReport<R> to output queue.

Parameters:

processor - Processor implementing WorkRequestProcessor interface.

shutdown

public void shutdown()

Shutdown ProducerConsumer worker thread-pool without waiting for completion of any pending work.

getWorkCnt

public int getWorkCnt()

Returns number of pending ProducerConsumer items (submitted to input queue for processing via put() method but not yet consumed by take() or blockingTake().

Returns:

Number of items in ProducerConsumer (either pending for processing or waiting to be consumed).

hasWork

public boolean hasWork()

Returns true if there are items in ProducerConsumer that are either pending for processing or waiting to be consumed.

Returns:

True if there were more items put() to ProducerConsumer than consumed by take() or blockingTake().

put

public void put(WorkRequest<T> workRequest)

Blocking put workRequest to ProducerConsumer input queue.

Parameters:

workRequest - item to be processed.

take

public WorkReport<R> take()
                   throws InterruptedException

Blocking take from ProducerConsumer output queue that can be interrupted.

Returns:

item returned by processor's processItem().

Throws:

InterruptedException - if interrupted before an element becomes available.

blockingTake

public WorkReport<R> blockingTake()

Blocking take from ProducerConsumer output queue (catches exceptions and retries forever).

Returns:

item returned by processor's processItem().