org.gridgain.grid.kernal.processors.hadoop.shuffle

Class GridHadoopShuffleJob<T>

java.lang.Object

org.gridgain.grid.kernal.processors.hadoop.shuffle.GridHadoopShuffleJob<T>

All Implemented Interfaces:

AutoCloseable

public class GridHadoopShuffleJob<T>
extends Object
implements AutoCloseable

Shuffle job.

Field Summary

Fields

Modifier and Type	Field and Description
protected ConcurrentMap<Long,org.gridgain.grid.lang.GridBiTuple<GridHadoopShuffleMessage,org.gridgain.grid.util.future.GridFutureAdapterEx<?>>>	sentMsgs

Constructor Summary

Constructors

Constructor and Description
GridHadoopShuffleJob(T locReduceAddr, org.gridgain.grid.logger.GridLogger log, org.gridgain.grid.hadoop.GridHadoopJob job, org.gridgain.grid.util.offheap.unsafe.GridUnsafeMemory mem, int reducers, boolean hasLocMappers)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
org.gridgain.grid.GridFuture<?>	flush()
boolean	initializeReduceAddresses(T[] reduceAddresses)
org.gridgain.grid.hadoop.GridHadoopTaskInput	input(org.gridgain.grid.hadoop.GridHadoopTaskInfo taskInfo)
void	onShuffleAck(GridHadoopShuffleAck ack)
void	onShuffleMessage(GridHadoopShuffleMessage msg)
org.gridgain.grid.hadoop.GridHadoopTaskOutput	output(org.gridgain.grid.hadoop.GridHadoopTaskInfo taskInfo)
boolean	reducersInitialized()
void	startSending(String gridName, org.gridgain.grid.util.lang.GridInClosure2X<T,GridHadoopShuffleMessage> io)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sentMsgs

protected ConcurrentMap<Long,org.gridgain.grid.lang.GridBiTuple<GridHadoopShuffleMessage,org.gridgain.grid.util.future.GridFutureAdapterEx<?>>> sentMsgs

Constructor Detail

GridHadoopShuffleJob

public GridHadoopShuffleJob(T locReduceAddr,
                    org.gridgain.grid.logger.GridLogger log,
                    org.gridgain.grid.hadoop.GridHadoopJob job,
                    org.gridgain.grid.util.offheap.unsafe.GridUnsafeMemory mem,
                    int reducers,
                    boolean hasLocMappers)
                     throws org.gridgain.grid.GridException

Parameters:

locReduceAddr - Local reducer address.

log - Logger.

job - Job.

mem - Memory.

reducers - Number of reducers for job.

hasLocMappers - True if

Throws:

org.gridgain.grid.GridException

Method Detail

initializeReduceAddresses

public boolean initializeReduceAddresses(T[] reduceAddresses)

Parameters:

reduceAddresses - Addresses of reducers.

Returns:

True if addresses were initialized by this call.

reducersInitialized

public boolean reducersInitialized()

Returns:

True if reducers addresses were initialized.

startSending

public void startSending(String gridName,
                org.gridgain.grid.util.lang.GridInClosure2X<T,GridHadoopShuffleMessage> io)

Parameters:

gridName - Grid name.

io - IO Closure for sending messages.

onShuffleMessage

public void onShuffleMessage(GridHadoopShuffleMessage msg)
                      throws org.gridgain.grid.GridException

Parameters:

msg - Message.

Throws:

org.gridgain.grid.GridException - Exception.

onShuffleAck

public void onShuffleAck(GridHadoopShuffleAck ack)

Parameters:

ack - Shuffle ack.

close

public void close()
           throws org.gridgain.grid.GridException

Specified by:

close in interface AutoCloseable

Throws:

org.gridgain.grid.GridException

flush

public org.gridgain.grid.GridFuture<?> flush()
                                      throws org.gridgain.grid.GridException

Returns:

Future.

Throws:

org.gridgain.grid.GridException

output

public org.gridgain.grid.hadoop.GridHadoopTaskOutput output(org.gridgain.grid.hadoop.GridHadoopTaskInfo taskInfo)
                                                     throws org.gridgain.grid.GridException

Parameters:

taskInfo - Task info.

Returns:

Output.

Throws:

org.gridgain.grid.GridException - If failed.

input

public org.gridgain.grid.hadoop.GridHadoopTaskInput input(org.gridgain.grid.hadoop.GridHadoopTaskInfo taskInfo)
                                                   throws org.gridgain.grid.GridException

Parameters:

taskInfo - Task info.

Returns:

Input.

Throws:

org.gridgain.grid.GridException - If failed.