org.gridgain.grid.kernal.processors.hadoop.taskexecutor.external.communication

Class GridHadoopExternalCommunication

java.lang.Object

org.gridgain.grid.kernal.processors.hadoop.taskexecutor.external.communication.GridHadoopExternalCommunication

public class GridHadoopExternalCommunication
extends Object

Hadoop external communication class.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GridHadoopExternalCommunication.ProcessHandshakeMessage Process ID message.

Field Summary

Fields

Modifier and Type	Field and Description
static long	DFLT_CONN_TIMEOUT Default connection timeout (value is 1000ms).
static long	DFLT_MAX_CONN_TIMEOUT Default Maximum connection timeout (value is 600,000ms).
static int	DFLT_MSG_QUEUE_LIMIT Default message queue limit per connection (for incoming and outgoing .
static int	DFLT_PORT Default port which node sets listener to (value is 47100).
static int	DFLT_PORT_RANGE Default local port range (value is 100).
static int	DFLT_RECONNECT_CNT Default reconnect attempts count (value is 10).
static int	DFLT_SELECTORS_CNT Default count of selectors for TCP server equals to "Math.min(4, Runtime.getRuntime().availableProcessors())".
static boolean	DFLT_TCP_NODELAY Default value for TCP_NODELAY socket option (value is true).
static String	OUT_OF_RESOURCES_TCP_MSG IPC error message.

Constructor Summary

Constructors

Constructor and Description
GridHadoopExternalCommunication(UUID parentNodeId, UUID procId, org.gridgain.grid.marshaller.GridMarshaller marsh, org.gridgain.grid.logger.GridLogger log, ExecutorService execSvc, String gridName)

Method Summary

Methods

Modifier and Type	Method and Description
protected GridHadoopCommunicationClient	createNioClient(GridHadoopProcessDescriptor desc)
protected GridHadoopCommunicationClient	createShmemClient(GridHadoopProcessDescriptor desc, int port)
protected GridHadoopCommunicationClient	createTcpClient(GridHadoopProcessDescriptor desc) Establish TCP connection to remote hadoop process and returns client.
long	getConnectTimeout()
int	getLocalPort() Gets local port for socket binding.
int	getLocalPortRange()
long	getMaxConnectTimeout() Gets maximum connection timeout.
int	getMessageQueueLimit()
int	getOutboundMessagesQueueSize()
int	getReconnectCount()
int	getSelectorsCount()
int	getSharedMemoryPort() Gets shared memory port to accept incoming connections.
int	getSocketReceiveBuffer()
int	getSocketSendBuffer()
boolean	isDirectBuffer()
boolean	isTcpNoDelay()
GridHadoopProcessDescriptor	localProcessDescriptor() Gets local process descriptor.
protected void	notifyListener(GridHadoopProcessDescriptor desc, GridHadoopMessage msg)
void	sendMessage(GridHadoopProcessDescriptor desc, GridHadoopMessage msg)
void	setConnectTimeout(long connTimeout) Sets connect timeout used when establishing connection with remote nodes.
void	setDirectBuffer(boolean directBuf) Sets flag to allocate direct or heap buffer in SPI.
void	setListener(GridHadoopMessageListener lsnr) Sets Hadoop communication message listener.
void	setLocalPort(int locPort) Sets local port for socket binding.
void	setLocalPortRange(int locPortRange) Sets local port range for local host ports (value must greater than or equal to 0).
void	setMaxConnectTimeout(long maxConnTimeout) Sets maximum connect timeout.
void	setMessageQueueLimit(int msgQueueLimit) Sets message queue limit for incoming and outgoing messages.
void	setReconnectCount(int reconCnt) Sets maximum number of reconnect attempts used when establishing connection with remote nodes.
void	setSelectorsCount(int selectorsCnt) Sets the count of selectors te be used in TCP server.
void	setSharedMemoryPort(int shmemPort) Sets local port to accept shared memory connections.
void	setSocketReceiveBuffer(int sockRcvBuf) Sets receive buffer size for sockets created or accepted by this SPI.
void	setSocketSendBuffer(int sockSndBuf) Sets send buffer size for sockets created or accepted by this SPI.
void	setTcpNoDelay(boolean tcpNoDelay) Sets value for TCP_NODELAY socket option.
void	start() Starts communication.
void	stop()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

OUT_OF_RESOURCES_TCP_MSG

public static final String OUT_OF_RESOURCES_TCP_MSG

IPC error message.

See Also:

Constant Field Values

DFLT_PORT

public static final int DFLT_PORT

Default port which node sets listener to (value is 47100).

See Also:

Constant Field Values

DFLT_CONN_TIMEOUT

public static final long DFLT_CONN_TIMEOUT

Default connection timeout (value is 1000ms).

See Also:

Constant Field Values

DFLT_MAX_CONN_TIMEOUT

public static final long DFLT_MAX_CONN_TIMEOUT

Default Maximum connection timeout (value is 600,000ms).

See Also:

Constant Field Values

DFLT_RECONNECT_CNT

public static final int DFLT_RECONNECT_CNT

Default reconnect attempts count (value is 10).

See Also:

Constant Field Values

DFLT_MSG_QUEUE_LIMIT

public static final int DFLT_MSG_QUEUE_LIMIT

Default message queue limit per connection (for incoming and outgoing .

See Also:

Constant Field Values

DFLT_SELECTORS_CNT

public static final int DFLT_SELECTORS_CNT

Default count of selectors for TCP server equals to "Math.min(4, Runtime.getRuntime().availableProcessors())".

See Also:

Constant Field Values

DFLT_PORT_RANGE

public static final int DFLT_PORT_RANGE

Default local port range (value is 100). See setLocalPortRange(int) for details.

See Also:

Constant Field Values

DFLT_TCP_NODELAY

public static final boolean DFLT_TCP_NODELAY

Default value for TCP_NODELAY socket option (value is true).

See Also:

Constant Field Values

Constructor Detail

GridHadoopExternalCommunication

public GridHadoopExternalCommunication(UUID parentNodeId,
                               UUID procId,
                               org.gridgain.grid.marshaller.GridMarshaller marsh,
                               org.gridgain.grid.logger.GridLogger log,
                               ExecutorService execSvc,
                               String gridName)

Parameters:

parentNodeId - Parent node ID.

procId - Process ID.

marsh - Marshaller to use.

log - Logger.

execSvc - Executor service for message notification.

gridName - Grid name.

Method Detail

setLocalPort

public void setLocalPort(int locPort)

Sets local port for socket binding.

If not provided, default value is DFLT_PORT.

Parameters:

locPort - Port number.

getLocalPort

public int getLocalPort()

Gets local port for socket binding.

Returns:

Local port.

setLocalPortRange

public void setLocalPortRange(int locPortRange)

Sets local port range for local host ports (value must greater than or equal to 0). If provided local port (see setLocalPort(int)} is occupied, implementation will try to increment the port number for as long as it is less than initial value plus this range.

If port range value is 0, then implementation will try bind only to the port provided by setLocalPort(int) method and fail if binding to this port did not succeed.

Local port range is very useful during development when more than one grid nodes need to run on the same physical machine.

If not provided, default value is DFLT_PORT_RANGE.

Parameters:

locPortRange - New local port range.

getLocalPortRange

public int getLocalPortRange()

Returns:

Local port range.

setSharedMemoryPort

public void setSharedMemoryPort(int shmemPort)

Sets local port to accept shared memory connections.

If set to -1 shared memory communication will be disabled.

If not provided, shared memory is disabled.

Parameters:

shmemPort - Port number.

getSharedMemoryPort

public int getSharedMemoryPort()

Gets shared memory port to accept incoming connections.

Returns:

Shared memory port.

setConnectTimeout

public void setConnectTimeout(long connTimeout)

Sets connect timeout used when establishing connection with remote nodes.

0 is interpreted as infinite timeout.

If not provided, default value is DFLT_CONN_TIMEOUT.

Parameters:

connTimeout - Connect timeout.

getConnectTimeout

public long getConnectTimeout()

Returns:

Connection timeout.

setMaxConnectTimeout

public void setMaxConnectTimeout(long maxConnTimeout)

Sets maximum connect timeout. If handshake is not established within connect timeout, then SPI tries to repeat handshake procedure with increased connect timeout. Connect timeout can grow till maximum timeout value, if maximum timeout value is reached then the handshake is considered as failed.

0 is interpreted as infinite timeout.

If not provided, default value is DFLT_MAX_CONN_TIMEOUT.

Parameters:

maxConnTimeout - Maximum connect timeout.

getMaxConnectTimeout

public long getMaxConnectTimeout()

Gets maximum connection timeout.

Returns:

Maximum connection timeout.

setReconnectCount

public void setReconnectCount(int reconCnt)

Sets maximum number of reconnect attempts used when establishing connection with remote nodes.

If not provided, default value is DFLT_RECONNECT_CNT.

Parameters:

reconCnt - Maximum number of reconnection attempts.

getReconnectCount

public int getReconnectCount()

Returns:

Reconnect count.

setDirectBuffer

public void setDirectBuffer(boolean directBuf)

Sets flag to allocate direct or heap buffer in SPI. If value is true, then SPI will use ByteBuffer.allocateDirect(int) call. Otherwise, SPI will use ByteBuffer.allocate(int) call.

If not provided, default value is true.

Parameters:

directBuf - Flag indicates to allocate direct or heap buffer in SPI.

isDirectBuffer

public boolean isDirectBuffer()

Returns:

Direct buffer flag.

setSelectorsCount

public void setSelectorsCount(int selectorsCnt)

Sets the count of selectors te be used in TCP server.

If not provided, default value is DFLT_SELECTORS_CNT.

Parameters:

selectorsCnt - Selectors count.

getSelectorsCount

public int getSelectorsCount()

Returns:

Number of selectors to use.

setTcpNoDelay

public void setTcpNoDelay(boolean tcpNoDelay)

Sets value for TCP_NODELAY socket option. Each socket will be opened using provided value.

Setting this option to true disables Nagle's algorithm for socket decreasing latency and delivery time for small messages.

For systems that work under heavy network load it is advisable to set this value to false.

If not provided, default value is DFLT_TCP_NODELAY.

Parameters:

tcpNoDelay - True to disable TCP delay.

isTcpNoDelay

public boolean isTcpNoDelay()

Returns:

TCP_NO_DELAY flag.

setSocketReceiveBuffer

public void setSocketReceiveBuffer(int sockRcvBuf)

Sets receive buffer size for sockets created or accepted by this SPI.

If not provided, default is 0 which leaves buffer unchanged after socket creation (OS defaults).

Parameters:

sockRcvBuf - Socket receive buffer size.

getSocketReceiveBuffer

public int getSocketReceiveBuffer()

Returns:

Socket receive buffer size.

setSocketSendBuffer

public void setSocketSendBuffer(int sockSndBuf)

Sets send buffer size for sockets created or accepted by this SPI.

If not provided, default is 0 which leaves the buffer unchanged after socket creation (OS defaults).

Parameters:

sockSndBuf - Socket send buffer size.

getSocketSendBuffer

public int getSocketSendBuffer()

Returns:

Socket send buffer size.

setMessageQueueLimit

public void setMessageQueueLimit(int msgQueueLimit)

Sets message queue limit for incoming and outgoing messages.

When set to positive number send queue is limited to the configured value. 0 disables the size limitations.

If not provided, default is DFLT_MSG_QUEUE_LIMIT.

Parameters:

msgQueueLimit - Send queue size limit.

getMessageQueueLimit

public int getMessageQueueLimit()

Returns:

Message queue size limit.

setListener

public void setListener(GridHadoopMessageListener lsnr)

Sets Hadoop communication message listener.

Parameters:

lsnr - Message listener.

getOutboundMessagesQueueSize

public int getOutboundMessagesQueueSize()

Returns:

Outbound message queue size.

start

public void start()
           throws org.gridgain.grid.GridException

Starts communication.

Throws:

org.gridgain.grid.GridException - If failed.

localProcessDescriptor

public GridHadoopProcessDescriptor localProcessDescriptor()

Gets local process descriptor.

Returns:

Local process descriptor.

stop

public void stop()
          throws org.gridgain.grid.GridException

Throws:

org.gridgain.grid.GridException

sendMessage

public void sendMessage(GridHadoopProcessDescriptor desc,
               GridHadoopMessage msg)
                 throws org.gridgain.grid.GridException

Throws:

org.gridgain.grid.GridException

createNioClient

@Nullable
protected GridHadoopCommunicationClient createNioClient(GridHadoopProcessDescriptor desc)
                                                 throws org.gridgain.grid.GridException

Parameters:

desc - Process descriptor.

Returns:

Client.

Throws:

org.gridgain.grid.GridException - If failed.

createShmemClient

@Nullable
protected GridHadoopCommunicationClient createShmemClient(GridHadoopProcessDescriptor desc,
                                                       int port)
                                                   throws org.gridgain.grid.GridException

Parameters:

desc - Process descriptor.

port - Port.

Returns:

Client.

Throws:

org.gridgain.grid.GridException - If failed.

createTcpClient

protected GridHadoopCommunicationClient createTcpClient(GridHadoopProcessDescriptor desc)
                                                 throws org.gridgain.grid.GridException

Establish TCP connection to remote hadoop process and returns client.

Parameters:

desc - Process descriptor.

Returns:

Client.

Throws:

org.gridgain.grid.GridException - If failed.

notifyListener

protected void notifyListener(GridHadoopProcessDescriptor desc,
                  GridHadoopMessage msg)

Parameters:

desc - Sender process descriptor.

msg - Communication message.

toString

public String toString()

Overrides:

toString in class Object