org.gridgain.grid.kernal.ggfs.hadoop

Class GridGgfsHadoopOutProc

java.lang.Object

org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopOutProc

All Implemented Interfaces:

GridGgfsHadoop, GridGgfsHadoopEx, GridGgfsHadoopIpcIoListener

public class GridGgfsHadoopOutProc
extends Object
implements GridGgfsHadoopEx, GridGgfsHadoopIpcIoListener

Communication with external process (TCP or shmem).

Constructor Summary

Constructors

Constructor and Description
GridGgfsHadoopOutProc(int port, String grid, String ggfs, org.apache.commons.logging.Log log) Constructor for shmem endpoint.
GridGgfsHadoopOutProc(String host, int port, String grid, String ggfs, org.apache.commons.logging.Log log) Constructor for TCP endpoint.

Method Summary

Methods

Modifier and Type	Method and Description
void	addEventListener(GridGgfsHadoopStreamDelegate desc, GridGgfsHadoopStreamEventListener lsnr) Adds event listener that will be invoked when connection with server is lost or remote error has occurred.
Collection<org.gridgain.grid.ggfs.GridGgfsBlockLocation>	affinity(org.gridgain.grid.ggfs.GridGgfsPath path, long start, long len) Command to get affinity for given path, offset and length.
GridGgfsHadoopStreamDelegate	append(org.gridgain.grid.ggfs.GridGgfsPath path, boolean create, Map<String,String> props) Open file for output appending data to the end of a file.
void	close(boolean force) Close connection.
void	closeStream(GridGgfsHadoopStreamDelegate desc) Close server stream.
org.gridgain.grid.ggfs.GridGgfsPathSummary	contentSummary(org.gridgain.grid.ggfs.GridGgfsPath path) Gets path summary.
GridGgfsHadoopStreamDelegate	create(org.gridgain.grid.ggfs.GridGgfsPath path, boolean overwrite, boolean colocate, int replication, long blockSize, Map<String,String> props) Command to create file and open it for output.
Boolean	delete(org.gridgain.grid.ggfs.GridGgfsPath path, boolean recursive) Command to delete given path.
void	flush(GridGgfsHadoopStreamDelegate delegate) Flush output stream.
GridGgfsStatus	fsStatus() Performs status request.
GridGgfsHandshakeResponse	handshake(String logDir) Perform handshake.
org.gridgain.grid.ggfs.GridGgfsFile	info(org.gridgain.grid.ggfs.GridGgfsPath path) Command to retrieve file info for some GGFS path.
Collection<org.gridgain.grid.ggfs.GridGgfsFile>	listFiles(org.gridgain.grid.ggfs.GridGgfsPath path) Command to get list of files in directory.
Collection<org.gridgain.grid.ggfs.GridGgfsPath>	listPaths(org.gridgain.grid.ggfs.GridGgfsPath path) Command to get directory listing.
Boolean	mkdirs(org.gridgain.grid.ggfs.GridGgfsPath path, Map<String,String> props) Command to create directories.
void	onClose() Callback invoked when the IO is being closed.
void	onError(long streamId, String errMsg) Callback invoked when remote error occurs.
GridGgfsHadoopStreamDelegate	open(org.gridgain.grid.ggfs.GridGgfsPath path) Command to open file for reading.
GridGgfsHadoopStreamDelegate	open(org.gridgain.grid.ggfs.GridGgfsPath path, int seqReadsBeforePrefetch) Command to open file for reading.
org.gridgain.grid.util.lang.GridPlainFuture<byte[]>	readData(GridGgfsHadoopStreamDelegate desc, long pos, int len, byte[] outBuf, int outOff, int outLen) Asynchronously reads specified amount of bytes from opened input stream.
void	removeEventListener(GridGgfsHadoopStreamDelegate desc) Removes event listener that will be invoked when connection with server is lost or remote error has occurred.
Boolean	rename(org.gridgain.grid.ggfs.GridGgfsPath src, org.gridgain.grid.ggfs.GridGgfsPath dest) Command to rename given path.
Boolean	setTimes(org.gridgain.grid.ggfs.GridGgfsPath path, long accessTime, long modificationTime) Sets last access time and last modification time for a file.
org.gridgain.grid.ggfs.GridGgfsFile	update(org.gridgain.grid.ggfs.GridGgfsPath path, Map<String,String> props) Command to update file properties.
void	writeData(GridGgfsHadoopStreamDelegate desc, byte[] data, int off, int len) Writes data to the stream with given streamId.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GridGgfsHadoopOutProc

public GridGgfsHadoopOutProc(String host,
                     int port,
                     String grid,
                     String ggfs,
                     org.apache.commons.logging.Log log)
                      throws IOException

Constructor for TCP endpoint.

Parameters:

host - Host.

port - Port.

grid - Grid name.

ggfs - GGFS name.

log - Client logger.

Throws:

IOException - If failed.

GridGgfsHadoopOutProc

public GridGgfsHadoopOutProc(int port,
                     String grid,
                     String ggfs,
                     org.apache.commons.logging.Log log)
                      throws IOException

Constructor for shmem endpoint.

Parameters:

port - Port.

grid - Grid name.

ggfs - GGFS name.

log - Client logger.

Throws:

IOException - If failed.

Method Detail

handshake

public GridGgfsHandshakeResponse handshake(String logDir)
                                    throws org.gridgain.grid.GridException

Perform handshake.

Specified by:

handshake in interface GridGgfsHadoop

Parameters:

logDir - Log directory.

Returns:

Future with handshake result.

Throws:

org.gridgain.grid.GridException - If failed.

close

public void close(boolean force)

Close connection.

Specified by:

close in interface GridGgfsHadoop

Parameters:

force - Force flag.

info

public org.gridgain.grid.ggfs.GridGgfsFile info(org.gridgain.grid.ggfs.GridGgfsPath path)
                                         throws org.gridgain.grid.GridException

Command to retrieve file info for some GGFS path.

Specified by:

info in interface GridGgfsHadoop

Parameters:

path - Path to get file info for.

Returns:

Future for info operation.

Throws:

org.gridgain.grid.GridException - If failed.

update

public org.gridgain.grid.ggfs.GridGgfsFile update(org.gridgain.grid.ggfs.GridGgfsPath path,
                                         Map<String,String> props)
                                           throws org.gridgain.grid.GridException

Command to update file properties.

Specified by:

update in interface GridGgfsHadoop

Parameters:

path - GGFS path to update properties.

props - Properties to update.

Returns:

Future for update operation.

Throws:

org.gridgain.grid.GridException - If failed.

setTimes

public Boolean setTimes(org.gridgain.grid.ggfs.GridGgfsPath path,
               long accessTime,
               long modificationTime)
                 throws org.gridgain.grid.GridException

Sets last access time and last modification time for a file.

Specified by:

setTimes in interface GridGgfsHadoop

Parameters:

path - Path to update times.

accessTime - Last access time to set.

modificationTime - Last modification time to set.

Throws:

org.gridgain.grid.GridException - If failed.

rename

public Boolean rename(org.gridgain.grid.ggfs.GridGgfsPath src,
             org.gridgain.grid.ggfs.GridGgfsPath dest)
               throws org.gridgain.grid.GridException

Command to rename given path.

Specified by:

rename in interface GridGgfsHadoop

Parameters:

src - Source path.

dest - Destination path.

Returns:

Future for rename operation.

Throws:

org.gridgain.grid.GridException - If failed.

delete

public Boolean delete(org.gridgain.grid.ggfs.GridGgfsPath path,
             boolean recursive)
               throws org.gridgain.grid.GridException

Command to delete given path.

Specified by:

delete in interface GridGgfsHadoop

Parameters:

path - Path to delete.

recursive - True if deletion is recursive.

Returns:

Future for delete operation.

Throws:

org.gridgain.grid.GridException - If failed.

affinity

public Collection<org.gridgain.grid.ggfs.GridGgfsBlockLocation> affinity(org.gridgain.grid.ggfs.GridGgfsPath path,
                                                                long start,
                                                                long len)
                                                                  throws org.gridgain.grid.GridException

Command to get affinity for given path, offset and length.

Specified by:

affinity in interface GridGgfsHadoop

Parameters:

path - Path to get affinity for.

start - Start position (offset).

len - Data length.

Returns:

Future for affinity command.

Throws:

org.gridgain.grid.GridException - If failed.

contentSummary

public org.gridgain.grid.ggfs.GridGgfsPathSummary contentSummary(org.gridgain.grid.ggfs.GridGgfsPath path)
                                                          throws org.gridgain.grid.GridException

Gets path summary.

Specified by:

contentSummary in interface GridGgfsHadoop

Parameters:

path - Path to get summary for.

Returns:

Future that will be completed when summary is received.

Throws:

org.gridgain.grid.GridException - If failed.

mkdirs

public Boolean mkdirs(org.gridgain.grid.ggfs.GridGgfsPath path,
             Map<String,String> props)
               throws org.gridgain.grid.GridException

Command to create directories.

Specified by:

mkdirs in interface GridGgfsHadoop

Parameters:

path - Path to create.

Returns:

Future for mkdirs operation.

Throws:

org.gridgain.grid.GridException - If failed.

listFiles

public Collection<org.gridgain.grid.ggfs.GridGgfsFile> listFiles(org.gridgain.grid.ggfs.GridGgfsPath path)
                                                          throws org.gridgain.grid.GridException

Command to get list of files in directory.

Specified by:

listFiles in interface GridGgfsHadoop

Parameters:

path - Path to list.

Returns:

Future for listFiles operation.

Throws:

org.gridgain.grid.GridException - If failed.

listPaths

public Collection<org.gridgain.grid.ggfs.GridGgfsPath> listPaths(org.gridgain.grid.ggfs.GridGgfsPath path)
                                                          throws org.gridgain.grid.GridException

Command to get directory listing.

Specified by:

listPaths in interface GridGgfsHadoop

Parameters:

path - Path to list.

Returns:

Future for listPaths operation.

Throws:

org.gridgain.grid.GridException - If failed.

fsStatus

public GridGgfsStatus fsStatus()
                        throws org.gridgain.grid.GridException

Performs status request.

Specified by:

fsStatus in interface GridGgfsHadoop

Returns:

Status response.

Throws:

org.gridgain.grid.GridException - If failed.

open

public GridGgfsHadoopStreamDelegate open(org.gridgain.grid.ggfs.GridGgfsPath path)
                                  throws org.gridgain.grid.GridException

Command to open file for reading.

Specified by:

open in interface GridGgfsHadoop

Parameters:

path - File path to open.

Returns:

Future for open operation.

Throws:

org.gridgain.grid.GridException - If failed.

open

public GridGgfsHadoopStreamDelegate open(org.gridgain.grid.ggfs.GridGgfsPath path,
                                int seqReadsBeforePrefetch)
                                  throws org.gridgain.grid.GridException

Command to open file for reading.

Specified by:

open in interface GridGgfsHadoop

Parameters:

path - File path to open.

Returns:

Future for open operation.

Throws:

org.gridgain.grid.GridException - If failed.

create

public GridGgfsHadoopStreamDelegate create(org.gridgain.grid.ggfs.GridGgfsPath path,
                                  boolean overwrite,
                                  boolean colocate,
                                  int replication,
                                  long blockSize,
                                  @Nullable
                                  Map<String,String> props)
                                    throws org.gridgain.grid.GridException

Command to create file and open it for output.

Specified by:

create in interface GridGgfsHadoop

Parameters:

path - Path to file.

overwrite - If true then old file contents will be lost.

colocate - If true and called on data node, file will be written on that node.

replication - Replication factor.

props - File properties for creation.

Returns:

Stream descriptor.

Throws:

org.gridgain.grid.GridException - If failed.

append

public GridGgfsHadoopStreamDelegate append(org.gridgain.grid.ggfs.GridGgfsPath path,
                                  boolean create,
                                  @Nullable
                                  Map<String,String> props)
                                    throws org.gridgain.grid.GridException

Open file for output appending data to the end of a file.

Specified by:

append in interface GridGgfsHadoop

Parameters:

path - Path to file.

create - If true, file will be created if does not exist.

props - File properties.

Returns:

Stream descriptor.

Throws:

org.gridgain.grid.GridException - If failed.

readData

public org.gridgain.grid.util.lang.GridPlainFuture<byte[]> readData(GridGgfsHadoopStreamDelegate desc,
                                                           long pos,
                                                           int len,
                                                           @Nullable
                                                           byte[] outBuf,
                                                           int outOff,
                                                           int outLen)

Asynchronously reads specified amount of bytes from opened input stream.

Specified by:

readData in interface GridGgfsHadoopEx

Parameters:

desc - Stream delegate.

pos - Position to read from.

len - Data length to read.

outBuf - Optional output buffer. If buffer length is less then len, all remaining bytes will be read into new allocated buffer of length {len - outBuf.length} and this buffer will be the result of read future.

outOff - Output offset.

outLen - Output length.

Returns:

Read data.

writeData

public void writeData(GridGgfsHadoopStreamDelegate desc,
             byte[] data,
             int off,
             int len)
               throws IOException

Writes data to the stream with given streamId. This method does not return any future since no response to write request is sent.

Specified by:

writeData in interface GridGgfsHadoopEx

Parameters:

desc - Stream delegate.

data - Data to write.

off - Offset.

len - Length.

Throws:

IOException - If failed.

flush

public void flush(GridGgfsHadoopStreamDelegate delegate)
           throws IOException

Flush output stream.

Specified by:

flush in interface GridGgfsHadoopEx

Parameters:

delegate - Stream delegate.

Throws:

IOException - If failed.

closeStream

public void closeStream(GridGgfsHadoopStreamDelegate desc)
                 throws IOException

Close server stream.

Specified by:

closeStream in interface GridGgfsHadoopEx

Parameters:

desc - Stream delegate.

Throws:

IOException - If failed.

addEventListener

public void addEventListener(GridGgfsHadoopStreamDelegate desc,
                    GridGgfsHadoopStreamEventListener lsnr)

Adds event listener that will be invoked when connection with server is lost or remote error has occurred. If connection is closed already, callback will be invoked synchronously inside this method.

Specified by:

addEventListener in interface GridGgfsHadoopEx

Parameters:

desc - Stream delegate.

lsnr - Event listener.

removeEventListener

public void removeEventListener(GridGgfsHadoopStreamDelegate desc)

Removes event listener that will be invoked when connection with server is lost or remote error has occurred.

Specified by:

removeEventListener in interface GridGgfsHadoopEx

Parameters:

desc - Stream delegate.

onClose

public void onClose()

Callback invoked when the IO is being closed.

Specified by:

onClose in interface GridGgfsHadoopIpcIoListener

onError

public void onError(long streamId,
           String errMsg)

Callback invoked when remote error occurs.

Specified by:

onError in interface GridGgfsHadoopIpcIoListener

Parameters:

streamId - Stream ID.

errMsg - Error message.