org.nd4j.parameterserver.distributed.v2.util

Class MessageSplitter

java.lang.Object

org.nd4j.parameterserver.distributed.v2.util.MessageSplitter

public class MessageSplitter
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicLong	memoryUse
protected Map<String,ChunksTracker>	trackers

Constructor Summary

Constructors

Constructor and Description
MessageSplitter()

Method Summary

Modifier and Type	Method and Description
static MessageSplitter	getInstance() This method returns shared instance of MessageSplitter
<T extends VoidMessage> org.nd4j.common.primitives.Optional<T>	merge(@NonNull VoidChunk chunk) This method tries to merge using files tracker
<T extends VoidMessage> org.nd4j.common.primitives.Optional<T>	merge(@NonNull VoidChunk chunk, long memoryLimit)
void	release(String messageId) This method removes specified messageId from tracking
void	reset()
Collection<VoidChunk>	split(@NonNull VoidMessage message, int maxBytes) This method splits VoidMessage into chunks, and returns them as Collection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

trackers

protected Map<String,ChunksTracker> trackers

memoryUse

protected final AtomicLong memoryUse

Constructor Detail

MessageSplitter

public MessageSplitter()

Method Detail

getInstance

public static MessageSplitter getInstance()

This method returns shared instance of MessageSplitter

Returns:

split

public Collection<VoidChunk> split(@NonNull
                                   @NonNull VoidMessage message,
                                   int maxBytes)
                            throws IOException

This method splits VoidMessage into chunks, and returns them as Collection

Parameters:

message -

Returns:

Throws:

IOException

merge

public <T extends VoidMessage> org.nd4j.common.primitives.Optional<T> merge(@NonNull
                                                                            @NonNull VoidChunk chunk)

This method tries to merge using files tracker

Type Parameters:

T -

Parameters:

chunk -

Returns:

release

public void release(String messageId)

This method removes specified messageId from tracking

Parameters:

messageId -

merge

public <T extends VoidMessage> org.nd4j.common.primitives.Optional<T> merge(@NonNull
                                                                            @NonNull VoidChunk chunk,
                                                                            long memoryLimit)

Type Parameters:

T -

Parameters:

chunk -

memoryLimit -

Returns:

reset

public void reset()