org.apache.accumulo.master.replication

Class UnorderedWorkAssigner

java.lang.Object

org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner

org.apache.accumulo.master.replication.UnorderedWorkAssigner

All Implemented Interfaces:

WorkAssigner

public class UnorderedWorkAssigner
extends DistributedWorkQueueWorkAssigner

Read work records from the replication table, create work entries for other nodes to complete.

Uses the DistributedWorkQueue to make the work available for any tserver. This approach does not consider the locality of the tabletserver performing the work in relation to the data being replicated (local HDFS blocks).

The implementation allows for multiple tservers to concurrently replicate data to peer(s), however it is possible that data for a table is replayed on the peer in a different order than the master. The SequentialWorkAssigner should be used if this must be guaranteed at the cost of replication throughput.

Field Summary

Fields inherited from class org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner

conf, conn, maxQueueSize, workQueue, zooCache

Constructor Summary

Constructors

Constructor and Description
UnorderedWorkAssigner()
UnorderedWorkAssigner(AccumuloConfiguration conf, Connector conn)

Method Summary

Modifier and Type	Method and Description
protected void	cleanupFinishedWork() Iterate over the queued work to remove entries that have been completed.
String	getName()
protected Set<String>	getQueuedWork()
protected Set<String>	getQueuedWork(ReplicationTarget target)
protected int	getQueueSize()
protected void	initializeQueuedWork() Initialize the queuedWork set with the work already sent out
protected boolean	queueWork(org.apache.hadoop.fs.Path path, ReplicationTarget target) Distribute the work for the given path with filename
protected void	removeQueuedWork(ReplicationTarget target, String queueKey) Remove the given work from the internal state
protected void	setQueuedWork(Set<String> queuedWork)
protected boolean	shouldQueueWork(ReplicationTarget target)

Methods inherited from class org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner

assignWork, configure, createWork, getConf, getConnector, getMaxQueueSize, getWorkQueue, getZooCache, initializeWorkQueue, isWorkRequired, setConf, setConnector, setMaxQueueSize, setWorkQueue, setZooCache

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnorderedWorkAssigner

public UnorderedWorkAssigner()

UnorderedWorkAssigner

public UnorderedWorkAssigner(AccumuloConfiguration conf,
                             Connector conn)

Method Detail

getName

public String getName()

getQueuedWork

protected Set<String> getQueuedWork()

setQueuedWork

protected void setQueuedWork(Set<String> queuedWork)

initializeQueuedWork

protected void initializeQueuedWork()

Initialize the queuedWork set with the work already sent out

Specified by:

initializeQueuedWork in class DistributedWorkQueueWorkAssigner

queueWork

protected boolean queueWork(org.apache.hadoop.fs.Path path,
                            ReplicationTarget target)

Distribute the work for the given path with filename

Specified by:

queueWork in class DistributedWorkQueueWorkAssigner

Parameters:

path - Path to the file being replicated

target - Target for the file to be replicated to

Returns:

True if the work was queued, false otherwise

cleanupFinishedWork

protected void cleanupFinishedWork()

Iterate over the queued work to remove entries that have been completed.

Specified by:

cleanupFinishedWork in class DistributedWorkQueueWorkAssigner

shouldQueueWork

protected boolean shouldQueueWork(ReplicationTarget target)

Specified by:

shouldQueueWork in class DistributedWorkQueueWorkAssigner

Returns:

Can replication work for the given ReplicationTarget be submitted to be worked on.

getQueueSize

protected int getQueueSize()

Specified by:

getQueueSize in class DistributedWorkQueueWorkAssigner

Returns:

the size of the queued work

getQueuedWork

protected Set<String> getQueuedWork(ReplicationTarget target)

Specified by:

getQueuedWork in class DistributedWorkQueueWorkAssigner

Parameters:

target - Target for the work

Returns:

Queued work for the given target

removeQueuedWork

protected void removeQueuedWork(ReplicationTarget target,
                                String queueKey)

Description copied from class: DistributedWorkQueueWorkAssigner

Remove the given work from the internal state

Specified by:

removeQueuedWork in class DistributedWorkQueueWorkAssigner