org.apache.hadoop.hbase.replication

Class TableBasedReplicationQueuesClientImpl

java.lang.Object

org.apache.hadoop.hbase.replication.TableBasedReplicationQueuesClientImpl

All Implemented Interfaces:

ReplicationQueuesClient

@InterfaceAudience.Private
public class TableBasedReplicationQueuesClientImpl
extends Object
implements ReplicationQueuesClient

Implements the ReplicationQueuesClient interface on top of the Replication Table. It utilizes the ReplicationTableBase to access the Replication Table.

Field Summary

Fields

Modifier and Type	Field and Description
protected Abortable	abortable
static byte[]	CF_QUEUE
static byte[]	COL_QUEUE_OWNER
static byte[]	COL_QUEUE_OWNER_HISTORY
protected org.apache.hadoop.conf.Configuration	conf
static String	QUEUE_HISTORY_DELIMITER
static TableName	REPLICATION_TABLE_NAME Name of the HBase Table used for tracking replication
static String	ROW_KEY_DELIMITER

Constructor Summary

Constructors

Constructor and Description
TableBasedReplicationQueuesClientImpl(org.apache.hadoop.conf.Configuration conf, Abortable abortable)
TableBasedReplicationQueuesClientImpl(ReplicationQueuesClientArguments args)

Method Summary

Modifier and Type	Method and Description
protected String	buildClaimedQueueHistory(String originalHistory, String oldServer) Creates a "|" delimited record of the queue's past region server owners.
protected String	buildQueueRowKey(String serverName, String queueId) Build the row key for the given queueId.
List<String>	getAllPeersFromHFileRefsQueue() Get list of all peers from hfile reference queue.
List<String>	getAllQueues(String serverName) Get a list of all queues for the specified region server.
Set<String>	getAllWALs() Load all wals in all replication queues from ZK.
int	getHFileRefsNodeChangeVersion() Get the change version number of replication hfile references node.
boolean	getInitializationStatus() Get whether the Replication Table has been successfully initialized yet
List<String>	getListOfReplicators() Get a list of all region servers that have outstanding replication queues.
protected List<String>	getLogsInQueue(byte[] rowKey)
List<String>	getLogsInQueue(String serverName, String queueId) Get a list of all WALs in the given queue on the given region server.
protected Table	getOrBlockOnReplicationTable() Attempts to acquire the Replication Table.
protected ResultScanner	getQueuesBelongingToServer(String server) Get the queue id's and meta data (Owner and History) for the queues belonging to the named server
protected String	getRawQueueIdFromRowKey(String rowKey) Parse the original queueId from a row key
List<String>	getReplicableHFiles(String peerId) Get a list of all hfile references in the given peer.
void	init() Initialize the replication queue client interface.
protected byte[]	queueIdToRowKey(String serverName, String queueId) Returns a queue's row key given either its raw or reclaimed queueId
protected List<String>	readWALsFromResult(Result queue) Read all of the WAL's from a queue into a list

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REPLICATION_TABLE_NAME

public static final TableName REPLICATION_TABLE_NAME

Name of the HBase Table used for tracking replication

CF_QUEUE

public static final byte[] CF_QUEUE

COL_QUEUE_OWNER

public static final byte[] COL_QUEUE_OWNER

COL_QUEUE_OWNER_HISTORY

public static final byte[] COL_QUEUE_OWNER_HISTORY

ROW_KEY_DELIMITER

public static final String ROW_KEY_DELIMITER

See Also:

Constant Field Values

QUEUE_HISTORY_DELIMITER

public static final String QUEUE_HISTORY_DELIMITER

See Also:

Constant Field Values

conf

protected final org.apache.hadoop.conf.Configuration conf

abortable

protected final Abortable abortable

Constructor Detail

TableBasedReplicationQueuesClientImpl

public TableBasedReplicationQueuesClientImpl(ReplicationQueuesClientArguments args)
                                      throws IOException

Throws:

IOException

TableBasedReplicationQueuesClientImpl

public TableBasedReplicationQueuesClientImpl(org.apache.hadoop.conf.Configuration conf,
                                             Abortable abortable)
                                      throws IOException

Throws:

IOException

Method Detail

init

public void init()
          throws ReplicationException

Description copied from interface: ReplicationQueuesClient

Initialize the replication queue client interface.

Specified by:

init in interface ReplicationQueuesClient

Throws:

ReplicationException

getListOfReplicators

public List<String> getListOfReplicators()

Get a list of all region servers that have outstanding replication queues. These servers could be alive, dead or from a previous run of the cluster.

Specified by:

getListOfReplicators in interface ReplicationQueuesClient

Returns:

a list of server names

getLogsInQueue

public List<String> getLogsInQueue(String serverName,
                                   String queueId)

Description copied from interface: ReplicationQueuesClient

Get a list of all WALs in the given queue on the given region server.

Specified by:

getLogsInQueue in interface ReplicationQueuesClient

Parameters:

serverName - the server name of the region server that owns the queue

queueId - a String that identifies the queue

Returns:

a list of WALs, null if this region server is dead and has no outstanding queues

getAllQueues

public List<String> getAllQueues(String serverName)

Description copied from interface: ReplicationQueuesClient

Get a list of all queues for the specified region server.

Specified by:

getAllQueues in interface ReplicationQueuesClient

Parameters:

serverName - the server name of the region server that owns the set of queues

Returns:

a list of queueIds, null if this region server is not a replicator.

getAllWALs

public Set<String> getAllWALs()

Description copied from interface: ReplicationQueuesClient

Load all wals in all replication queues from ZK. This method guarantees to return a snapshot which contains all WALs in the zookeeper at the start of this call even there is concurrent queue failover. However, some newly created WALs during the call may not be included.

Specified by:

getAllWALs in interface ReplicationQueuesClient

getHFileRefsNodeChangeVersion

public int getHFileRefsNodeChangeVersion()
                                  throws org.apache.zookeeper.KeeperException

Description copied from interface: ReplicationQueuesClient

Get the change version number of replication hfile references node. This can be used as optimistic locking to get a consistent snapshot of the replication queues of hfile references.

Specified by:

getHFileRefsNodeChangeVersion in interface ReplicationQueuesClient

Returns:

change version number of hfile references node

Throws:

org.apache.zookeeper.KeeperException

getAllPeersFromHFileRefsQueue

public List<String> getAllPeersFromHFileRefsQueue()
                                           throws org.apache.zookeeper.KeeperException

Description copied from interface: ReplicationQueuesClient

Get list of all peers from hfile reference queue.

Specified by:

getAllPeersFromHFileRefsQueue in interface ReplicationQueuesClient

Returns:

a list of peer ids

Throws:

org.apache.zookeeper.KeeperException - zookeeper exception

getReplicableHFiles

public List<String> getReplicableHFiles(String peerId)
                                 throws org.apache.zookeeper.KeeperException

Description copied from interface: ReplicationQueuesClient

Get a list of all hfile references in the given peer.

Specified by:

getReplicableHFiles in interface ReplicationQueuesClient

Parameters:

peerId - a String that identifies the peer

Returns:

a list of hfile references, null if not found any

Throws:

org.apache.zookeeper.KeeperException - zookeeper exception

getInitializationStatus

public boolean getInitializationStatus()

Get whether the Replication Table has been successfully initialized yet

Returns:

whether the Replication Table is initialized

buildQueueRowKey

protected String buildQueueRowKey(String serverName,
                                  String queueId)

Build the row key for the given queueId. This will uniquely identify it from all other queues in the cluster.

Parameters:

serverName - The owner of the queue

queueId - String identifier of the queue

Returns:

String representation of the queue's row key

getRawQueueIdFromRowKey

protected String getRawQueueIdFromRowKey(String rowKey)

Parse the original queueId from a row key

Parameters:

rowKey - String representation of a queue's row key

Returns:

the original queueId

queueIdToRowKey

protected byte[] queueIdToRowKey(String serverName,
                                 String queueId)

Returns a queue's row key given either its raw or reclaimed queueId

Parameters:

queueId - queueId of the queue

Returns:

byte representation of the queue's row key

buildClaimedQueueHistory

protected String buildClaimedQueueHistory(String originalHistory,
                                          String oldServer)

Creates a "|" delimited record of the queue's past region server owners.

Parameters:

originalHistory - the queue's original owner history

oldServer - the name of the server that used to own the queue

Returns:

the queue's new owner history

getLogsInQueue

protected List<String> getLogsInQueue(byte[] rowKey)

readWALsFromResult

protected List<String> readWALsFromResult(Result queue)

Read all of the WAL's from a queue into a list

Parameters:

queue - HBase query result containing the queue

Returns:

a list of all the WAL filenames

getQueuesBelongingToServer

protected ResultScanner getQueuesBelongingToServer(String server)
                                            throws IOException

Get the queue id's and meta data (Owner and History) for the queues belonging to the named server

Parameters:

server - name of the server

Returns:

a ResultScanner over the QueueIds belonging to the server

Throws:

IOException

getOrBlockOnReplicationTable

protected Table getOrBlockOnReplicationTable()
                                      throws IOException

Attempts to acquire the Replication Table. This operation will block until it is assigned by the CreateReplicationWorker thread. It is up to the caller of this method to close the returned Table

Returns:

the Replication Table when it is created

Throws:

IOException