org.apache.hadoop.hbase.master

Class SplitWALManager

java.lang.Object

org.apache.hadoop.hbase.master.SplitWALManager

@InterfaceAudience.Private
public class SplitWALManager
extends Object

Create SplitWALProcedure for each WAL which need to split. Manage the workers for each SplitWALProcedure. Total number of workers is (number of online servers) * (HBASE_SPLIT_WAL_MAX_SPLITTER). Helps assign and release workers for split tasks. Provide helper method to delete split WAL file and directory. The user can get the SplitWALProcedures via splitWALs(crashedServer, splitMeta) can get the files that need to split via getWALsToSplit(crashedServer, splitMeta) can delete the splitting WAL and directory via deleteSplitWAL(wal) and deleteSplitWAL(crashedServer) can check if splitting WALs of a crashed server is success via isSplitWALFinished(walPath) can acquire and release a worker for splitting WAL via acquireSplitWALWorker(procedure) and releaseSplitWALWorker(worker, scheduler) This class is to replace the zk-based WAL splitting related code, MasterWalManager, SplitLogManager, ZKSplitLog and ZKSplitLogManagerCoordination can be removed after we switch to procedure-based WAL splitting.

See Also:

for the original distributed split WAL manager.

Constructor Summary

Constructors

Constructor and Description
SplitWALManager(MasterServices master)

Method Summary

Modifier and Type	Method and Description
ServerName	acquireSplitWALWorker(Procedure<?> procedure) Acquire a split WAL worker
void	addUsedSplitWALWorker(ServerName worker) When master restart, there will be a new splitWorkerAssigner.
void	archive(String wal) Archive processed WAL
void	deleteWALDir(ServerName serverName)
List<org.apache.hadoop.fs.FileStatus>	getWALsToSplit(ServerName serverName, boolean splitMeta)
boolean	isSplitWALFinished(String walPath)
void	releaseSplitWALWorker(ServerName worker, MasterProcedureScheduler scheduler) After the worker finished the split WAL task, it will release the worker, and wake up all the suspend procedures in the ProcedureEvent
List<Procedure>	splitWALs(ServerName crashedServer, boolean splitMeta)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitWALManager

public SplitWALManager(MasterServices master)
                throws IOException

Throws:

IOException

Method Detail

splitWALs

public List<Procedure> splitWALs(ServerName crashedServer,
                                 boolean splitMeta)
                          throws IOException

Throws:

IOException

getWALsToSplit

public List<org.apache.hadoop.fs.FileStatus> getWALsToSplit(ServerName serverName,
                                                            boolean splitMeta)
                                                     throws IOException

Throws:

IOException

archive

public void archive(String wal)
             throws IOException

Archive processed WAL

Throws:

IOException

deleteWALDir

public void deleteWALDir(ServerName serverName)
                  throws IOException

Throws:

IOException

isSplitWALFinished

public boolean isSplitWALFinished(String walPath)
                           throws IOException

Throws:

IOException

acquireSplitWALWorker

public ServerName acquireSplitWALWorker(Procedure<?> procedure)
                                 throws ProcedureSuspendedException

Acquire a split WAL worker

Parameters:

procedure - split WAL task

Returns:

an available region server which could execute this task

Throws:

ProcedureSuspendedException - if there is no available worker, it will throw this exception to WAIT the procedure.

releaseSplitWALWorker

public void releaseSplitWALWorker(ServerName worker,
                                  MasterProcedureScheduler scheduler)

After the worker finished the split WAL task, it will release the worker, and wake up all the suspend procedures in the ProcedureEvent

Parameters:

worker - worker which is about to release

scheduler - scheduler which is to wake up the procedure event

addUsedSplitWALWorker

public void addUsedSplitWALWorker(ServerName worker)

When master restart, there will be a new splitWorkerAssigner. But if there are splitting WAL tasks running on the region server side, they will not be count by the new splitWorkerAssigner. Thus we should add the workers of running tasks to the assigner when we load the procedures from MasterProcWALs.

Parameters:

worker - region server which is executing a split WAL task