org.apache.hadoop.hbase.master

Class MasterWalManager

java.lang.Object

org.apache.hadoop.hbase.master.MasterWalManager

@InterfaceAudience.Private
public class MasterWalManager
extends Object

This class abstracts a bunch of operations the HMaster needs when splitting log files e.g. finding log files, dirs etc.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.PathFilter	NON_META_FILTER Filter *out* WAL files that are for the hbase:meta Region; i.e.

Constructor Summary

Constructors

Constructor and Description
MasterWalManager(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, MasterServices services)
MasterWalManager(MasterServices services)

Method Summary

Modifier and Type	Method and Description
void	archiveMetaLog(ServerName serverName) The hbase:meta region may OPEN and CLOSE without issue on a server and then move elsewhere.
Set<ServerName>	getFailedServersFromLogFolders() Deprecated. With proc-v2, we can record the crash server with procedure store, so do not need to scan the wal directory to find out the splitting wal directory any more. Leave it here only because RecoverMetaProcedure(which is also deprecated) uses it.
org.apache.hadoop.fs.FileSystem	getFileSystem()
Set<ServerName>	getLiveServersFromWALDir() Get Servernames that COULD BE 'alive'; excludes those that have a '-splitting' suffix as these are already being split -- they cannot be 'alive'.
Set<ServerName>	getServerNamesFromWALDirPath(org.apache.hadoop.fs.PathFilter filter) Returns listing of ServerNames found by parsing WAL directory paths in FS.
Set<ServerName>	getSplittingServersFromWALDir() Get Servernames which are currently splitting; paths have a '-splitting' suffix.
org.apache.hadoop.fs.FileStatus[]	getWALDirPaths(org.apache.hadoop.fs.PathFilter filter) Returns List of all RegionServer WAL dirs; i.e.
void	splitLog(ServerName serverName)
void	splitLog(Set<ServerName> serverNames)
void	splitLog(Set<ServerName> serverNames, org.apache.hadoop.fs.PathFilter filter) This method is the base split method that splits WAL files matching a filter.
void	splitMetaLog(ServerName serverName) Specialized method to handle the splitting for meta WAL
void	splitMetaLog(Set<ServerName> serverNames) Specialized method to handle the splitting for meta WAL
void	stop()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NON_META_FILTER

public static final org.apache.hadoop.fs.PathFilter NON_META_FILTER

Filter *out* WAL files that are for the hbase:meta Region; i.e. return user-space WALs only.

Constructor Detail

MasterWalManager

public MasterWalManager(MasterServices services)
                 throws IOException

Throws:

IOException

MasterWalManager

public MasterWalManager(org.apache.hadoop.conf.Configuration conf,
                        org.apache.hadoop.fs.FileSystem fs,
                        MasterServices services)
                 throws IOException

Throws:

IOException

Method Detail

stop

public void stop()

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

getSplittingServersFromWALDir

public Set<ServerName> getSplittingServersFromWALDir()
                                              throws IOException

Get Servernames which are currently splitting; paths have a '-splitting' suffix.

Throws:

IOException

getLiveServersFromWALDir

public Set<ServerName> getLiveServersFromWALDir()
                                         throws IOException

Get Servernames that COULD BE 'alive'; excludes those that have a '-splitting' suffix as these are already being split -- they cannot be 'alive'.

Throws:

IOException

getServerNamesFromWALDirPath

public Set<ServerName> getServerNamesFromWALDirPath(org.apache.hadoop.fs.PathFilter filter)
                                             throws IOException

Returns listing of ServerNames found by parsing WAL directory paths in FS.

Throws:

IOException

getWALDirPaths

public org.apache.hadoop.fs.FileStatus[] getWALDirPaths(org.apache.hadoop.fs.PathFilter filter)
                                                 throws IOException

Returns List of all RegionServer WAL dirs; i.e. this.rootDir/HConstants.HREGION_LOGDIR_NAME.

Throws:

IOException

getFailedServersFromLogFolders

@Deprecated
public Set<ServerName> getFailedServersFromLogFolders()
                                                           throws IOException

Deprecated. With proc-v2, we can record the crash server with procedure store, so do not need to scan the wal directory to find out the splitting wal directory any more. Leave it here only because RecoverMetaProcedure(which is also deprecated) uses it.

Inspect the log directory to find dead servers which need recovery work

Returns:

A set of ServerNames which aren't running but still have WAL files left in file system

Throws:

IOException

splitLog

public void splitLog(ServerName serverName)
              throws IOException

Throws:

IOException

splitMetaLog

public void splitMetaLog(ServerName serverName)
                  throws IOException

Specialized method to handle the splitting for meta WAL

Parameters:

serverName - logs belonging to this server will be split

Throws:

IOException

splitMetaLog

public void splitMetaLog(Set<ServerName> serverNames)
                  throws IOException

Specialized method to handle the splitting for meta WAL

Parameters:

serverNames - logs belonging to these servers will be split

Throws:

IOException

splitLog

public void splitLog(Set<ServerName> serverNames)
              throws IOException

Throws:

IOException

splitLog

public void splitLog(Set<ServerName> serverNames,
                     org.apache.hadoop.fs.PathFilter filter)
              throws IOException

This method is the base split method that splits WAL files matching a filter. Callers should pass the appropriate filter for meta and non-meta WALs.

Parameters:

serverNames - logs belonging to these servers will be split; this will rename the log directory out from under a soft-failed server

Throws:

IOException

archiveMetaLog

public void archiveMetaLog(ServerName serverName)

The hbase:meta region may OPEN and CLOSE without issue on a server and then move elsewhere. On CLOSE, the WAL for the hbase:meta table may not be archived yet (The WAL is only needed if hbase:meta did not close cleanaly). Since meta region is no long on this server, the ServerCrashProcedure won't split these leftover hbase:meta WALs, just leaving them in the WAL splitting dir. If we try to delete the WAL splitting for the server, it fail since the dir is not totally empty. We can safely archive these hbase:meta log; then the WAL dir can be deleted.

Parameters:

serverName - the server to archive meta log